Categories
Generally

Stalled connections using SCP/SFTP/SSH with large files through VPN connections

Sometimes you have to push data using SCP/SFTP/SSH via a VPN connection on a customer server. Especially if Cisco VPN hardware is being used and the files are large, the transfer sometimes is stalled and fails even. This also happens with other hardware as well and may be because of a poor implementation of SACK. It might help to switch SACK of in the kernel network stack.

The Problem

The TCP SACK or Selective Acknowledgment packets are the new way to tell a client which packets already been received. SACK tells something like “Ok I already got packet a to m” where as ACK still tells the client “Ok I got a” .. “Ok I got b” .. “Ok I got c”.

Due to poor implementations on many network devices the SACK packets aren’t always handled correctly. If the client doesn’t receive any acknowledgments it the stops the transfer. That’s when the connection becomes stalled.

Possible solution

On Linux it is easy to disable SACK to enable ACK again. There is little to no downside to that. The maximum network output will drop a bit. But that’s about it. Also SACK may be vulnerable to DOS attacks where a small amount of packets may cause the client to sent lots of data.

First try to disable SACK temporarily.

echo 0 > /proc/sys/net/ipv4/tcp_sack

If the transfer is now no longer stalled, you can make this setting permanent. Just add the following line at the end of /etc/sysctl.conf

# Disable sack
/proc/sys/net/ipv4/tcp_sack = 0

In case you are using ipv6 just replace ipv4 with ipv6.

Links

SACK: TCP Selective Acknowledgments (SACK)
by Jeremy Stretch

When to turn TCP SACK off?
via serverfault

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.