You might find that increasing the TCP receive buffer on the receiving side will help.
This got me wondering. I know Oracle requires minimums for some kernel settings like net.core.rmem_default and rmem_max and similar for wmem, ie:
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
Looking at my primary and standby, they look to still be the CentOS defaults for CentOS7 and CentOS6.
Primary (CentOS7)
net.core.rmem_default = 212992
net.core.rmem_max = 212992
net.ipv4.tcp_rmem = 4096 87380 6291456
net.core.wmem_default = 212992
net.core.wmem_max = 212992
net.ipv4.tcp_wmem = 4096 16384 4194304
Standby (CentOS6)
net.core.rmem_default = 124928
net.core.rmem_max = 124928
net.ipv4.tcp_rmem = 4096 87380 4194304
net.core.wmem_default = 124928
net.core.wmem_max = 124928
net.ipv4.tcp_wmem = 4096 16384 4194304
FWIW: these machines are VMWare with 8 cores and over 100GB of memory. I'm assuming we have gigabit ethernet within the datacenter but the circuit between the two is 200 Mbps.
