[e2e] Unexpected small reduction in cwnd during iperf test

[Xiaoliang (David) Wei]

Hi Toby,

> Can anyone think in what circumstances a TCP instance might reduce cwnd
> by a small amount, without there being any change in ssthresh?  I
> detected this with a script that periodically (every 0.25 seconds)
> collects TCP stats.  The TCP transfer in question was an iperf test
> running between a measurement point in a New York PoP and
> another in Budapest.  netstat (run before and after the test) confirms
> that there was no packet loss, but there were two re-transmitted

I wonder how netstat could tell if there is packet loss? You mean "no packet 
loss at the local NIC?"

> Time (s)
> 15.50 cwnd:5931 ssthresh:4740
> 15.75 cwnd:5931 ssthresh:4740
> 16.00 cwnd:5931 ssthresh:4740
> ...
> 23.75 cwnd:5931 ssthresh:4740
> 24.00 cwnd:5926 ssthresh:4740 <==GLITCH?

as the cwnd before this glitch has been kept to 5931 unchanged for several 
RTTs (I assume the RTT in your case is in the order of 100ms across the 
Atlantic), I guess one possibility is that there is some ack packet 
reordering.

In this case, Linux TCP enters Disorder state. Since ack is not in order and 
no SACK (since only ack is disordered), the sender cannot send any new 
packets and the actually number of packets in flight keep decreasing. When 
the inorder ack comes back finally, the number of packets in flight is 
several packets smaller than congestion window. Upon the arrival of this 
in-order ack comes, sender exists Disorder state and the tcp_moderate_cwnd 
is called, reducing the congestion window to be # of packets in flight plus 
3...

just one explanation... not sure if it is of your case. More measurements on 
the TCP congestion avoidance state is helpful, I think. Or, if the 
experiment is repeatable and you could have the tcpdump trace during this 
events, it is also helpful.

> 24.25 cwnd:5927 ssthresh:4740
> 24.50 cwnd:5928 ssthresh:4740
> 24.75 cwnd:5929 ssthresh:4740
> 25.00 cwnd:5930 ssthresh:4740
> 25.25 cwnd:5931 ssthresh:4740
> 25.50 cwnd:5931 ssthresh:4740
>
> The hosts are both Linux kernel's 2.6.13, using BIC congestion control.
...


-David
---------------------------------------------------------
Xiaoliang (David) Wei
http://davidwei.org    Graduate Student, Netlab, Caltech
======================================