[e2e] TCP un-friendly congestion control

[Guglielmo Morandin]

Yes, I was assuming 1500 bytes MTU.
Increasing the MTU improves the situation because you can get the same 
average bandwidth
with less drops, but it does not allow you to go faster than what you 
can achieve with mtu 1500
and the same amount of buffering in the network.
If the cwnd is changing with the expected sawtooth pattern and you 
don't have a lot (significant percentage of the bw*delay
product) of buffers at the bottleneck router, you can still get only 
75% of the bottleneck rate.

You can achieve the bottleneck rate indefinitely if the bottleneck 
router has 12.5 MB of buffers and tail drop,
at the cost of having rtt between 100 and 200 ms.
In that case the window would oscillate, but the minimum value could be 
big enough
to sustain full rate.

With MTU 9K, you have 1380 packets per window at 1GBps and 100 ms.
So your window can oscillate between 690 and 1380 packets.
You need 690 rtts to go from 50% to 100% of the bottleneck rate,
and you need less than one drop every 690*1035 = 714000  packets, which 
is much better
than one every 24 million, as in the 1500 bytes mtu case.

Isn't it an unfair advantage over other tcp connections that are not 
using jumbo frames :-)?
Of course you did it also so that less computing power was required
to actually execute the protocol, assuming your limit is cpu, not 
memory bandwidth.

Do you have loss stats available for your experiments, possibly over 
time?

If you properly tune your window so that your maximum achievable 
bandwidth is just
smaller than the bottleneck bw, you can reach constant rate with a 
stock tcp,
no self-inflicted congestion drops and no additional delay.
Only when other traffic is really using part of your bandwidth
you will get drops, as opposed to letting the congestion window grow 
unlimited.
That implies a-priori knowledge of the bandwidth and delay of your 
network,  tough :-)
So my statement should have been

>> I doubt that 750Mbps can be sustained by standard tcp with 1500 bytes 
>> MTU in a real
>> network, unless available capacity is much bigger or tcp  window is 
>> limited to avoid
>> regular drops.

-Guglielmo

On Saturday, June 7, 2003, at 01:14 PM, Les Cottrell wrote:

> I agree with what you say, just one minor nit, on your statement about 
> TCP
> not being able to sustain 750Mbps, I suspect you should qualify that 
> with
> standard 1500Byte MTUs. With a large MTU TCP's AIMD recovery is 
> improved
> and it can perform much better, in fact the recent Internet2 land speed
> records were achieved (923Mbits/s sustained over 1Gbits/s bottleneck, 
> and
> 2.36Gbits/s over OC48/2.5Gbits/s bottleneck) with stock TCP and 9000 
> Byte
> frames.
>
> See for example:
>
> http://sravot.home.cern.ch/sravot/Networking/10GbE/10GbE_test.html or
> http://www-iepm.slac.stanford.edu/monitoring/bulk/10ge/
>
> Les Cottrell
> Mail Stop 97, Stanford Linear Accelerator Center, POB 4349, Stanford, 
> CA 94309
> Phone (650)926-2523, FAX (650)926-3329
> URL http://www.slac.stanford.edu/~cottrell/cottrell.html
>
> On Sat, 7 Jun 2003, Guglielmo Morandin wrote:
>
>> I doubt that 750Mbps can be sustained by standard tcp in a real
>> network, unless available capacity is much bigger.
>> But bandwidth is not THAT cheap.
>>
>> Thanks
>> -Guglielmo
>>
>>
>>