[e2e] TCP un-friendly congestion control

[Les Cottrell]

Despite what you say about getting the max throughput with the optinaml
window with single stream stock TCP with 1500byte MTU we were unable to
get close to the bandwidth capacity (see the 2nd figure in
http://www-iepm.slac.stanford.edu/monitoring/bulk/fast/)

I regret we do not have loss statistics. We do have some traces taht
still need to be analyzed. Since the link was
lightly loaded most of the losses came from our own over-running slow
start and also
continuing to do additive increase after reaching the capacity of the link
and thus inducing congestion and loss after the buffers filled.

Again a bit of a nit but doesn't a larger MTU give slightly larger
throughput since the slope of recovery is faster, there are more sawtooths
in the same time interval and thus more flat
tops at 1Gbps as the offered traffic >=
traffic sendable and the router queue fills up before tail drop kicks in
and one get a loss which halves the congestion window. See for example
http://www-iepm.slac.stanford.edu/monitoring/bulk/fast/iperf-snv1-gva4-20030131.gif
from http://www-iepm.slac.stanford.edu/monitoring/bulk/fast/


Les Cottrell
Mail Stop 97, Stanford Linear Accelerator Center, POB 4349, Stanford, CA 94309
Phone (650)926-2523, FAX (650)926-3329
URL http://www.slac.stanford.edu/~cottrell/cottrell.html

On Sat, 7 Jun 2003, Guglielmo Morandin wrote:

> Yes, I was assuming 1500 bytes MTU.
> Increasing the MTU improves the situation because you can get the same
> average bandwidth
> with less drops, but it does not allow you to go faster than what you
> can achieve with mtu 1500
> and the same amount of buffering in the network.
> If the cwnd is changing with the expected sawtooth pattern and you
> don't have a lot (significant percentage of the bw*delay
> product) of buffers at the bottleneck router, you can still get only
> 75% of the bottleneck rate.
>
> You can achieve the bottleneck rate indefinitely if the bottleneck
> router has 12.5 MB of buffers and tail drop,
> at the cost of having rtt between 100 and 200 ms.
> In that case the window would oscillate, but the minimum value could be
> big enough
> to sustain full rate.
>
> With MTU 9K, you have 1380 packets per window at 1GBps and 100 ms.
> So your window can oscillate between 690 and 1380 packets.
> You need 690 rtts to go from 50% to 100% of the bottleneck rate,
> and you need less than one drop every 690*1035 = 714000  packets, which
> is much better
> than one every 24 million, as in the 1500 bytes mtu case.
>
> Isn't it an unfair advantage over other tcp connections that are not
> using jumbo frames :-)?
> Of course you did it also so that less computing power was required
> to actually execute the protocol, assuming your limit is cpu, not
> memory bandwidth.
>
> Do you have loss stats available for your experiments, possibly over
> time?
>
> If you properly tune your window so that your maximum achievable
> bandwidth is just
> smaller than the bottleneck bw, you can reach constant rate with a
> stock tcp,
> no self-inflicted congestion drops and no additional delay.
> Only when other traffic is really using part of your bandwidth
> you will get drops, as opposed to letting the congestion window grow
> unlimited.
> That implies a-priori knowledge of the bandwidth and delay of your
> network,  tough :-)
> So my statement should have been
>
> >> I doubt that 750Mbps can be sustained by standard tcp with 1500 bytes
> >> MTU in a real
> >> network, unless available capacity is much bigger or tcp  window is
> >> limited to avoid
> >> regular drops.
>
> -Guglielmo
>
> On Saturday, June 7, 2003, at 01:14 PM, Les Cottrell wrote:
>
> > I agree with what you say, just one minor nit, on your statement about
> > TCP
> > not being able to sustain 750Mbps, I suspect you should qualify that
> > with
> > standard 1500Byte MTUs. With a large MTU TCP's AIMD recovery is
> > improved
> > and it can perform much better, in fact the recent Internet2 land speed
> > records were achieved (923Mbits/s sustained over 1Gbits/s bottleneck,
> > and
> > 2.36Gbits/s over OC48/2.5Gbits/s bottleneck) with stock TCP and 9000
> > Byte
> > frames.
> >
> > See for example:
> >
> > http://sravot.home.cern.ch/sravot/Networking/10GbE/10GbE_test.html or
> > http://www-iepm.slac.stanford.edu/monitoring/bulk/10ge/
> >
> > Les Cottrell
> > Mail Stop 97, Stanford Linear Accelerator Center, POB 4349, Stanford,
> > CA 94309
> > Phone (650)926-2523, FAX (650)926-3329
> > URL http://www.slac.stanford.edu/~cottrell/cottrell.html
> >
> > On Sat, 7 Jun 2003, Guglielmo Morandin wrote:
> >
> >> I doubt that 750Mbps can be sustained by standard tcp in a real
> >> network, unless available capacity is much bigger.
> >> But bandwidth is not THAT cheap.
> >>
> >> Thanks
> >> -Guglielmo
> >>
> >>
> >>
>