[Tsvwg] Re: [e2e] Are you interested in TOEs and related issues

[Marko Zec]

On Friday 05 March 2004 19:50, H.K. Jerry Chu wrote:
> >> Bursty traffic may be ok for LAN, but is considered evil for WAN.
> >> TOE doesn't suffer this problem (since it is stateful), and may be
> >> a necessary evil until jumbo frames become universal.
> >
> >actually the interesting observation that this discussion raises
> >is that the frame size/link MTU should be picked large enough to
> > make the end-to-end pps rate bounded (but not too low) for well
> > behaved flows.
> >
> >In 1980 the ethernet MTU allowed some 800pps -- going much higher
> > makes little sense anyways, because even if the end nodes can cope
> > with higher rates, the control loops cannot not react fast enough
> > (you have to factor in propagation delays too).
> >
> >On the negative side, however, is that that very large MTUs require
> >a lot of buffering to be preallocated on the receive side, and
> >possibly even at the routers (at least, those without hw-assisted
> >forwarding planes).
>
> It's mainly the TCP receive window, not the PMTU that dictates how
> much buffering will be needed. Routers that do cut-through need not
> buffer the whole pkt. More significant is the drop of the DRAM price
> and increase of the DRAM size making memory a non-issue in many cases
> today.


If the DRAM price alone were the only factor preventing vendors from 
implementing unlimited packet queues, we would already have seen tons 
of such devices on the market over the past 10 years or so. Excessive 
queuing delays on routers are bad, especially for high-speed traffic 
managed by TCP-like congestion control schemes. The queue lengths on 
routers have always been and will always present a tradeoff between a 
smallest usable window for handling traffic bursts, and the desire of 
keeping the queueing delays as low as possible on the other hand.


>
> Over the past decade many components involved in providing high-speed
> networking have scaled up an order of magnitude. This including link
> bandwidth, CPU speed, I/O bus, memory size..., but not the Ethernet
> MTU and certain TCP parameters (such as the every-other-pkt acking
> policy). This is really hurting the throughput performance of the
> hosts. IMHO the amount of burstiness by TCP over WAN should be
> allowed to scale up an order of magnitude too. If stretch ACKs are
> fully adopted into TCP algorithm (see rfc2525 for a number of issues
> with stretch acks), one can use LSO on the transmit side, and
> per-flow pkt coalescing on the receive side to provide effectively a
> simple, stateless AAL5 layer for the Ethernet "cells" without
> requiring jumbo frames or complex TOE engine.
>
> BTW, I asked a few transport folks in Minneapolis IETF about how
> "evil" is traffic burst in today's enviroment, but did not get any
> concrete answer. Perhaps this topic should be discussed in tsvwg or
> tcpm.


Because queues in todays routers have finite maximum lengths, and this 
model is unlikely to change in the forseeable future, excessive traffic 
bursts will be more likely subject to drop-tail policing than other 
kinds of more smoothly shaped traffic. More than that, the bursty 
traffic will not only have less chance of reaching its target with all 
fragments in place, but it will also most probably do much harm to 
other innocent and otherwise well-behaving flows as well.

Marko


>
> Jerry
>
> Sr. Staff Engineer
> Solaris Networking & Security Technology
> Sun Microsystems, Inc.
>
> >I don't have a good answer but going much higher than 16Kbytes MTUs
> >seems unlikely... and at 10Gig this is still close to 100kpps.
> >
> >	cheers
> >	luigi