[Tsvwg] Re: [e2e] Are you interested in TOEs and related issues

[H.K. Jerry Chu]

>> Bursty traffic may be ok for LAN, but is considered evil for WAN.
>> TOE doesn't suffer this problem (since it is stateful), and may be
>> a necessary evil until jumbo frames become universal.
>
>actually the interesting observation that this discussion raises
>is that the frame size/link MTU should be picked large enough to make the
>end-to-end pps rate bounded (but not too low) for well behaved flows.
>
>In 1980 the ethernet MTU allowed some 800pps -- going much higher makes
>little sense anyways, because even if the end nodes can cope with
>higher rates, the control loops cannot not react fast enough
>(you have to factor in propagation delays too).
>
>On the negative side, however, is that that very large MTUs require
>a lot of buffering to be preallocated on the receive side, and
>possibly even at the routers (at least, those without hw-assisted
>forwarding planes).

It's mainly the TCP receive window, not the PMTU that dictates how much
buffering will be needed. Routers that do cut-through need not
buffer the whole pkt. More significant is the drop of the DRAM price
and increase of the DRAM size making memory a non-issue in many cases
today.

Over the past decade many components involved in providing high-speed
networking have scaled up an order of magnitude. This including link
bandwidth, CPU speed, I/O bus, memory size..., but not the Ethernet MTU
and certain TCP parameters (such as the every-other-pkt acking policy).
This is really hurting the throughput performance of the hosts. IMHO the
amount of burstiness by TCP over WAN should be allowed to scale up an
order of magnitude too. If stretch ACKs are fully adopted into TCP
algorithm (see rfc2525 for a number of issues with stretch acks), one
can use LSO on the transmit side, and per-flow pkt coalescing on the
receive side to provide effectively a simple, stateless AAL5 layer for
the Ethernet "cells" without requiring jumbo frames or complex TOE engine.

BTW, I asked a few transport folks in Minneapolis IETF about how "evil"
is traffic burst in today's enviroment, but did not get any concrete
answer. Perhaps this topic should be discussed in tsvwg or tcpm.

Jerry

Sr. Staff Engineer
Solaris Networking & Security Technology
Sun Microsystems, Inc.

>
>I don't have a good answer but going much higher than 16Kbytes MTUs
>seems unlikely... and at 10Gig this is still close to 100kpps.
>
>	cheers
>	luigi