[Tsvwg] Re: [e2e] Are you interested in TOEs and related issues

[H.K. Jerry Chu]

>
><SNIP>
...
>> >The gains are by optimizing data movement and not by
>> >    offloading protocol processing.
>> 
>> LSO is a 'stateless' offload (much like checksum offload) so one doesn't
>> need a TOE to support it; 
>
>It depends. If the kernel does a write based on open window, then yes
>you are right. But if you have full TOE and you pass large chunks of
>data to TOE (does wonder when sendfile/sendfilev is in play) and let
>TOE figure out when to send it, the gains increase. In the later case,
>TOE needs teh TCP state.
>
>> most modern NICs (including our 10GbE Adapter) do LSO. Also, for normal
>> frames LSO has quite dramatic effect at 10GbE rates, but for Jumbo
>> frames performance gain diminishes to ~7%.

One fundamental performance problem high speed Ethernet imposes on
the hosts is from the tiny size of the standard Ethernet frames.
It's almost like throwing ATM cells at the host 10 years ago but
without any hardware adaptation.

LSO and ack coalescing are the adpatation layer for Ethernet cells.
They can help to alleviate this problem but unfortunately they either
induce, or live on traffic bursts. (Just imagine how useful LSO will
be if the ack-every-other-pkt policy provides only 2-MSS worth of
usable window, and how much back-to-back bursts will be caused by
coaliscing many acks.)

Bursty traffic may be ok for LAN, but is considered evil for WAN.
TOE doesn't suffer this problem (since it is stateful), and may be
a necessary evil until jumbo frames become universal.

Jerry

>
>Do you have any insight on why you would see a degradation with Jumbo
>frames? As long as you are stateless, the jumbograms should show
>no degradation. 
>
>Thanks,
>Sunay
>
>
>> 
>> 
>> > 5) TOE is necessary for RDMA, iSCSI etc. for layering 
>> > reasons. I am not
>> >    involved with RDMA so someone who is an expert can 
>> > probably comment on
>> >    this part.
>> > 6) TOE based NIC are already making pretty good headway in 
>> > embedded space.
>> >    The technology is already maturing so why not use it in 
>> > broader market.
>> > 
>> > Note that the above claims are in no particular order of 
>> > importance and made my TOE vendors in general. Of these, I 
>> > personally do agree with 1 and 4 but that iteself doesn't 
>> > mean that TOE will make it in general purpose networking.
>> > 
>> > It would be interesting to see if you and others in the list 
>> > agree or disagree with these claims.
>> > 
>> > Thanks,
>> > Sunay
>> > 
>> > -- 
>> > 
>> > Sunay Tripathi
>> > Senior Staff Engineer,
>> > Solaris Kernel Networking,
>> > Sun MicroSystems Inc.
>> > 
>> > email: sunay at eng.sun.com		 Phone:	650-786-6007 (W)
>> > 
>> > 
>> > 
>> > 
>> 
>> 
>> Regards, 
>> Leonid Grossman
>> www.s2io.com
>> 
>> 
>> > _______________________________________________
>> > tsvwg mailing list
>> > tsvwg at ietf.org
>> > https://www1.ietf.org/mailman/listinfo/tsvwg
>> > 
>> 
>
>
>-- 
>Sunay Tripathi
>Solaris Kernel Networking,
>Sun MicroSystems Inc.
>
>email: sunay at eng.sun.com		 Phone:	650-786-6007 (W)
>
>
>