[e2e] UDP checksum field?

[Cannara]

I'll add a funny (if you're not using Oracle TNS gateways) SQL transport
example that still exists today, despite being pointed out to Oracle about a
decade ago.  When Network General was adding more SQL decodes to the
Sniffer(r), in the '90s, we had a presentation on the Oracle transport (TNS)
underlying SQL Net traffic.  TNS rode on Netware SPP, or TCP, etc.  The fellow
went into packet fields in detail and explained how Oracle also made gateway
software available for Sun boxes to go from an Oracle system to an IBM SNA db
system.  The gateway received SQL on TNS on TCP on IP on Ethernet (for
instance) and spit out SQL on TNS or whatever IBM wanted.  

As he expounded on TNS pkt fields, a few hands went up -- "What's the checksum
field for if it's always 0?" asked a few experienced network folks. The
presenter turned back to the slide show and said: "It's unimplemented for
now".  Without malice, another question was posed:  "Well if it's unused and
your gateway has bad memory, how do you know the data going into the db on the
other side will be good?"  The presenter, a highly lauded Oracle techy, looked
at the screen for a bit, looked back at the audience, shuffled his feet,
looked again at the screen, and finally said words like:  "I don't know".  

After the presentation, a letter was written to Oracle, copied to Ellison,
explaining exactly the problem and urging the TNS checksum be implemented.  No
response ever came back, and, if you look at a TNS packet today, the checksum
is still zero.  I guess no one has used the gateway software who cares about
their data.  :]

Alex

PS Note that "gateway" here is used in the proper sense, not for "router".

Lynne Jolitz wrote:
> 
> Yes, Lloyd is exactly right here. It is often the case that people turn off UDP checksums to "buy" more performance by relying on the CRC of the ethernet packet. It's not a stupid question - it's a very smart question, and a lot of smart people get fooled by this.
> 
> For example, the Sun datacenter back in the early 1990's had an NFS cluster project called Sunbox - an array of workstation CPUs that did divide and conquer to build a massive file server. It used an ethernet multiplexer to dynamically split the load. To buy back performance, they turned off the UDP checksum. It worked fine until they had a bad lot of ethernet boards with substandard memories - this wasn't picked up in tests because the test units were doing resends of the occasionally corrupted packets (UDP checksums usually was turned on), and in TCP the checksums would do resends as well. It was also a fairly rare problem, and the test periods were too short to pick up on the nature of this problem easily.
> 
> But when UDP checksums were turned off in normal use, the resulting NFS requests were corrupting the filesystem (which in this case were database files), forcing rebuilds and manual repairs of database tables.
> 
> As they were about to announce and release it, they suddenly discovered this problem - they noticed the corruption and in order to determine whether it was in the high level (stack or above) or lower levels, they turned on checksums and it worked immediately.
> 
> They then examined the failed checksum packets to traceback in the lower level stack-down through the link layer to discover where the corruption occured. With logic analyzers, they were able to observe the contents going into memory from the NIC on reception was different than the contents going out of the memory and traveling across the bus to the processor.
> 
> This is a surprisingly common problem in datacenters - sometimes the problem would be a switch, sometimes a configuration error, sometimes a programming error in the application, and so forth. I most recently experienced this problem with an overheated ethernet switch passing  video on an internal network.
> 
> I also ran into this at an Internet portal company where I was a manager. We were using NetApps file servers to mirror the daily information - NetApps at the time encouraged staff to turn off checksums to increase performance. The DBAs noticed problems and ended up doing frequent rebuilds, but couldn't figure out why. It took me a lot of time to convince my staff to turn on the checksums because they were told "they don't have to" by NetApps. Most datacenter staff work by cookbook, and this wasn't in the cookbook. When they finally tried it, it worked. This little problem cost us a lot of time and aggravation for very little (if any) performance gain.
> 
> Performance gain by turning off checksums now can be obviated through the use of intelligent NIC technologies like SiliconTCP (http://jolitz.telemuse.net/pubs/pt2001_01/item) and TOE that calculate the checksum as the packet is being received. But we don't have this in commodity switches yet, so check that switch if you're having problems.
> 
> Higher level checksums are worth it every time. Don't leave the server without them. :-)
> 
> Lynne Jolitz.
> 
> ----
> We use SpamQuiz.
> If your ISP didn't make the grade try http://lynne.telemuse.net
> 
> > -----Original Message-----
> > From: end2end-interest-bounces at postel.org
> > [mailto:end2end-interest-bounces at postel.org]On Behalf Of Lloyd Wood
> > Sent: Monday, April 04, 2005 2:48 AM
> > To: Faisal Aslam
> > Cc: end2end-interest at postel.org
> > Subject: Re: [e2e] UDP checksum field?
> >
> >
> > On Sun, 3 Apr 2005, Faisal Aslam wrote:
> >
> > > Why we have checksum field is in UDP header, as UDP does not provide
> > > data retransmission etc? I think it is used only to silently
> > > discarding a packet with wrong checksum (thats it?).
> >
> > yes - you need an end-to-end check against a corrupted packet. UDP
> > could have the checksum turned off, which proved disastrous for a
> > number of applications, subtly corrupted filing systems which didn't
> > have higher-level end2end checks etc.
> >
> > > Is there any  other application of checksum field?
> >
> > For other applications
> > http://www.faqs.org/rfcs/rfc3828.html
> >
> > UDP Lite originally sprang out of the observation that UDP has
> > redundant length information, and that this information could be
> > combined with the checksum (as in TCP/UDP) to give partial coverage.
> >
> > L.
> >
> > >
> > > Sorry if the question is too naive.
> > >
> > > Thanks
> > > Faisal