[e2e] Jon Crowcroft's thought questions...

[Dave Crocker]

Christian,

CH> One of the most successful Internet protocols, SMTP, is quite
CH> chatty. Much more chatty than, say, X.400. That did not prevent a
CH> large deployment, including in the narrow networks of 15 years ago.
CH> At some point, interoperability trumps performance.


SMTP experience provides one of the stronger examples that formed the
basis of my observations. Oddly, it is an example both of the problem of
chattiness and the benefits of efficiency.

Note that before SMTP we had a mechanism that was a simple file
transfer. In this sense it was like X.400's transfer mechanism. This
bulk transfer approach was pretty much the only aspect of X.400 that one
might call simple or efficient, thereby demonstrating that optimizing
one system characteristic is not sufficient, when there are other,
killing problems in a design.

The comment about 'trumping' supposes that there was any serious
competition between the two. There never was. X.400 was the clear winner
for roughly a decade, until folks noticed that it had almost no
installed base and that they could not use it.

As to the relative importance of different design attributes, I'd guess
that observing the importance of 'interoperability', in this case,
essentially masks the importance of 'installed base' and 'core technical
simplicity'.

SMTP's chattiness is due to its permitting multiple addressees. This
capability replaced the one addressee-per-posting of its predecessor.
So, relative to the total scenario for sending multi-address email prior
to SMTP, SMTP permitted an enormous improvement, both reducing total
cross-net exchanges and total bytes sent.

The reason for doing per-address responses, rather than a simple, bulk
transfer of both content and envelope (ie, addressees) was to cover the
case of having no valid addressees.  This would then permit the sender
to *not* send the content.  And this was viewed as a big savings in
network bandwidth and receiving server processing.

At that time, invalid addresses were pretty common, so the desire for
address verification before sending was important.

However the chattiness has been a serious problem.  And concern about it
has dictated some very significant changes in the use of SMTP over the
the years.

The ESMTP PIPELINE option dramatically reduces the protocol's logical
chattiness. By "logical" I mean that it changes where the sender must
actually wait for a response before proceeding. Pipelining allows
overlapping requests and responses. Hence, the protocol still has the
physical request/response "chunks" it had before, but the lock-step
client/server synchronization places are reduced substantially.

Even more telling is that the client/server means of agreeing to use
ESMTP (SMTP with options) rather than the non-negotiating, original
SMTP, is achieved with *no* additional round-trip exchanges. This was a
hard-fought point *specifically* due to experience with the problem of
cross-net round-trips.

This cross-net performance concern is also is the reason that server
implementations have moved towards having almost no real-time processing
of the addressee information, lest the delay in processing block the
sender. (Timeouts by clients, while receiving servers expanded addresses
that were aliases for a list, used to be a pretty serious problem.)

The general principle, here, is essentially a batch versus interactive
trade-off.  In high performance environments, real interactivity is
possible.  In low performance environments, any significant amount of
interactivity kills performance and scaling and, hence, utility.

So, in a "clean", high-bandwidth, low-latency environment, people tend
to design protocols that are like remote procedure call. The chattiness
-- or, rather, the client/server fine-grained synchrony -- is viable. In
noisy, low-bandwidth, high latency environments, those do not work.
Hence we must resort to much more asynchronous, batch styles of
communication. And that's why the Delay Tolerant Networking effort has
moved to a packet communication model that looks so much like email.

d/
--
 Dave Crocker <mailto:dcrocker at brandenburg.com>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>