[e2e] Clarifying the End-to-End Principle

[Shivkumar Kalyanaraman]

Dear folks,

With Blumenthal/Clark's paper (ACM Trans. on Internet Tech, Aug 2001, pp.
70-109), it is clear the end-to-end principle needs some
arguments, widening of scope, clarifications, and/or new principles need
to be added to the mix.

This note offers some thoughts to stir up a conversation of how to better
interpret the e2e principle as it stands today. Given the importance of
such guiding principles for the continued rapid pace of networking
innovations, I think some improved clarity is warranted.

Cheers
Shiv
================================================================
The E2E design principle states...

"...functions placed at the lower levels may be redundant or of little
value when compared to the cost of providing them at the lower
level..."

The communication version of the e2e argument (for the reliability example):

"The function in question can completely and correctly be implemented
only with the knowledge and help of the application standing at the
endpoints of the communications system. Therefore, providing that
questioned function as a feature of the communications systems itself
is not possible. (Sometimes an incomplete version of the function
provided by the communication system may be useful as a performance
enhancement)..."

In brief, the end-to-end argument emphasizes:
 function placement
 correctness
 completeness and
 overall system costs (allows a cost-performance tradeoff).

In other words, a system (or subsystem level) should consider only
functions that can be completely and correctly implemented within
it. All other functions are best moved to the system level where it
can be completely and correctly implemented. A placement of function
inside the network may not be needed by some applications, or may
even conflict with their needs.

--------------------------------------------------------------
Issues with the E2E principle:

1. The notion of the boundaries of a system, or its breakup into
  "levels" are not fully specified. Also, the communication version of
  the principle was given in the context of reliable file transfer,
  and therefore the e2e principle is not often cited in the design of
  purely network-layer mechanisms like routing etc

2. If the function specification includes a notion of performance, then
  e2e principle is silent on how to handle this case because network
  support may be required (eg: QoS requirements).

3. The e2e principle is silent on technology, economic and third-party
  issues (some of this raised in considerable detail in the
  Blumenthal/Clark paper). For example, did the circuit-switched system in
  early 1900s conform to the e2e argument? What if the placement of
  functions e2e is more expensive than placing it in the network due to
  technological/economic issues ?

--------------------------------------------------------------------


Issue #1: Definition of System and its Levels
---------------------------------------------

One way to address issue #1 is to require system designers to define
clear *boundaries* for system design. Eg: a single box design, a single
domain network design, a single ISP or locus of cooperating ISP
networks, the entire Internet etc...

Moreover, the *levels* of the system and their *ordering* need explicit
specification. In particular, a level is not the same as a protocol
layer. For eg: diff-serv specified edge- and interior-nodes as
different levels (both at layer 3), and the edge- is a higher level
compared to the interior. This justifies a disparate placement of
functions at the two levels, and a movement of functions to the higher
level.

Another example is that of routing. How do we apply the e2e principle
to design of routing? The communication version of
the e2e principle was given in the context of reliable file transfer,
and therefore it is not often cited in the design of
purely network-layer mechanisms like routing. Also, a good design in
conformance with the e2e principle but which does not deal with end-hosts
of the Internet may be dismissed as not in conformance with Internet design
principles...

One common misinterpretation is to believe that end-to-end
means move functions "to end-systems" (eg: expose full routing tables
to end-hosts) or "to transport layers or higher" (eg: Akamai's
content-delivery network).

Reg routing, the E2E principle can certainly be applied. In a flat
routing domain, routing is a network layer function and there is no
further set of levels. So the same set of functions (eg: as in RIP,
OSPF) is placed in all nodes. Even here, the use of default routing
allows different routers to choose varying degree of state to be
maintained provided a "core" -- i.e., some highest level of the system
maintains default free routes. However, in a hierarchical system, two
different "systems" (not "levels") are defined. In particular, the
inter-domain function cannot be moved to intra-domain routers and
therefore they cannot be thought of as two levels...

This lack of clarity extends to networking textbooks. I believe that
networking textbooks should emphasize the application of E2E principle
with more examples than just the perfunctory reliability example, and
re-visit its applicability when discussing particular topics.


Issue #2: E2E Principle and Performance Requirements
----------------------------------------------------
The e2e principle talks about function placement in the context of
correctness, completeness and allows a cost-performance
tradeoff to deal with overall system costs.  However, if the function
specification includes a notion of performance (eg: a leased-line
emulation, i.e., a guarantee of bandwidth, delay, jitter and packet
loss), then  e2e principle is silent on how to handle this case. It is
well known that the above QoS issues boil down to service isolation
and service differentiation sub-functions, which can be implemented
inside the network using components like shapers, schedulers, buffer
management etc.

Now, the performance requirement may be very stringent (eg: give me an
equivalent of a telephone circuit) or less stringent (eg: give me
priority service). If the specification is less stringent, then the
network support can be minimized (eg: diff-serv, CSFQ, weighted AIMD
etc). Else, for tighter performance guarantees (eg: delay/jitter
guarantees), one needs to place more complex functions in the network,
because it is impossible to provide isolation/differentiation sub-functions
end-to-end in such small time-scales. In such a case, have we violated
the end-to-end principle ?

I believe the e2e principle may be modified to state that if the
"correctness" of the service (eg: in this case, meeting the
performance requirement) also requires placement of sub-functions at
"lower levels,"  then that placement is allowed. However, this should
be accompanied with an argument why the sub-function cannot be placed
at higher levels (or should be revisited when newer implementations
(eg: CSFQ in QoS world) of the sub-functions allow their placement in
higher levels).

A sub-function implementation may involve both
state-complexity (eg: stateless vs per-flow state) and computation
complexity. CSFQ is an example where the state-complexity is minimized,
but some forwarding-plane computation is required.
Moreover, the placement of sub-functions in the data-plane could be
de-coupled from placement of sub-functions in the control-plane. For
example, in the case of QoS, a trust-boundary may be drawn at the
provider network edge which defines a sub-system, and within this
sub-system, the e2e principle argues for placing all possible
functions at the edge, i.e., the end-points within the trust
boundary. Now, data-plane functions like scheduling,
buffer-management, AQM etc could be placed at network edges, and
either a hop-by-hop signaling or server-based provisioning system
(control-plane) could be designed to manage QoS allocation. The IETF
has recognized this distinction when it decoupled the diff-serv
evolution from that of RSVP, and allowed alternate provisioning
protocols like COPS and SNMP.

Issue 3: Technology, Economic and Third-Party Issues:
--------------------------------------------------------

This is a pandora's box, as revealed by Blumenthal/Clark's paper, and
no clear "principles" seem to be suggested by them or others at this
time.

I dont have much to add to their paper, but would point out that the
end-to-end argument may be inapplicable in its current form if the
technology to implement functions at end-systems is non-existent or
probitively expensive. The E2E principle implicitly assumes that functions
are cheaper to implement (in terms of total economic or opportunity costs)
at higher levels compared to implementing them all at lower levels.

For example, did the circuit-switched system in early 1900s conform to
the e2e argument?  If the end-to-end principle is qualified to include
a compromise between available technology, cost and performance, then
we can say that in the early 1900s circuit-switched transmission
system may have conformed to the end-to-end principle. This is because
placing intelligence at the end points would not have been feasible since
it was very costly and nearly impossible technologically. Placing this
intelligence at the network level, and tying the intelligence to a
single application (voice) made a reasonable cost-performance
compromise *at the time*. But as the technology evolved, the entrenched
functionality in the network did not move out to higher-levels, thus
becoming increasingly inconsistent with the end-to-end principle.

A more recent example is that of NAT, which is clearly breaking the e2e
argument, if we maintain the notion that a public address is the address
of a physical network interface. If the Internet were designed with an
alternative notion that a public address is a "virtual network interface
IDs" (forseeing an address space shortage), then applications may
not have tied their semantics to physical network interface IDs. This
transforms into an argument between global virtual and physical interface
IDs... The E2E principle may argue for the latter, but the arguments are
less clear given the range of arguments in addressing (eg: GSE for
renumbering, IPv6 for autoconfiguration etc)...

In summary, these examples illustrate that it is unclear how the e2e
principle should be applied in several cases. As I mention above, I
believe that it is more broadly applicable than it directly implies, and
look forward to more examples from folks who have grappled with this
issue.

best
-Shiv
===
Shivkumar Kalyanaraman
Associate Professor, Dept of ECSE, Rensselaer Polytechnic Institute (RPI)
WWW: http://www.ecse.rpi.edu/Homepages/shivkuma