[e2e] Switched Ethernet is Not an End-to-End System; was Protocols breaking the end-to-end argument
Detlef Bosau
detlef.bosau at web.de
Fri Nov 13 06:42:25 PST 2009
Hi Richard,
to be honest: I thought a bit about answering to this post. However, I
well remember my first posts to this list some years ago and I well
remember some posts of people, who where both, new to this list and new
to distributed systems. So, people can learn from posts to this list,
myself explicitly included.
So, I think it to be worthwhile to add some comments.
Richard Bennett wrote:
> Detlef, I'm asking you to think abstractly about certain problems in
> the design of distributed systems. I regard many of
That's exactly what I'm doing. I simply does not matter whether a packed
is conveyed via a shared Ethernet or via a switched Ethernet. From an
abstract point of view, the packet is conveyed from a sender to a
receiver. And this process takes some time and the "server", i.e. the
conveying network, may process quite some packets in parallel. (I
recently talked about TCP multipath issues to some colleagues, and I got
the insight, perhaps many others got this long ago, that it simply does
not matter, whether one packet is conveyed via a Cheapernet segment
while a second one is conveyed via a fibre link and a third one via
WiFi. There are three packets on the fly.
Now: The endpoints are not interested in the details of transport here -
and in addition: there's no need for them to give it even a thought!
(O.k., you will object that the service times on different media may be
different. Granted. Leave it out for the moment.)
Do you see, what I'm doing here? I make an _abstraction_. I regard the
net as some kind of transportation pipeline with some average (or
expected) service time and a certain capacity. And from this point of
view, I well may ignore the details.
> the interpretations and applications of end-to-end notions to network
> management and regulation as flawed,
O.k., you're welcome to mention some concrete examples. But of course,
when I mention a concrete flaw, I must not only say: "This is wrong, and
I'm old enough and big enough and ugly enough to be correct here!" but I
have to give a compelling reason, why something is wrong.
> regardless of the intent of any of the authors of any particular paper
> decades ago, or even to their subsequent refinement by the original
> authors or others.
In fact, I'm not interested in the intention but in the results.
>
> One the problems that arises in distributed systems is access to
> shared resources, and the end-to-end solution to this problem is
> deeply flawed.
Why and where?
> In order to share resources according to a policy, there needs to be a
> system element enforcing the policy.
Exactly. (Certainly, you know Keshav's PhD dissertation.)
However, and I'm convinced Keshav will agree here: There are systems
_without_ such a system. And there are systems, where the amount of
resources is unknown. Think not only of any kinds of ad hoc systems and
spontaneous collaboration.
Think of an arbitrary "Linux installation party" - where some PC are
interconnected via a coax-"backbone".
Do you measure the coax backbone's length in advance to calculate its
storage capacity for packets?
Certainly not.
You simply use the system, and - surprise! - it works!
> In network operations, this function is indispensable and must be part
> of the network infrastructure. There are many
I've just given you and example, where this indispensable function can
well be spared.
> reasons for this, of course, but the chief ones are a) the lack of
> reliability and trust in end systems;
The less components a system will need, the more reliable a system will
be. (This basic insight is often referet to / abbreviated as KISS. Keep
It Small and Simple.)
> and b) the lack of information about system demand on the part of the
> end system, which in principle only knows that it wants and not what
> anyone else wants to do on the shared resource.
>
And particularly, an Ethernet switch in between has no knowledge about
the end system's needs.
> The idea that access to shared resources is best accomplished by
> uncoordinated end systems leads to a lot of grief.
>
One part of the grief is that you can read my answer ;-)
> What does this have to do with Jacobson's Algorithm? A lot, it turns
> out, as we see in the rise of systems that take the job of congestion
> mediation a couple of steps further, such as ECN and Re-ECN.
Which rise of system are you talking about? I definitely see a rise of
papers which propose design rules for ECN, showing that we did not find
compelling ones throughout the last decade.
However, without VJCC, this seminal discussion of ECN would not even be
possible.
> If you recognize that information about the state of shared resources
> is essential to their rational management, no problem.
>
I well recognize this. However, I recognize as well, that VJCC does its
job. And the algorithm is shamelessly simple.
> Of course, all of this is simply to say that optimal solutions to
> problems of resource management can only be made close to the resource
> in question,
Which is known from the works by Srinivasan Keshav and Frank Kelly and
many others.
However, it's a simple fact in practical engineering, that we do not
obtain optimal solutions for all problems, particularly the
aforementioned works run into some difficulties when applied to wireless
systems, but we obtain _practical_ solutions, we can well live with which.
Actually, we are that comfortable with these practical solutions, that
the only purpose of some of the optimal ones is to protect the bald
heads of PhD students from the cold by providing them warming hats.
And, of course, these practical solutions provide you with a punch-ball
as well ;-)
> but it doesn't address a later version of the end-to-end arguments
> principle, that sub-optimal solutions (from an efficiency of
> fair-sharing perspective) to such problems may be preferred if they
> lead to greater system generality,
Wow!
You're willing to accept sub-optimal solutions when they lead to greater
system generality!
Welcome to the club!
I think, about all of us do.
> opportunities for innovation, free speech, reliability, or some other
> reason.
>
> I hear this sort of argument being made very frequently these days,
> and it does have some resonance. Efficiency is
At least, it shows some effect eventually...
> after all a short-term goal, while generality is a long-term goal and
> innovation is a positive externality. But to appreciate that, one
> needs to think a bit abstractly.
>
...which may mean that we should rather not bother with irrelevant
problems of minor importance.
(@Joe: Hopefully, I was not too misbehaved here, but I really could not
resist...)
Detlef
--
Detlef Bosau Galileistraße 30 70565 Stuttgart
phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau
ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de
More information about the end2end-interest
mailing list