[e2e] opening multiple TCP connections getting popular

Thu Aug 30 10:59:17 PDT 2007

Joe,

At 16:57 30/08/2007, Joe Touch wrote:

>Lachlan Andrew wrote:
> > Greetings Joe,
> >
> > On 30/08/2007, Joe Touch <touch at isi.edu> wrote:
> >> Lachlan Andrew wrote:
> >>> I thought that Bob's suggestion was simply that the transport/network
> >>> layer provide a mechanism for charging based on the congestion caused.
> >>>   That sort of mechanism *is* "solvable at the transport layer".
> >> If we charge based on congestion caused, that's a network layer issue,
> >> not transport.
> >
> > True, the charging is network layer, but letting the application know
> > how much it is being charged so that it can change its rate is
> > transport layer, isn't it?
>
>It's transport layer if you're charging the app based on what the app
>sends to the transport. Unfortunately, the network can be just as
>congested by a retransmission as a first try, so this is network
>pricing. Although the transport and application may _react_ to it, it is
>generated (and thus the problem exists) at the network layer.

Here's where the transport layer comes in: Policing. Even tho it's "in the 
network", policing is a transport layer function.

We have proposed a bulk per-user policer based on measuring downstream 
congestion volume (basically a big token bucket of downstream congestion 
tokens for all flows at once). Despite being bulk across flows, this 
policer incentivises each flow's end-point transport to take care to manage 
congestion between themselves within the envelope of the bulk policer. But 
just because the policer is bulk, doesn't mean it's not transport.

The subtle point here is that I haven't actually proposed charging for 
congestion. Strictly I've proposed how to reveal a generally useful metric 
at the network layer - downstream congestion volume. Certainly that /could/ 
be charged for, but that wasn't my motivation... at least not for end-users 
(because there's loads of evidence users don't like unpredictable pricing 
etc etc).

Instead, we wanted to reveal a metric in network layer headers that would 
allow inline transport mechanism (policing) to be applied to it, not just 
out of band (charging, penalties etc).

(Hence the need for a metric about the expectation of downstream congestion 
so it could be policed at the ingress before the packet entered the 
internetwork.)

However, between networks I was motivated by congestion charging or any 
similar use of the metric (e.g. incorporation in inter-domain SLAs with 
penalties for exceeding thresholds etc). Strictly, that's application-layer 
(in the management-plane if that's how you see the world) on top of a 
network-layer metric.

> >> First, however, what does that mean?
> >>
> >> There are many issues there; it's not cut-and-dry.
> >
> > Agreed.  The question is whether the IETF should be working to address
> > those issues, or ignoring them.
>
>I've stated before - the IETF isn't the IRTF; some of this is research
>(i.e., what are the dimensions of the solution). Some of it is also a
>combination of policy and pricing - neither of which are IETF purvue either.

Of course I'm not expecting the IETF to do the policy, but I am expecting 
it to provide the protocol that can have policy applied to it. That's very 
much IETF land.

>The IETF is a good place to standardize a technical solution once we
>know what we're solving. At this point, "we need to do something" is
>insufficient rationale to proceed

"We need to do something" is a rather dismissive characterisation of a 
fully spec'd protocol proposal:
<http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-re-ecn-tcp-04.txt>

I can't win either way. When I put re-ECN to the IETF, everyone didn't 
understand why it was needed. Now I've explained why it's needed, people 
complain the IETF doesn't get involved in economics and policy. I'm not 
asking the IETF to get involved in economics and policy, I'm asking for a 
bit in a header. The wider industry does the policy. But the IETF needs to 
state the requirements it understands it is trying to meet.

>(and, as others have noted, it's not
>clear we _need_ to do anything).

Tell that to the CEO of any network operator. I think you're saying there's 
not an engineering problem. But that's because resource allocation problems 
are economic problems not engineering problems... However, this one has an 
engineering solution. Of course, the network engineering continues to 
_work_ with any particular sharing of resources, but that doesn't mean it's 
'working' in an economic sense.

Consider this: folks at the IETF can expend weeks on whether a certain 
sentence about the minutiae of a certain protocol field will be a MUST or a 
SHOULD. But apparently we don't need to spend any time at all on fixing the 
whole of resource allocation and accountability.

> >> A solution inside the transport layer solves only the most
> >> trivial variant, and leaves so many loose ends elsewhere it doesn't put
> >> a dent in the overall problem.
> >
> > Can't the same be said for enforcing flow-rate fairness?
>
>Excellent point. You'll note that we do nothing to enforce that. The
>goal is that the network congestion feedback is proportional the number
>of streams - there's no rule that prevents opening multiple streams (as
>was noted) or merging them either. The whole thing works mostly because
>the degenerate case - a stream per packet - is prohibitive.
>
> > If that does
> > more than put a dent in the overall problem, then congestion charging
> > can make a bigger dent.  If flow-rate fairness *doesn't* put more than
> > a dent in the overall problem, should we forget about "TCP
> > friendliness"?
>
>We don't enforce that either. We encourage that as a general principle,
>and assume (as above) that so long as the number of TCP streams is
>roughly proportional to the number of users, things will work fine.

Server farms would get a couple of flows? I think you might mean "roughly 
proportional to the amount different users are paying for their access". 
But then, why constrain ourselves so that usage has to be approx 
proportional to access link capacity? It's much more economically efficient 
to separate the two:
- access capacity is only about peak bit rate
- different users have extremely different demands for how much they use 
their capacity

>They
>don't work if the number of flows grows with the load; if we get to a
>point where that actually happens, then maybe we'll need to do somthing.

Indeed, we have got to that point.

And what about if the duration of the numerous flows from single users are 
also hundreds of times longer than those from others (who are paying the 
same)?

Indeed, we have got to that point too.

See for instance any graph on distribution of usage between different 
residential users. E.g. Figs 12 & 13 in Kenjiro Cho's paper on p2p usage in 
Japan in SGICOMM'06.
<http://www.sigcomm.org/sigcomm2006/discussion/showpaper.php?paper_id=21>

But anyway, why is it that the IETF can happily expend time on volumes of 
engineering trivia, but an economic efficiency problem is given a bar 
hundreds of times higher before the IETF should stir itself into action? If 
there was a 54% inefficiency in a protocol, the IETF would be in there 
trying to fix it. But if it's 54% economic inefficiency it's not important?

Bob

>Joe
>
>--
>----------------------------------------------------------------------
>Joe Touch                Sr. Network Engineer, USAF TSAT Space Segment
>                Postel Center Director & Research Assoc. Prof., USC/ISI
>

____________________________________________________________________________
Bob Briscoe, <bob.briscoe at bt.com>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 