[e2e] Short Fat Networks, tcp and Policers

[a simpleton]

See within (--->)

Puddinhead Wilson <puddinghead_wilson007 at yahoo.co.uk> wrote:
Hello Simpleton/all,
 
I have some queries

Please follow the >>




A) Minimum throughput
=======================

When you browse through tcp literature, you will
usually come across
the Bandwidth x Delay=Window formula. It has a very
straightforward 
meaning: If you want to get a full utilization of the
available bandwidth,
you better have a window that is large enough to fill
the RTT with packets.

Even a simpleton like myself knows this. But let us
ask the question in 
reverse: How *low* a bandwidth can we pass through a
line with a given, fixed,
delay? Some simple thinking will show you than it is
(almost) impossible to 
deliver less that a few MSS each RTT. Why? Because
even during slow start, tcp
sends a new packet once an ack is recieved.


>> excellent point, I would appreciate if this list
for one clarified the "MSS". Today we support jumbo
frames etc. Does this imply we increase the MSS
propotional to the MTU?



What is the implication of the above? That in order to
cause a tcp flow to use *less* than the
physically available link, one must *increase* the
RTT. Of course, this is exactly what shapers
do, by buffering the data. alternatively one could
buffer the acks ( and I think that "ack holding"
schemes have been proposed in the literature), but
that requires L3- L4 knowledge and treatment of 
piggybACKs.


>> Shapers also provide something else. They let you
differentiate services. Assume that one wants to
really run a "converged network" which carries voice,
video and data, one also needs the flexibility to have
CBR. Shappers can do that.
Eitherways, shappers or no shappers, if everything was
"policers" or say, slotted/TDM access based, there
would still always be a point in the network where one
would have to "buffer", and that is the point of
ingress into the "policed" core. This specifically
holds true for cases where we have say a 100Mbps LAN
and a X<100Mbps WAN slotted access bandwidth in front.

----> Agreed, but one way to use shapers to differentiate, is the diffserve scheme.  In diffserve, you don't allocate a shaper per *service* but per *class of service*. or in other  words, you aggergate all services with the same CoS into the same shaper. This is done inside the network. At the edges, you can either shape, or police. If you police, you will run into the problems that I have described.

B) tcp timeout Phase
========================

So what happens when you try to limit bandwidth below
the MSS/RTT limit? A policer achieves this
by discarding all packets that are non conforming, and
this will cause the session to run in a
burst-timeout-burst phase. Typical implementations set
timeouts to as much as several hundred msec.
In this phase ( that is , when ther required bandwidth
is less that Const X MSS/RTT), the bandwidth 
can range from zero to [Policer Burst Size]/[Tcp
Timeout] . This phase is not efficient, and causes
huge
bursts in the network. For timouts around 500 msec, a
4 mb/s session ( which is not unreasonal on LANs
and WANs) needs a policer burst of 512KB. since bursts
tend to get translated into buffer sizes, we see
that a single service eats up quite a large buffer
size. The irony being, that in the timeout phase, the
burst is not buffered, because if the burst was
buffered, the RTT would increase, and we wouldn't have
been
in the Timoeout phase in the first place.


>> agreed.

C) Partial window phase
========================

A reasonable way to generate sub-physical line rates,
( without adding to the RTT), is to cause 
the tcp to work at a window that is less that
[physical Rate]x[RTT]. The frame pattern would be
something like M frames every RTT, with MxMSS/RTT ~
[policed rate]. This is a much better behaviour
than the timeout phase, and a good policer design
should strive to reach this phase. As pointed out
before, this phase cannot exist when the required rate
is too low, or the RTT is too short.

>> Which is why TCP slow starts I presume (I believe
the more "literate" on the list can correct me if I am
wrong).
----> I dont think that slow start was "designed" with policers in mind.

D) Stability of the partial window phase
========================================

Eventually, because of slow start or congestion
avoidence, the number of frames in the window M
slowly creeps up, until the policer "realizes" that
the policed rate has been passed, and the policer
will discard a series of frames. If this is done
delicately enough ( suppose using a RED like
algorithm),
Fast retransmission will take place and the session
shall be able to slow start its way back to the target
of 
M frames per RTT. 


If the policing is too drastic, either an entire
window of M packets will be discarded, or the
fast-retransmit frame itself will be lost, and a
timeout will occur. 
E) tcp defence lines and policers
==================================
tcp has three defence lines against congestion
* self clocking
* congestion window + slow start
* retransmission timeout
Not only do they protect the network, they also
control the bandwidth that the application recieves. 
Policers neutralize completely the first line of
defence, since they have no effect on the RTT ( or on
the
more subtle inter packet gap ). The only way that
policers can indicate rate to the tcp layer is by
packet discard. Tcp responds to packet discard by
retransmission timeouts or fast retransmission, both
are considered inefficient, but compared to the huge
problems caused by timeouts, the slight ineffciency
caused by fast retransmission induced slow start, is
minor. The estimation of timeouts based on averaged
RTT
statistics are totally irrelevant when the actual
network performance is below 100msec, but the
recommended
tcp tick is 500msec.

>>Also consider the case of "how soon can that
feedback" be given to the sliding window mechanism so
as to detect that there was a "congestion". We are
talking about a system where the propogation delay or
duration of "feedback" is considerable compared to the
rate at which the input comes/packets are transmitted.

F) What is the point?!
======================
The points are 
1- policers are much easier to build that shapers, so
we should start to understand them.

>> Considering all the points you have made, it seems
shappers should be the obvious choice. They need not
be the "exact buffered" shappers as you mention them,
but they could be made such that:
a. minimal buffering is needed at the end points.
b. There is a way to differentiate CBR streams from
non CBR streams

---> I suppose that giving a shaper per service is a possibility, but the complexity, compared to a shaper per CoS is prohibitive. Policers, on the other hand , are relatively simple to implement, and scaling is not a big issue. Shapers per CoS are enough to support differentiation, I'd imagine that policed CBR streams would go into a single high prioirty buffer, instead of having to select among many eligible CBR shapers .  

2- acceptance tests are usually done with very short
RTT times, so that the timeout phase is
quite relevant.
3- All these exponential smoothings and estimations of
RTT RMS are useless if the smallest timeout
is 300msec, and typical LAN's are less that 50msec.

>>Why would you say that?

---> my point is that afaik, the clock used to measure rtt and rttvar has a granularity of a few hundred msec ( except in very recent tcp implementations, where the cpu clock is used ). 

4- alot of TCP improvements have been based around
"LFN"'s but it may turn out that alot of the broad
band networks are really "SFN"'s (Short Fat Networks).


>>Let me put it this way, performance is always
"relative". If you sit near an end point then it is
SFN for you, else it is a LFN. Would that not be true?
Eitherways, the idea is to get "more" out of an
existing infrastructure, not to say "this
infrastructure was not made for this".

---> I dont quite follow you here. Consider the following "practical" situation:

Your customer wants to bench mark  your equipment. She hooks up 2 pc's back to back through your traffic managment box, and starts pinging /iperfing when the traffic managment is disabled. She measures a beautiful 700usec delay and a 100 Mb/s  throughput ( say that its a 100bT FE interface). Now she tries to set the box to say 1Mb/s. There are two possibilites:

1) you are using a shaper, so that the delay jumps to about 12msec, and the bandwidth is fine.

2) you are using a policer. Every packet that gets transmitted has a 700usec delay, but there are alot of timeouts. Bandwidth is fine, because you have configured a large burst.

Both results need some explaining to the client. This is a fairly common situation.

G) The End
============
Thank you for you patience, comments are welcome.

Yet Another Simpleton








____________________________________________________________
Yahoo! Messenger - Communicate instantly..."Ping" 
your friends today! Download Messenger Now 
http://uk.messenger.yahoo.com/download/index.html


		
---------------------------------
Do you Yahoo!?
SBC Yahoo! - Internet access at a great low price.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.postel.org/pipermail/end2end-interest/attachments/20040515/86f09f3a/attachment.html