[e2e] Queue size of routers

[Dennis Ferguson]

Greg,

> i think routers in the core don't have a delay*bandwidth of buffer memory 
> (though i could be wrong).  i think they limp by with 5-20 ms of buffering.  
> if this is true, it may be the reason it works is that the "high level of 
> aggregation" at the core really does mean that the load is fairly constant, 
> without the peaks associated with looking at a single TCP's queue length.  
> again, note all the caveats, etc., above.

I believe (based solely on a single long-ago observation that an M/M/1/K
queuing model seemed to predict the measured behaviour of core routers
with short buffers pretty accurately) that the speculation in your third
sentence above may in fact be true.  To relate this to another recent
thread on this list, however, this seems like one of those things that
should require no speculation since it is not so difficult to measure, yet
I know of no good-quality data from "typical" core circuits which has been
published anywhere.  The network we've built is constructed with insufficient
instrumentation to enable us to understand what it is we've built with any
certainty, so we speculate.

There are, however, practical constraints which simplify the choice of how
much output buffer memory gets built into routers.  When picking memory
technology for output buffering there has generally been two choices, SRAM
or DRAM, with the best parts in either technology being those in common use
by the PC industry.  SRAM is lovely if you can get by with it, since its
interface often substantially reduces the complexity of the logic writing
and reading it, but it becomes prohibitively expensive in cost and power
at sizes beyond, say, 10-20 ms of output buffer.  If you doubt your ability
to sell a router with so little output buffering your only alternative is
to choose DRAM.  Getting good bandwidth-efficiency from DRAM makes your design
more complex, but if you have to do this then you'll probably find (given
that the PC industry has historically valued DRAM size over DRAM bandwidth)
that the number of chips you need to meet the bandwidth requirement is
such that it isn't possible to build an output buffer smaller than, say,
200 ms, since that's what you get if you build from PC-sized DRAMs and
using chips smaller than that would actually cost you more.

So the choice is between building routers with very modest output buffers
and routers with quite large output buffers, with nothing between those two
extremes making economic and technical sense.  Since buyers object to routers
with tiny output buffering modern core routers tend to have a lot of it, and
generally require you to go out of your way to configure smaller output
buffering if you want less than the hardware requires.

Dennis Ferguson