[e2e] bandwidth and file size

[Joe Touch]

> Soo-hyeong Lee wrote:
> 
> 
> Hello,
> 
> It is well known that link bandwidth of the Internet is growing very
> fast.
> Does the file size also grow at the same speed? Or is it much slower?
> 
> I have once read a Kleinrock's paper saying that future large
> bandwidth delay network must deal with the case where file size is
> relatively smaller than BDP(bandwidth delay product).

There are two follow-ups to that paper which describe the problem
in more detail, and give some additional context:

    J.D. Touch and D.J. Farber, "Reducing Latency in Communication,"
    Letter to the editor IEEE Communications Magazine, Vol. 31, No. 2, 
    Feb. 1993, pp. 8-9.
    http://www.isi.edu/touch/pubs/CMletter.html

    J.D. Touch, "Defining `High Speed' Protocols : Five Challenges & 
    an Example That Survives the Challenges," IEEE JSAC., special issue 
    on Applications Enabling Gigabit Networks, Vol. 13, No. 5, June
1995,
    pp. 828-835.
    http://www.isi.edu/touch/pubs/jsac95.html 

> However, I have found that file size also increases.
> In near 1991, file size (transferred by FTP) must have been 100B in
> average and mostly below 100 KB, according to Fig.2 (c) in Tcplib
> documentation http://irl.eecs.umich.edu/jamin/papers/tcplib/.
> In near 1997, it increases to 240KB in average, according to Table 1
> of Claffy's paper "the nature of the beast: recent traffic
> measurements from an Internet backbone," Inet, '98 at
> http://www.caida.org/outreach/papers/Inet98/

As Lars mentioned, these are FTP file sizes, not HTTP, the more dominant
case. Note also that bandwidths have gone up in that period as well, 
typically by at least a factor of 10, which means the BDP product
is 5x bigger than it was, relative to the average file size.

Which means you're much more likely to be operating in
'request/response'
mode than in 'streaming' mode.

> I think this issue is related with the performance of TCP congestion
> control in the large BDP environment, because small file size means
> that performance impact of initial slow-start gets more stressed and
> congestion-avoidance gets less important.
> I hope you will make kind comments.

This was addressed in further detail in a few places, including:

    John Heidemann, Katia Obraczka, and Joe Touch, 
    "Modeling the Performance of HTTP Over Several Transport Protocols"
    IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630.
    http://www.isi.edu/~johnh/PAPERS/Heidemann96a.html

Some of this led to the design of our TCP control block sharing
(RFC2140) and Hari Balakrishnan's similar Congestion Manager (Sigcomm
1999). 
In both cases it's possible to avoid slow-start by sharing and/or
reusing
congestion information.

Joe