[e2e] Can we revive T/TCP ? => persistent connections

[Michael Welzl]

> > In practice, this doesn't seem to be the case. In all the tests my
> > students did (not a thorough measurement study, just some
> > experiments), the server closed the connection after sending a page.
> >
> > I think this is due to the (quasi-)stateless operation that a HTTP  
> > server
> > can achieve this way - I mean, it's much more difficult to keep
> > connections open for a longer period, and close them only after a
> > timer expired, count the number of connections that should be cached,
> > etc. etc. ... if poorly implemented, this might also not scale so  
> > well.
> 
> Could you elaborate on how you did those tests?  A quick, highly  
> scientific
> check showed:
> 
> www.yahoo.com:  no connection caching
> Google:  caching
> microsoft.com:  caching
> www.cmu.edu:  caching
> 
> The connection timeouts on some of those are fairly short;  some are  
> long enough for subsequent clicks, many are only long enough to fetch  
> embedded objects (a few seconds).

ah, okay... well, perhaps it's because we didn't bother to set up a
proper testing procedure for this one, then. i remember checking it
once with one single web server (clicking + watching ethereal), then
(assuming i'm right, and connections aren't kept open) told them to
briefly check with some other web servers ... they came back and
said that connections were always closed by the server. perhaps
they took too much time between clicks, picked too few servers, whatever.

anyway, this is news to me, and i find it really interesting!


> >> (I'm sure there are scenarios where it will, of course.)  In the Grid
> >> context, if you're talking about a not-huge set of trusted nodes,
> >> they can cache those TCP connections for quite a long time.
> >
> > But they don't - neither in the smaller nor in the larger Grids that I
> > know of; I think it's because the notion of a "connection" is lost
> > in the (vertical) communication across layers.
> 
> I'd suggest that that's not the fault of T/TCP, but the fault of the  
> upper layers in the architecture...

yep, this seems like the right place, given the narrow scope
of usefulness of t/tcp you pointed out in your previous email ...
seems like it could only be useful for grid computing, where things
are in fact going wrong at a higher level.
 

> > Grid Services are usually implemented on top of SOAP, which is
> > stateless. How should SOAP tell HTTP to maintain a connection
> > when it can't know whether a Grid Service will be called again? The
> > decision to do so is up to the programmer, who however can't provide
> > the remote SOAP instance with the necessary information because
> > the notion of a "session" isn't part of SOAP.
> >
> > Could connections be cached in a transparent manner in such a
> > scenario (e.g. by tweaking something at the HTTP level, but not
> > above)? I think so, but I'm not 100% sure. Also, if it's possible,
> > why isn't it done? In a Grid, this would surely make sense.
> 
> Yes.  Some SOAP and XMLRPC libraries do this.  See, e.g.,
> 
> http://www.gnuenterprise.org/tools/common/docs/api/public/
> 
> "gnue.common.rpc.drivers.xmlrpc.ClientAdapter.ClientAdapter:  
> Implements an XML-RPC client adapter using persistent HTTP  
> connections as transport."

wow! thanks again for this very helpful pointer!

you really clarified a lot of things for me today.

cheers,
michael