[e2e] Fwd: TCP ex Machina

[Joe Touch]

Forwarded as requested.

Joe (as list admin)

Begin forwarded message:

> From: dpreed at reed.com
> Subject: Re: [e2e] TCP ex Machina
> Date: July 26, 2013 7:35:17 AM PDT
> To: "Jon Crowcroft" <Jon.Crowcroft at cl.cam.ac.uk>
> Cc: "Joe Touch" <touch at isi.edu>, "<end2end-interest at postel.org>" <end2end-interest at postel.org>
> 
> Jon -
>  
> Personally, your presumption that anyone is proposing an orthodoxy here is a bit stretched.  That's a "when did you stop beating your wife"-style comment.
>  
> Research is one thing.  Google, in this case, is not proposing research.  It is proposing *deployment* in the real world of an untested technical improvement that is proposed to solve a problem that may or may not exist at present.
>  
> Peer-review in our field (computer and communications systems research) is highly corrupt.  I realize this is a serious charge.   It is not that the peer-reviewers (the people) are corrupt individuals.  What I mean is that passing peer-review is taken as a stamp of "certification" by those who eagerly want to deploy systems, make money, regulate systems, etc.
>  
> In the scientific method (Baconian or whatever) there is no "peer review" step.   The gold standard is a mix of replication (which implies replicability) and correlation with other results, and a vast and deep examination of the context and validity of applicability of the results in the "real world".  Peer review *CANNOT* replace this, and should never do so.  (otherwise, we should hold peer-reviewers accountable for any and all harm caused by the papers they review.  This cannot be achieved when most reviewers delegate the reviews to graduate students, as is the norm in our profession.
>  
> Getting back from peer-review, I believe Joe (and I) are expressing a concern that these proposals by Google and various academics are not *research proposals*.  One does not issue a research proposal by issuing a press release, or promoting the idea prior to testing on your blog as if it were *policy*.
>  
> Given the power Google has to "just deploy it, dammit", it is, I think, responsible engineering (not science) practice for engineers whose professional concern includes impact on the public to point out that the proposals are absolutely untested, have certain risks of misbehavior, and may damage the performance or behavior of a vast number of *already deployed* applications.
>  
> It would please me greatly if Joe would allow this message to be posted on e2e, just this once.
>  
> I think we should be very clear that the Internet as it is today is not a science-project, but a real public work, even though it is not the work of a government and probably should never be "governed" (even by the ITU).
>  
> Engineering responsibility (not the scientific method alone) should be at least as important in its evolution.
>  
> That doesn't mean that new ideas should not be tested scientifically and deployed experimentally.  But we should be *careful* not to conflate "orthodoxy" with engineering caution.
>  
> And it is perfectly fine, in my mind, to challenge those who would *promote* ideas that are not even tested (only simulated in toy environments or run in testbeds that are far from the real world).
>  
> The sad thing about our "science" is that the presumption that network traffic is generated by independent, simple Markov processes and are Gaussian (in the sense that aggregation reduces variance) is not even questioned by those who do peer review, much less those who have the experimental disproof of this staring them in the face (academics).
>  
> Our field has a bad case of *dogmatism* in its experimental assumptions.  Not orthodoxy, but an arrogance that because an assumption makes a problem appear to be tractable mathematically that the assumption is true.
>  
> That dogmatism is shared with neo-classical economic theory.   Despite years of being horribly wrong, we still suffer from the arrogance of that field.
>  
> Let's not let the field we share become like that.  It's a danger to society now that the Internet is central.
>  
> 
> 
> On Friday, July 26, 2013 12:53am, "Jon Crowcroft" <Jon.Crowcroft at cl.cam.ac.uk> said:
> 
> > I'm sorry, but this smacks of "orthodoxy"
> > which is anathema to good science.
> > 
> > the reality is that the nature of
> > most of the "optimization" work that makes it into good conferences
> > (nsdi, icnp, sigcomm, pick your fave)
> > are in the nature of the word you use - extensions
> > 
> > they do not "sidestep" anything - they change the behaviour in the
> > situation where the extesion applies and not otherwise, and are
> > typically "backwards compatible" as much as possible, as the one
> > true orthodoxy that is around is that any decent paper evaluates
> > the tcp-atlantis or tcp-erewhon both in the new scenario, and in a
> > mix with tcp-vanilla or tcp-realworld, to see what the impact of
> > each on the other is, because thats what scientists do when they do
> > their evaluation properly - they want to know what happens in the
> > real world, not just in some bubble world
> > 
> > however, it is also legitimate science
> > to ask what happens in some brave new world,
> > if one has the capability to change everything -
> > 
> > some of these brave new worlds aren't small -
> > they are places like all cell phones,
> > or in a data center with a few million cores,
> > where one might actually have the capability
> > to have a flag day
> > (ship the tcp-flagrante to all devices and enable)...
> > 
> > that new world might not share the same primary goal -
> > 
> > for example, in some data center it might be
> > perfectly reasonable to sidestep your goal
> > as the modest owner of the data center might actually
> > want to maximise total crosssectional throughput
> > for some odd reason...
> > 
> > I just dont buy the homogenous trope any more -
> > what is more, we have dynamic systems that
> > rebuild, relink, reload, reboot -
> > we can decide which is the appropriate behaviour -
> > indeed, many of these systems do this anyhow at some level
> > (when switching from wifi to 3g/4g,
> > when switching from hadoop using tcp to do reduce/shuffle phase,
> > to video streaming over tcp to a customer -
> > in lots of other cases)
> > 
> > for these people,
> > what is wrong with experimenting and publishing
> > papers on the results of the experiments? it would be wrong
> > not to publically propose and expose evaluations of these
> > extensions or modifications.
> > 
> > in any case,
> > TCP and IP were not perfect at birth
> > through some sort of intelligent design
> > which magically foresaw all these new scenarios-
> > 
> > they were prototypes that expressed some very cool ideas,
> > but missed some details and I believe we learned some new stuff
> > since anyhow...
> > 
> > so of course there might be extensions which would kick in,
> > in the wrong circumstances and do harm -
> > 
> > so do you believe any OS vendor or
> > community would ship those?
> > 
> > do you believe that unless the papers about them showed a net win,
> > anyone would use them?
> > 
> > do you think we shouldn't propose ideas,
> > that occasionally turn out to be wrong?
> > how else will things move on?
> > 
> > in any case, the TCP ex Machina paper isn't this at all
> > so on the topic of evolving (and evaluating in a complex rich way)
> > new variants of CC, I think one way you could think about this paper
> > is the following
> > 
> > instead of waiting a duty cycle of 1 grad-student-year for a next
> > TCP congestion control idea,
> > you use a big cluster of machines and evolutionary computing
> > to run at the rate of around 100 graduate students-per-week
> > and pick the best according to some metric -
> > 
> > you could, if you like, pick a different metric - Fred's proposed
> > one - that's fine - the paper just illustrates that we have a
> > capability to do some thing which was probably rather difficult to
> > consider in 1976 or 1988 (pick your preferred incept date here)...
> > 
> > I'd quite like DNS lookup and search to go faster anyhow- but i
> > dont think the people working on that want the download times of
> > the PDFs of their papers to slow down or be unrelable as a
> > result...
> > 
> > Do you have specific examples of work that would do that?
> > 
> > cheers
> > jon
> > 
> > 
> > In missive <51F19708.3050206 at isi.edu>, Joe Touch typed:
> > 
> > >>FWIW, I've been concerned about a number of proposed "extensions" to
> > >>TCP, either to optimize it for DNS (TCP-CT) or for Google (various).
> > >>
> > >>My concern is because these optimizations sidestep my view of the
> > >>primary goal of TCP as described below.
> > >>
> > >>Joe
> > >>
> > >>On 7/25/2013 2:00 PM, Joe Touch wrote:
> > >>>
> > >>>
> > >>> On 7/21/2013 11:35 PM, Fred Baker (fred) wrote:
> > >>>> But I think maximizing throughput while minimizing the
> > probability of
> > >>>> loss is probably the holy grail.
> > >>>
> > >>> It also depends on whether you measure that as an individual or as a
> > group.
> > >>>
> > >>> I tend to think of TCP as supporting the most participants having
> > >>> reliable communication as a primary goal, and a secondary goal to
> > >>> maximize (though not optimize) the aggregate data per unit time --
> > >>> subject to the primary goal.
> > >>>
> > >>> I.e., it's not about max BW for a given connection, and it also
> > almost
> > >>> never means minimizing the transfer time for a single connection.
> > >>>
> > >>> Joe
> > 
> > cheers
> > 
> > jon
> > 
> >