[e2e] TCP "experiments"

[Jon Crowcroft]

at some point you need to do live testing - this is done in
pharmaceuticals (c.f.clinical trials), food (gm crops), 
power systems, transportation, pretty much every sector (hey even
politics) - 

it is n't avbout academic v. commerical - 

and there's a non-zero risk things wont work out

the point is that there are a number of more important players than
the IETF who (also) have an interest in not breaking the internet, 
who do these experiments - and they monitor stuff and they
(usually) report on it - and they operate within real world
constraints but need those constraints (as everyone's said, the
internet doesn't have nice theoertical traffic that stays still
while you do your experiment, nor does it just have vanilla routers
that forard any and all standards-compliant packets) 
to tell if the experimental results are going to be useful or not

the internet isn't just one production network - 
its an ecosystem with lots of components - 

just oone R&E network I'm connected to specifically allows 
for experiments (with support from its operational team) - 
its a modest 10-100Gbps backbone with
around 7M hosts....
it also runs production services and has AUPs etc

we've always done experiments on "production" networks - 
every day is an experiment on planet earth by the human race...
hmm - maybe that's one place we should be a bit more careful -
is it good or bad that it isn't run by the IETF:)


In missive <a062408a4ce19c1bbe794@[10.10.24.75]>, John Day typed:

 >>As those with experience in the product world or
 >>operational world will know, the distinction here
 >>is between research/development and
 >>product/operations.  One never does experiments
 >>with a production network.
 >>
 >>There was a time when there was little choice.
 >>The cost of building a network of any size was
 >>astronomical, or at least very high.  So one had
 >>to do experiments on live networks and be as
 >>careful as you could.
 >>
 >>Today that is not the case.  An arbitrary network
 >>of several hundred nodes or even a few thousand
 >>is not that big a deal.
 >>
 >>Joe has a point.  An production network is no
 >>place to be doing experiments.  In fact, quite
 >>the opposite.   Experiments should be done on
 >>experimental networks.  (to state the opposite.)
 >>
 >>Take care,
 >>
 >>
 >>
 >>At 9:24 AM -0700 7/27/13, Joe Touch wrote:
 >>>On Jul 26, 2013, at 9:48 PM, Jon Crowcroft <Jon.Crowcroft at cl.cam.ac.uk> wro=
 >>te:
 >>>
 >>>>  while linux (pick your flavour) cubic isn't vanilla, neither is
 >>>>  microsoft's compound tcp - the latter might have seen a bit more
 >>>>  eval in the literature but the former has seen a lot more big iron
 >>>>  deployment and doesn't appear to have broken the internet yet
 >>>
 >>>How would we know? They're not instrumented.
 >>>These are not experiments; they're deployments.
 >>>
 >>>Even Schr=F6dinger's cat eventually sees the light
 >>>of day (as much as there is a cat in the first
 >>>place).
 >>>
 >>>>  (although there are rumours and reports of corner case problems)
 >>>>  but i dont think either of these are "non tcp" - they are variants
 >>>>  on CC behaviour....
 >>>
 >>>Which is a specified standard, which these mechanisms violate.
 >>>
 >>>You do bring up a valid point about the subject
 >>>line, so I've changed it for this thread going
 >>>forward.
 >>>
 >>>>  also - the ability to do any deployment testing of a new tcp in
 >>>>  anger _requires you_ to be wireline compatible with TCP because of
 >>>>  the "non stadard" but ubiquitous NATs and other middleboxes
 >>>
 >>>The environment doesn't support safe
 >>>experiments, but that is not a valid excuse for
 >>>unsafe ones.
 >>>...
 >>>>  so the gold standard you quite reasonably want to hold people to,
 >>>>  to show their work doesn't do harm in the wild,
 >>>>  requires them to do "harm" by making
 >>>>  their new variant TCP appear chameleon like,
 >>>>  vanilla TCP, so they can get results
 >>>
 >>>So they do harm to avoid doing harm?
 >>>
 >>>They have failed because of their first step.
 >>>
 >>>Joe
 >>

 cheers

   jon