[e2e] Is sanity in NS2?

[George Riley]

I've restrained from replying until now, but must jump in.

Yogesh Prem Swami wrote:
> I have a somewhat general question about simulations. My question is
> that is there any scientific reason why simulations should be able to
> predict the behavior of a real packet transmission phenomena? Unless  
> you
> make the assumption that packet transmission/interaction is a
> non-chaotic phenomenon (chaotic, as used in physics), there is no  
> reason
> to believe why a simulation would be able to model real world events.

What chaotic behavior are you talking about?  The behavior of a TCP
endpoint is completely deterministic.  Given a set of data demands on
a TCP connection, and a given network topology with no competing
flows,  the endpoint will do exactly the
same thing every single time.  The behavior of a router with a fixed  
size
DropTail queue and a fixed set of links will behave exactly the same way
every time, given an arrival pattern of packets.  Both of the above  
statements
are ignoring small variations in interrupt response times and process  
scheduling
at the systems in question, but these variations are small and can be  
ignored
without affecting measured results.

The job of the simulator is to recreate these deterministic and well  
known
behaviors, without having to construct a real network.  A high-quality
simulator, such as ns2 or GTNetS does exactly this.  A competent  
programmer
can read the RFC's describing what the TCP endpoint is supposed to do,
and implement a model that does exactly that.  BTW, if you haven't seen
the Georgia Tech Network SImulator (GTNetS), it might be worth a quick
look.  Google for GTNetS and check it out from our CVS repository.

The above statements are bit tongue-in-cheek of course, because we all
agree that networks are in fact chaotic.  But where does this chaotic
behavior come from?  It comes from unknown and unpredictable behavior
of the end-system applications and users.  Again, any good simulator can
model this as well, as both ns2 and GTNetS can do.  The randomness
is due to randomness in the data demand at endpoints, ignoring
intentional randomness is certain AQM methods such as RED.
But again, the job of the simulator is NOT to intentionally produce
chaotic behavior in a network, but rather to predict what that network
would do, given a set of (possibly random) inputs.  I claim that a good
simulation tool can and does do exactly that.

Simulation is not supposed to be a replacement for actual in-vitro
experimentation, but must be used in cases where such experiments
are not possible.  Consider Sally's recent work on TCP Quick-Start.
The Quick-Start implementation requires support from routers, to
react to and modify a new options field in the TCP header.  Is Sally
supposed to somehow get access to Cisco IOS source code and modify
it to support the new header? Of course not.  Is she supposed to get
access to Linux source code (easily done) and hack it to support
the new option (not so easily done)?  Again of course not.  She uses
a simulation tool that she is quite comfortable with, and constructs
both router behavior and end-system behavior as she imagines it will
be implemented in the future, and demonstrates the differences in
performance using Quick-Start.  In this case, clearly a field experiment
is not possible.

The criticism that simulation tools, such as ns2, are responsible for a
plethora of poor quality networking research is just bogus.  The  
simulator
is a tool who's job it is to predict what a given network will do under  
a
given set of conditions.  The researcher is responsible for deciding  
what
is a reasonable set of conditions to be used to study a given problem.
ns2 and GTNetS and the other simulation tools will uphold their end of
the bargain, it is the researcher's responsibility to uphold their end.

George


------------------------------------------------------------------------ 
--------------------
George F. Riley, Assistant Professor
The School of Electrical and Computer Engineering at Georgia Tech
Atlanta, GA  30332-0250
(404)-894-4767
E-Mail: riley at ece.gatech.edu

ECE3090 Web Page: http://users.ece.gatech.edu/~riley/ece3090/
ECE6110 Web Page: http://users.ece.gatech.edu/~riley/ece6110/