[e2e] simulation, sanity, citation, and storage

[Jon Crowcroft]

How perhaps not to do mobile nets research :-

1. The random waypoint mobility model
(probably
the most commonly used model for how devices move that is used in
100s of papers for simulation of how good my new mobile ad hoc routing
protocol is compared to yours) -
has non stationary distribution of velocity in pretty much any
plausible 2D space, so you
don't know when to stop/start simulators using the random waypoint
mobility model:

This paper is whats wrong, and how to fix it:
"Perfect Simulation and Stationarity of a Class of Mobility Models,"
J.-Y. Le Boudec and M. Vojnovic, IEEE Infocom 2005, Miami, FL, 2005
(Infocom 2005 Best Paper Award)
http://ica1www.epfl.ch/RandomTrip.

2. Many simulation papers have not only got non reproduceable results,
but the quality of simulation papers is apparently
getting worse over time:

This paper is about how this trend has progressed, and should be
essential reading for all 1st year grad students:)

"{MANET} Simulation Studies: The Incredibles",
by
Stuart Kurkowski and Tracy Camp and Michael Colagrosso,
mc2r, 2005, volume 9, 4, pages "50--61",
"http://doi.acm.org/10.1145/1096166.1096174"

Its pretty "interesting" how bad things are out there:)



Two possible social solutions:-

a) on the one hand,
so how about we introduce the idea (e..g via Vint Cerf, now at google)
to Google Scholar, of _negative_ citations. it ought to be easy to add
to the pagerank algorithm:)

Given we find that a numnber of papers have been written that
are rooted on some baseline reference (e.g. the original
NS code for mobile IP, or the original random waypoint)
when we then show that the NS code is wrong, or the random waypoint
model cannot be used to compare results in simulation experiments due
to lack of stationarity, those papers all suddenly have their
citation count reduced:)

b) on the other hand,
if people post their _code_, then bugfixes to the  code would accrue
_positive_ citations. we could use
google base for this 
http://base.google.com/base/default
could be a repository for simulation
code, configurations and draft papers
(A bit like the science preprints idea, but for _supporting_ evidence.

cheers

jon