[e2e] was double blind, now reproduceable results

[Michael Savoric]

#Saad Biaz wrote:
#The source of the problem in this matter is the tendency of
our community
#to accept only papers that tell a happy story, a successful
story. We
#forget that we learn more from a sad, than a happy story.

Hi,
I can absolutely agree with that statement above.

During my current research activities I have read several
papers describing
(distributed congestion-control) mechanisms which sounds
very well,
at least by the first reading. To not speak only generally, 
I will give you some example mechanisms: EWA and XCP.

I have made the experience that these mechanisms do not
perform
very well in my own simulations considering different (and I
want to say:
more realistic) network characteristics regarding, e.g., the
available bandwidth or traffic load.

Using these results as a starting point for own research, I
have identified the
disadvantages of these mechanisms and find adaptations of
the original
mechanisms that reaches a much higher performance in my
simulations and in most
cases also in simulations reproducing the scenarios
considered in the original
papers.
 
So, I wanted to publish some of these (not only negative but
also constructive)
results on a prominent conference to inform the research
community about my results.
What happened?
The main review answer was that I have described not a novel
approach,
"only" an adaptation of the original mechanism was
considered.

This looks to me that it is more important to have a new
mechanism than to
demonstrate the weaknesses of existing mechanisms and (!) to
improve them.
In the end, my improvements of the original algorithms are
not published (yet)
so that the original mechanisms look like they are
consistently accepted by the
research community.

What are the main conclusions of my short story?

I think, it is necessary to reproduce results (if it is
possible).
But since also simulations based on curious assumptions can
be
reproduced, reproduction is not the only thing.
It is much more important to reflect about earlier research
results
to identify weaknesses in the considered simulation or
measurement
scenarios or even in the mechanisms.
Unfortunately, these weaknesses are in most cases "hidden",
i.e., they might
be hard to find requiring a lot of work.
And then it is twice a pity that such important results are
not published,
or if they are published they are published on a conference
with a lower
reputation reaching fewer researchers.

Best regards,
Michael Savoric

======================================================
E-Mail:		savoric at ee.tu-berlin.de

Phone:		(+49 30) 314-23840

Fax:		(+49 30) 314-23818

Postal address:	Technical University Berlin
		Telecommunication Networks Group (TKN)
		Einsteinufer 25, 10587 Berlin
======================================================