[e2e] double bland reviewing
David P. Reed
dpreed at reed.com
Thu Apr 29 04:00:35 PDT 2004
Separate the two problems: conference integrity and constructive feedback.
If an author needs honest and constructive feedback, that is best obtained
by seeking out reviewers separate from a "judging" process such as the
program committee. The goal of the program committee and peer review
judgment of quality is not well aligned to the process of improving
unselected papers. Selection comes first, and improving comes after.
I doubt that eliminating reviewer anonymity would eliminate cheap shots and
logrolling (which happen at selection - most reviewers usually engage with
papers they like). The game would then just turn to low ratings with
minimal justification.
Cheap shots and logrolling should be obvious to the program committee, who
should disregard those reviews in any case.
If you want to eliminate the effect of cheap shots and logrolling, make the
program committee process transparent and public - it's there (in my
experience) that the integrity of a conference is lost.
More information about the end2end-interest
mailing list