NIPS & double blind reviewing
Barak Pearlmutter
bap at cs.unm.edu
Wed Dec 18 00:30:47 EST 2002
I've heard three objections to blinded reviews. To my mind, none of
them quite hold water.
OBJECTION 1: It is hard to conceal the authors' identity against the
industrious/perceptive/clueful reviewer. Sometimes clues are
unavoidable.
WHY IT DOESN'T HOLD WATER: So what? In that case blinding isn't any
different from the current situation, so why are you objecting? Not
all reviewers have these abilities, so blinding will work completely
on them. Besides, even the most perceptive reviewer won't figure it
out for all papers, only for some. And even when they think they've
figured it out, being 80% sure of the author is, psychologically,
very different from being 100% sure. Plus, starting an active
search for the author's identity might give a reviewer pause ...
OBJECTION 2: Sometimes the reviewer actually needs to know the
author, eg for theory papers where whether a proof sketch is
believable depends on the author.
WHY IT DOESN'T HOLD WATER: Err, really? Well, if the reviewer feels
themselves to be in that situation, they can either say so in the
review, or ask the program committee for the author's name with a
brief explanation as to why. It certainly seems healthy,
particularly in this (surely quite rare, and therefore low amortized
overhead) situation, to have the first pass through the paper be
blind!
OBJECTION 3: The author might be a well known plagiarist/crackpot/liar.
WHY IT DOESN'T HOLD WATER: This is the program committee's job.
Anyway it would be easy enough to reveal the authors' names to the
reviewers *after* they have their reviews in, so they can bring such
an extraordinary situation to the program committee's attention.
More information about the Connectionists
mailing list