No subject

[Chris J.C. Burges]

Sue Becker writes:

>> Two of the key factors NIPS reviewers are asked to comment
>> on are a paper's significance and originality. Very often
>> work is submitted to NIPS that is only a marginal
>> advancement over the author's previous work, or worse yet,
>> the same paper has already appeared at another conference or
>> in a journal. In the course of reviewing for NIPS I have
>> often looked at an author's web page, past NIPS proceedings
>> etc to assess the closeness to the author's previously
>> published work.  Double-blind reviewing would make it much
>> more difficult to detect this sort of thing.

To me, this is the first compelling argument against double blind
reviewing put forward in the debate so far (it is not in Dale
Schuurmans' list).  However I think the issue Sue raises can be
addressed as follows.  Require that, if authors have closely related
work that has been published or submitted elsewhere, they send in a
copy of the single closest work to that submitted to NIPS, together
with a VERY brief description of how the NIPS submission is different.
The session chair (not the reviewer) then incorporates this into
his/her decision.  If an author abuses this trust, a penalty can be
applied, much as the IEEE applies a (severe) penalty in similar
circumstances (immediate rejection, immediate withdrawal of all
submitted manuscripts by any of the authors, and prohibitions against
all of the authors in any IEEE publication for one year: see e.g.
http://www.ieee.org/organizations/society/sp/infotsa.html ).

Yes, this requires a bit more effort on the session chair's part
(although only some submissions will need to do this).  But actually
whether or not double blind reviewing is adopted, I think this is a
separate issue, and maybe a good idea for NIPS anyway.  In previous
years, NIPS encouraged submission of work that had appeared in part
elsewhere, provided it would be new and interesting to the NIPS
community.  This year a different policy was adopted, requiring
stricter originality, and perhaps it will need some enforcement policy
for it to work.  After all, regardless of the blind reviewing issue,
Sue's method - checking up on the author's web page - won't work for
people who don't have web pages or who do not put recently submitted
material on their web page (as many people don't).

-- Chris Burges