proposal: archival NIPS technical report series (Re: double blind review)

[tbreuel@parc.com]

On Fri, Dec 20, 2002 at 06:51:08AM -0800, Christine Bolbirer wrote:
> If what the author is communicating in his or her paper is truly ground
> breaking and worthy of recognition then with enough effort it will be
> recognized (if not by the NIPS reviewers then the scientific community in
> general).

It may be recognized, but there is a good chance that the original author
will not be getting the credit.

The reality is that most ideas, even ground breaking ones, are "in the
air", and given a year or two, several people will come up with them.
Each rejection introduces a publication delay of several months.
And rejections are more likely the more ground breaking a new idea is.

If we have a system (and to some degree we do), where unknown researchers
have to live with a couple of rejections for their papers describing
new ideas while well-known researchers get their papers accepted right
away on the strength of their reputation, you really stack the deck very
strongly against unknown researchers being able to make a splash
with something new.

And that's the key problem: a rejection not only deprives authors
of the chance to add a publication to their resumes, more importantly,
it deprives them of the opportunity to establish precedence for their
ideas.  The latter is far worse than the former.

In practice, people try to get around this these days and establish
precedence for their ideas by putting drafts or memos on their web sites,
but that creates its own set of problems.  Those papers are not archival,
they can't be referenced, and their publication dates are not verifiable.
And if nobody cites them, even if such informal publications were to
establish precedence, what would be the point?

The way to address this problem traditionally has been to create archival
publications that do not require a review.  The National Academy of
Sciences has something like that for its members.  And larger universities
and research labs used to have archival technical report series, but many
of them have become non-archival (an archival publication is one with
a verifiable publication date and content, and expected to be available
indefinitely, usually established by having printed copies archived at
multiple libraries).

Since few people these days have the ability to publish in archival
technical report series, maybe what we need is to establish a NIPS
technical report series: something that is archival, searchable, and
has verifiable publication dates but is not peer reviewed and would not
preclude later publication in a peer reviewed conference or journal.

NIPS technical reports would replace informal and non-archival publication
of drafts and memos.  They would establish precedence for good and novel
ideas until reviewers are ready to accept them into a peer-reviewed
publication.  And unlike "Joe's memo with an uncertain publication
date on http://webhosting.com/~joe/memo.pdf", they would have a
persistent and verifiable citation and publication date.

Here at PARC, we have built something like that for internal publications.
We use cryptographic signatures to ensure the validty of publication
dates and document contents.  Abstracts and signatures (and, if desired,
content) are disseminated via an E-mail list so that many people will
have a record of the publication and its signature.  I don't know whether
it's completely tamper-proof, but it's probably no worse than the
traditional paper-based systems.

Thomas.