reviewing

[Prof. Michael Stiber]

Geoffrey Hinton writes:
 > 
 > In the old days, there had to be a way to decide what to print because
 > printing and circulation were bottlenecks.  But now that we have the
 > web, the problem is clearly how to decide what to read.  It is very

This is the basis of one of the recurring themes of arguments over
"modernizing" the publication review process.  Suggestions usually
revolve around letting something like market forces decide which
papers are really most significant.  This is based on the assumption
that the only motivation for the current review system is filtering,
and that we can all save ourselves some time by using technology to
speed the filtering process (perhaps by distributing it among a large
number of independently-operating processors, each doing a much
smaller amount of work than current reviewers do :^)).

For many (most?) people publishing, however, peer review serves
another purpose: it provides an ongoing, relatively objective
performance review.  This is done incrementally, so the review is
already distributed along time and among a number of people.  When
someone is asked to review a package of information about a colleague
for promotion or tenure, for example, it is perfectly reasonable to
rely on past reviews --- publication record --- to give a broad
overview of _one aspect_ of performance.  Lacking that, I suppose a
conscientious reviewer would be faced with the enormous task of
carefully reading a good fraction of the candidate's work.  The same
would be true, if less extensive, for annual performance reviews.  In
effect, all that time "saved" in streamlining the paper review process
would just pop up elsewhere.

Mike Stiber

-- 
Prof. Michael Stiber                    stiber at u.washington.edu
Computing and Software Systems          http://faculty.washington.edu/stiber
University of Washington, Bothell       tel: +1-425-352-5280
Box 358534, 18115 Campus Way NE         fax: +1-425-352-5216
Bothell, WA 98011-8246 USA