Open Review

[Ken Laws]

> From David Tam:
> I think a totally honest system has to be doubly-open ...

I agree.  But now you're talking about a "scientific revolution."
If reviewers are not guaranteed anonymity, many -- most? -- of
the better-qualified people will refuse to review.  (Consider
legal liability, for instance.  Will professional societies
indemnify reviewers against malpractice claims?  And, liability
aside, how many of the "top people" want to spend their time
feuding with colleagues or answering challenges from offended
authors?)

Not that feuding isn't an acceptable alternative.  Louis Pasteur
feuded bitterly with opponents of his bacterial theory of anthrax
and other diseases; eventually he won.  Lister fought for
antisepsis; Jenner (and Pasteur) fought for immunization.
They won, just as Galileo won against the might of the Church.

But if that's the system, either it has to be the whole system
-- no one can escape just by boycotting one or several conferences
-- or you have to pay a few good people to become knowlegeable
critics.  There's a long tradition of professional critics
in art, drama, literature, politics, entertainment, travel
services, and fine dining.  There's only one reason that we
have no such pundits in science -- we offer no financial support
for such a career.

Professional critics have their own critics, of course --
individually and as an institution -- but that's just part of the
doubly-open review system.  I think it's healthy and I'd love
to see a scientific journalism illuminate our field.  It won't
happen on the initiative of those now in power, as they need
the shadows to keep the current system going.  (Questions about
the quality of graduate education, necessity of the research
being done, and exploitation of students and postdocs are best
not asked.  They won't lead to reform, but to funds being
withdrawn from our field.)

The revolution will happen, but through grass-roots self-publishing.
Tenure committees are still committed to counting papers in
prestigious forums, but that will change when the current
journals and conferences collapse.  Online journals and "conferences"
will take over -- or evolve from the existing channels -- but
self-publication will become an increasingly important way of
sharing results.  And with that comes the need for amateur and
professional reviewers.  Unpaid reviews will predominate within
each discipline, but paid reviewers, abstracters, journalists,
and the like will follow the discussions and report significant
findings to researchers in nearby fields -- and to funding
agencies and the general public.  Reports of these gatekeepers
will in turn be reviewed, with some being acknowledged as more
reliable than others.  Eventually the tenure committees will
start looking at the reviews rather than publication counts.

Can this work in Computer Science?  It already does, in much of
the computer hardware world.  We may not think of Infoworld or
Computerworld as part of the academic press, but they do pick
up important stories from time to time.  EE Times broke the
news of the Pentium bug, and often carries Colin Johnson's
reports on neural-network hardware advances.  Other articles
cover ARPA funding for ANN initiatives, or other news of interest
to professional researchers and developers.

What distinguishes an industry from a scientific discipline?
It is largely the presence of commercial journalism.  An industry
has at least a weekly trade magazine to keep everyone informed
of what's happening, what resources are available, and where
the jobs are.  Of course there has to be money pumping around
also, or the trade magazine couldn't flourish -- but online
journalism may be able to operate much more cheaply.  (Or may
not.  The role of advertising hasn't yet been established,
and it is advertising that pays for most trade publications.)

Before doubly-open review can take hold, with professional
journalists, columnists, and the like to contribute and
to referee, there's still a bit of pioneering to be done.
I'm working on one approach, trying to build a professional
association and publication ab initio, taylored to the
online age.  The association, Computists International,
is a mutual-aid society for AI/IS/CS researchers.  Our
flagship publication is the weekly Computists' Communique,
a cross between a newsletter and a news wire (with echoes
of Reader's Digest and Johnny Carson).  The Communique
hasn't grown enough yet to have regular columnists or
deep critical analysis of scientific controversies, but it
comes closer than most other publications.  The connectionists
disussion stream is one of many from which I draw material
on inference, pattern recognition, and related topics.

Last year, I tried to offer connectionists free issues
of the Communique -- one per month, in what I call my
Full Moon subset.  The announcement was refused by your
moderator as not being entirely related to neural-network
theory.  Assuming that this message gets past the gatekeepers,
I'd like to make the offer again.  Contact me at the
address below (or reply to this message, IF that won't
send your message back to the connectionists list).
Mention "connectionists," ask for the Full Moon subset,
and have your full name somewhere in the message.  I'll sign
you up for one free issue per month, just to introduce my
service and to keep in touch with you.  It will give you
a good look at the kind of "niche journalism" the net will
currently support.  I hope some of you will take up the
torch, starting similar publications that are specific to
your own interests.  The world would be better for having
an online high-signal newsletter devoted to connectionism.

                                -- Ken Laws

Dr. Kenneth I. Laws; Computists International; laws at ai.sri.com.
Ask about the free Full Moon subset of the Computists' Communique.
-------