referee idea

[Ivo Kwee]

Hi,

I saw the postings on connectionists about referees. As a matter of 
fact, I bounced a related idea of open review for JMLR to Leslie just a 
week ago. I think it is relevant and might be in fact feasible [hope L. 
doesn't mind me quoting her...]. About L's comment on "sifting": 
additionally we can keep a personal list of what are the latest reviewed 
papers of some (important) person, personal "recommendations" (see 
Amazon), reviewers ranking, global download statistics, online 
versioning etc. The good thing about "the Amazon way" is that you can 
also read the reviewers comment themselves, which is mostly not 
available in most journals but is quite useful (as an exception I think 
the J. of Am. Statistics does include reviewers comments).

Ivo Kwee
IDSIA



Ivo Kwee wrote:

> Leslie,
>
> Why not make JMRL even more radical? Get away with an Editorial Board 
> at all? Do a referee system like Amazon.com, publish papers 
> immediately and let *everyone* act as referee immediately, but also 
> give points to referees themselves (start the current Board members as 
> "veterans"). This gives also fair credit to student/researchers that 
> referee papers on behalf of someone.
>
> What do you think?
>
> Ivo
>

Leslie Pack Kaelbling wrote:

> It's an interesting idea.  Might be fun to try in parallel with the 
> usual system (you can start a new journal that works this way!).
>
> I guess I'm ultimately an elitist.  I think that a minority of the 
> community have better insight, understanding, and taste than the rest, 
> and that they should decide what gets published.
>
> Published is probably the wrong word here.  In some sense, because of 
> things like eprint archives, everybody can publish their own work, 
> which is great and important.  So I see the role of journals as really 
> giving an imprimatur.  Some group of people thinks these (few) papers 
> are good.
>
> As more and more information becomes available, we'll even pay for 
> people to sift it for us, and find the good parts.  Of course, "good" 
> to me is "bad" to someone else, and so that other community should 
> find  some other editor to sift out the stuff they like.
>
> The reason I subscribe to some magazines is that my taste is aligned 
> with that of the editors.  Even if a huge superset of that material 
> were available online, I'd pay for the (paper or electronic) magazine, 
> because I don't have time to do the sifting for myself.
>
> - L
>