Connectionists: [EXT] If you believe in your work ...

Sat Jul 23 14:57:34 EDT 2022

Hi Jean-Marc, Richard, Tsvi et al.,                                                              Re: publications, 

The proposed multidimensional assessment method does sound interesting but it inherits some of the problems we encounter with the current model.
Looking at things like download volume, "number of reads" (really hard to tell what has been read, and whether it has been read carefully), number of citations can (and I predict will) elicit behaviours aimed at maximising those numbers, diminishing the reliability of the estimate when it comes to ascertaining scientific value. Number of active people in a given field or subfield, cliques that cite each other aggressively, can all inflate the estimate - this is a social problem within scientific research that requires attention. Popularity contests should not coarsen a discourse which is all about the search for truth. 

Re: paradigm shift in funding models & publications  - long overdue. All the best!

Anand Ramamoorthy

    On Saturday, 23 July 2022 at 17:39:31 BST, Fellous, Jean-Marc - (fellous) <fellous at arizona.edu> wrote:  

@ Richard, Frederick, Tsvi:

Points taken. 

Regarding funding: Delegating the funding decision to one person is probably dangerous. Note that technically, this is the case at NIH institutes (the director is the only one who makes the final decision, but in practice, that decision is built on the basis of peer review and program officer inputs), and similar at NSF (the PO has more authority in funding, but again, within limits). One other possibility for allowing innovative ideas to push through the ‘politics’ and ‘settled culture’ might be to set aside (say) 20% of the fund to randomly fund ‘sound’ short (2 years?) proposals that were placed far from funding threshold by reviewers. Injecting ‘noise’ so to speak to get out of local minima…

Regarding publications: There are so many journals out there, and the arXiv options. The issue of course is integrity, trust, reproducibility etc. To some extent this is what peer review attempts to address, but there is no guarantee. May be a hybrid system, where researchers can post their work in ArXiv type venue, but with minimal initial ‘sanity’ checks (i.e. moderated, not reviewed), followed by voluntary or invited anonymous reviews? The quality/strength of the article would be assessed multi-dimensionally by the number of reviews/responses, number of reads, number of downloads, number of citations, availability of code/data, posting of negative results… etc.

Can’t help but think we could be on the verge of a paradigm shift in publications and funding models… we need new ideas, and the courage to try them out!

Best,

Jean-Marc

From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu>On Behalf Of Richard Loosemore
Sent: Tuesday, July 19, 2022 10:40 AM
To: connectionists at mailman.srv.cs.cmu.edu
Subject: Re: Connectionists: [EXT] If you believe in your work ...

External Email

Jean-Marc,

The problem is systemic; it goes down into the very roots of modern "science".

The only solution that $20m could buy would be:

1) An institute run by someone with ethical principles, who would use the money to attract further funding until it could actually take on board researchers with creative ideas and ethical principles, and then free them from the yoke of publish-crap-in-quantity-or-perish.

2) An AI/Cognitive system development tool that would allow people to build and explore complex cognitive systems without being shackled to one particular architecture (like deep learning and its many descendents).

A propos of (2) that is one thing I proposed in a (rejected) grant proposal. It would have cost $6.4m.

Best,

Richard

-- 
Richard Loosemore
Cornell University
...
rpl72 at cornell.edu

On 7/19/22 11:31 AM, Fellous, Jean-Marc - (fellous) wrote:

Assuming there are funders on the list, and funding-related people, including program officers (and believe or not, there are!): if you had $20M to invest in the sort of things we do on this list: how would we make things better? Can we brainstorm an alternative system that allows for innovating publications and effective funding?

Jean-Marc

From: Connectionists<connectionists-bounces at mailman.srv.cs.cmu.edu> on behalf of Richard Loosemore<rloosemore at susaro.com>
Sent: Monday, July 18, 2022 1:28 PM
To: connectionists at mailman.srv.cs.cmu.edu<connectionists at mailman.srv.cs.cmu.edu>
Subject: [EXT]Connectionists: If you believe in your work ... 

External Email

On 7/17/22 11:52 AM, Grossberg, Stephen wrote:

> ... if you believe in your work, and the criticisms of it are not valid, do not give up. ...

> ... all criticisms by reviewers are valuable and should be taken into account in your revision.

> Even if a reviewer's criticisms are, to your mind, wrong-headed, they represent the
> viewpoint of a more-than-usually-qualified reader who has given you the privilege
> of taking enough time to read your article.

Really?

1) I believe in my work, and the criticisms of it are not valid.  I did not give up, and the net result of not giving up was ... nothing.

2) No reviewer who has ever commented on my work has shown the slightest sign that they understood anything in it.

3) Good plumbers are more than usually qualified in their field, and if one of those gave you the privilege of taking enough time to read your article and give nonsensical comments, would you pay any attention to their viewpoint?

** - **

I have spent my career fighting against this system, to no avail.

I have watched charlatans bamboozle the crowd with pointless mathematics, and get published.

I have watched people use teams of subordinates to pump out streams of worthless papers that inflate their prestige.

I have written grant proposals that were exquisitely tuned to the stated goal of the grant, and then watched as the grant money went to people whose proposals had only the faintest imaginable connection to the stated goal of the grant.

** - **

The quoted remarks, above, somehow distilled all of that history and left me shaking with rage at the stupidity.

I have been a member of the Connectionists mailing list since the early 1990s, and before that I had been working on neural nets since 1980.

No more.

Best,

Richard

-- 

Richard Loosemore

Cornell University

...

rpl72 at cornell.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220723/588f9dbf/attachment.html>