Connectionists: Rising Stars in AI Symposium at KAUST: Protocol Flaws

Wed Mar 16 01:48:59 EDT 2022

Thank you Juyang,

" Please suggest what we should do, as much resource is being wasted."

A suggestion: Have a student or a group of students redo the studies (at
least those for which the code and data are available) and publish the
correct performance results.

Write down: The original study reported the performance of A. The true
performance is B. And B < A.

This is a lot of work. I know.

Would you say that this entire issue is a part of the general replicability
crisis in science?

Danko

Dr. Danko Nikolić
www.danko-nikolic.com
https://www.linkedin.com/in/danko-nikolic/
--- A progress usually starts with an insight ---

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Mar 16, 2022 at 1:00 AM Juyang Weng <juyang.weng at gmail.com> wrote:

> Dear Danko,
>
> Since this issue is rampant in AI, I take the liberty to give a CC to
> connectionist.
> (1) Misleading.  Yes, I agree with you.   Before I was using fraud and
> research misconduct, but Nature EIC suggested using a softer term which I
> agreed to.
> (2) Typo: thank you immensely.
> (3) "p-value in statistical science":  I agree with you.  My point is to
> show the necessity. They are not the same.
> (4) "How about other papers in Nature that you did not mention?"  I might
> miss some machine learning papers in Nature since 2015.  But almost all
> machine learning papers since 2015 in Nature and Science suffer from the
> protocol flaw of Post-Selections Using Test Sets.  This protocol flaw is
> many more machine learning papers that use neural networks,
> reinforcement learning, swarm mode, reservoir mode, evolutionary mode, and
> so on.   Please suggest what we should do, as much resource is being wasted.
> Best regards,
> -John
>
> On Mon, Mar 14, 2022 at 5:54 AM Danko Nikolic <danko.nikolic at gmail.com>
> wrote:
>
>> Dear Juyang,
>>
>> I just read your post and immediately read the linked paper. I find it
>> interesting. Thank you for that. I am amazed that the extent of the problem
>> is so high.
>>
>> I also learned something. I am now completing one study and I realize I
>> did not decouple the random seeds. Thank you for pointing out the need for
>> such decoupling.
>>
>> Here some comments from my side:
>>
>>  - you mention that the results are "misleading", but it seems to me that
>> a more strong term should be used: the results are "too optimistic". The
>> real performances are worse than what has been reported.
>>
>>   - found a typo: at one place "ransom seed" should be "random seed".
>>
>>    - a minor comment: You say: " This is similar to, but more transparent
>> than, so-called p-value in statistical science. " I would say that  the two
>> things are not too similar. Reporting average + SD is more similar to
>> standard error of the measurement.
>>
>>   Finally, I have a question: How about other papers in Nature that you
>> did not mention? Are there any other papers that have done it right? Should
>> they be mentioned too? Or in other words, do you have any impression of
>> which percentage of papers have done it right? Is it maybe that all of the
>> papers are misleading? Or maybe the split is 50-50?
>>
>> I hope you don't mind me sending my thoughts.
>>
>> Best,
>>
>> Danko
>>
>>
>>
>>
>>
>>
>> Dr. Danko Nikolić
>> www.danko-nikolic.com
>> https://www.linkedin.com/in/danko-nikolic/
>> --- A progress usually starts with an insight ---
>>
>>
>> On Mon, Mar 14, 2022 at 7:53 AM Juyang Weng <juyang.weng at gmail.com>
>> wrote:
>>
>>> Dear Juergen,
>>> Your service is appreciated but what you are doing is risky.
>>> I predict that a large number of them, if not all, are "rising stars" of
>>> protocol flaws.
>>> Please read why:
>>> ------------------------------
>>> Message: 2
>>> Date: Thu, 10 Mar 2022 17:21:37 -0500
>>> From: Juyang Weng <juyang.weng at gmail.com>
>>> To: Post Connectionists <connectionists at mailman.srv.cs.cmu.edu>
>>> Subject: Connectionists: A challenge to Post-Selections in Deep
>>>         Learning
>>> Message-ID:
>>>         <CAJmX=
>>> 6Bx139Ux0iA5PwvEEPzjyXiUcU+WQAcUSO9oOQiSCnkxA at mail.gmail.com>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Through a review of AI papers published in Nature since 2015, this report
>>> discusses the technical flaws called Post-Selection in the charged
>>> papers.
>>> This report suggests the appropriate protocol, explains reasons for the
>>> protocol, why what the papers have done is inappropriate and therefore
>>> yields misleading results. The charges below are applicable to whole
>>> systems and system components, and in all learning modes, including
>>> supervised, reinforcement, and swarm learning modes, since the concepts
>>> about training sets, validation sets, and test sets all apply. A
>>> reinforcement-learning algorithm includes not only a handcrafted form of
>>> task-specific, desired answers but also values of all answers, desired
>>> and
>>> undesired. A supervised learning method typically does not provide values
>>> for intermediate steps (e.g., hidden features), but in contrast, a
>>> reinforcement learning mode must provide values for intermediate steps
>>> using a greedy search (e.g., time discount). Casting dice is the key
>>> protocol flaw that owes a due transparency about all losers (e.g., how
>>> good
>>> they are). A commercial product is impractical if it requires every
>>> customer to cast dice and almost all trained ?lives? must cause accidents
>>> and be punished by deaths except the luckiest ?life?. All the losers and
>>> the luckiest are unethically determined by so called ?unseen? (in fact
>>> should be called ?first seen?) test sets but the human programmer saw all
>>> the scores before he decided who are losers and who is the luckiest.
>>> Such a
>>> deep learning methodology gives no product credibility.
>>>
>>>
>>> http://www.cse.msu.edu/%7eweng/research/2021-06-28-Report-to-Nature-specific-PSUTS.pdf
>>> <http://www.cse.msu.edu/~weng/research/2021-06-28-Report-to-Nature-specific-PSUTS.pdf>
>>>
>>> ----
>>> Date: Fri, 11 Mar 2022 04:57:02 +0000
>>> From: Schmidhuber Juergen <juergen at idsia.ch>
>>> To: "connectionists at cs.cmu.edu" <connectionists at cs.cmu.edu>
>>> Subject: Connectionists: Rising Stars in AI Symposium at KAUST
>>> Message-ID: <A95E83F0-0E51-451F-B532-BF0FB9DF0CCD at supsi.ch>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> The AI Initiative at KAUST is hosting the inaugural "Rising Stars in AI
>>> Symposium" at KAUST from March 13-15. This event is geared towards young
>>> researchers (including Ph.D. students, PostDocs and young faculty), who
>>> have recently published promising work at leading AI venues. There will be
>>> dozens of brief in-person presentations about papers recently accepted at
>>> major AI conferences such as NeurIPS, CVPR, EMNLP, ACL, ICML, ICLR, etc.
>>> All speakers will start with an intro for non-AI experts.
>>>
>>> To view the complete program and for more event details, please visit
>>> the symposium website:
>>>
>>> https://cemse.kaust.edu.sa/ai/aii-symp-2022
>>>
>>> The symposium will be limited to in-person attendance. So, if you are
>>> interested in joining the event, please register here:
>>>
>>>
>>> https://docs.google.com/forms/d/e/1FAIpQLSfNnc3N9sGJwkRfePzsZakQSkunhxRadnecGSOd1m7-F7At-A/viewform?hl=en
>>>
>>> We will try our best to accommodate those who register to attend in
>>> person.
>>>
>>> J?rgen Schmidhuber
>>> Director, AI Initiative, KAUST
>>> https://people.idsia.ch/~juergen/kaust-2021.html
>>> https://people.idsia.ch/~juergen/kaust-2021-hiring.html
>>>
>>> --
>>> Juyang (John) Weng
>>>
>>
>
> --
> Juyang (John) Weng
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220316/80680b4a/attachment.html>