Connectionists: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton

Sun Oct 20 20:27:47 EDT 2024

Michael,

Providing relevant scientific history is not a “non-answer”.

Your comment is meant to be provocative, but it is misguided.

If you are so interested in the issue that you raised, then please put your money where your mouth is.

Please answer the question that you asked me, in the manner that you want to hear it answered.

I will read your answer with interest.

Best,

Steve

From: Michael Arbib <arbib at usc.edu>
Date: Sunday, October 20, 2024 at 6:29 PM
To: Grossberg, Stephen <steve at bu.edu>, connectionists at mailman.srv.cs.cmu.edu <connectionists at mailman.srv.cs.cmu.edu>
Cc: Michael Arbib <arbib at usc.edu>
Subject: RE: Connectionists: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton
Dear Steve:

Two non-answers! My request was serious. You criticize the current selection for the Physics Prize (but do not discuss the Chemistry prize) and remind us of your many important achievements – which certainly reads as if you are making the case that your own achievements as Nobel-worthy. This led me to wonder who would be on your short list of others whose work you see as equally foundational for neural networks whether for AI or BT. Putting this in other terms, if you had written a historical book on the field rather than a personal summation, say with the title “Conscious Mind, Neuronal Brain: How Each Brain Makes a Mind,” who would be on the short list of the main living contributors other than ourselves (!), Hopfield, Hinton and Hassabis? Would, for example, you include Amari? Or Sutton and Barto for reinforcement learning? Or …? It seems to me that such an attempt would mark a worthy transition for our community from complaining about the Nobel Committee to getting our colleagues to share an interesting perspective on the key ideas of our field. Thus, your list need not (cannot!) be exhaustive but could catalyze a lively discussion.

All the best

Michael

From: Grossberg, Stephen <steve at bu.edu>
Sent: Sunday, October 20, 2024 2:40 PM
To: Michael Arbib <arbib at usc.edu>; connectionists at mailman.srv.cs.cmu.edu
Cc: Stephen Grossberg <steve at cns.bu.edu>; Michael Arbib <arbib at usc.edu>
Subject: Re: Connectionists: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton

Michael, I should add that if, as you suggested, I was also being considered for a Nobel Prize for my work in neural networks and AI, cognitive neuroscience, theoretical and mathematical psychology et al. , then I would need to recuse myself
Michael,

I should add that if, as you suggested, I was also being considered for a Nobel Prize for my work in neural networks and AI, cognitive neuroscience, theoretical and mathematical psychology et al., then I would need to recuse myself from evaluating the work of other potential candidates in those areas of expertise.

Best again,

Steve

From: Grossberg, Stephen <steve at bu.edu<mailto:steve at bu.edu>>
Date: Sunday, October 20, 2024 at 5:18 PM
To: Michael Arbib <arbib at usc.edu<mailto:arbib at usc.edu>>, connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu> <connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>>
Cc: Stephen Grossberg <steve at cns.bu.edu<mailto:steve at cns.bu.edu>>, Michael Arbib <arbib at usc.edu<mailto:arbib at usc.edu>>
Subject: Re: Connectionists: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton
Dear Michael,

It’s good to hear from you!

I hope that you and Prue are well.

The task that you mention below is too important to reply in hypotheticals.

If I am ever asked to recommend Nobel Prize winners, I would do what I have done when I served on, and chaired, tenure committees.

I would do a deep dive into the most important contributions of everyone being considered, including the work of scientists on whose shoulders they built, and go from there, in consultation with everyone on the committee.

Best as ever,

Steve

From: Michael Arbib <arbib at usc.edu<mailto:arbib at usc.edu>>
Date: Sunday, October 20, 2024 at 4:00 PM
To: Grossberg, Stephen <steve at bu.edu<mailto:steve at bu.edu>>, connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu> <connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>>
Cc: Stephen Grossberg <steve at cns.bu.edu<mailto:steve at cns.bu.edu>>, Michael Arbib <arbib at usc.edu<mailto:arbib at usc.edu>>
Subject: Re: Connectionists: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton
You don't often get email from arbib at usc.edu<mailto:arbib at usc.edu>. Learn why this is important<https://urldefense.com/v3/__https:/aka.ms/LearnAboutSenderIdentification__;!!LIr3w8kk_Xxm!qhp2qhS6Tn-W9vW8z1FjG3J-qKYAZaUyGlszq2vqINvXJm5PAdQk12gErhgjVO-7HXYuRpej7Q$>
Dear Steve

If you had to choose 3 people other than yourself working in neural networks  (AI and/or BT) to share a Nobel Prize (you can even pick the field!) who would they be?

Best wishes

Michael

Get Outlook for iOS<https://urldefense.com/v3/__https:/aka.ms/o0ukef__;!!LIr3w8kk_Xxm!qhp2qhS6Tn-W9vW8z1FjG3J-qKYAZaUyGlszq2vqINvXJm5PAdQk12gErhgjVO-7HXbU46kCoA$>
________________________________
From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu<mailto:connectionists-bounces at mailman.srv.cs.cmu.edu>> on behalf of Grossberg, Stephen <steve at bu.edu<mailto:steve at bu.edu>>
Sent: Sunday, October 20, 2024 11:16 AM
To: connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu> <connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>>
Cc: Stephen Grossberg <steve at cns.bu.edu<mailto:steve at cns.bu.edu>>
Subject: Re: Connectionists: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton

Dear Connectionists colleagues,

Here are some short summaries of the history of neural network discoveries, as I experienced it, that are relevant to the recent Nobel Prizes to Hopfield and Hinton:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

[A diagram of a scientific experiment    Description automatically generated with medium confidence]

THE NOBEL PRIZES IN PHYSICS TO HOPFIELD AND HINTON
FOR MODELS THEY DID NOT DISCOVER: THE CASE OF HOPFIELD

Here I will summarize my concerns about the Hopfield award.

I published articles in 1967 – 1972 in the Proceedings of the National Academy of Sciences that introduced the Additive Model that Hopfield used in 1984. My articles proved global theorems about the limits and oscillations of my Generalized Additive Models. See sites.bu.edu/steveg<https://urldefense.com/v3/__http:/sites.bu.edu/steveg__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLIn_Ms0Xg$> for these articles.

For example:

Grossberg, S. (1971). Pavlovian pattern learning by nonlinear neural networks. Proceedings of the National Academy of Sciences, 68, 828-831.
https://lnkd.in/emzwx4Tw<https://urldefense.com/v3/__https:/lnkd.in/emzwx4Tw__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLK4nSkiuQ$>

This article illustrates that my mathematical results were part of a research program to develop biological neural networks that provide principled mechanistic explanations of psychological and neurobiological data.

Later, Michael Cohen and I published a Liapunov function that included the Additive Model and generalizations thereof in 1982 and 1983 before Hopfield (1984) appeared.

For example,

Cohen, M.A. and Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 815-826.
https://lnkd.in/eAFAdvbu<https://urldefense.com/v3/__https:/lnkd.in/eAFAdvbu__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLJJNuUBmA$>

I was told that Hopfield knew about my work before he published his 1984 article, without citation.

Recall that I started my neural networks research in 1957 as a Freshman at Dartmouth College.

That year, I introduced the biological neural network paradigm, as well as the short-term memory (STM), medium-term memory (MTM), and long-term memory (LTM) laws that are used to this day, including in the Additive Model, to explain data about how brains make minds.

See the review in https://lnkd.in/gJZJtP_W<https://urldefense.com/v3/__https:/lnkd.in/gJZJtP_W__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLIaN5CydA$> .

When I started in 1957, I knew no one else who was doing neural networks. That is why my colleagues call me the Father of AI.

I then worked hard to create a neural networks community, notably a research center, academic department, the International Neural Network Society, the journal Neural Networks, multiple international conferences on neural networks, and Boston-area research centers, while training over 100 gifted PhD students, postdocs, and faculty to do neural network research. See the Wikipedia page.

That is why I did not have time or strength to fight for priority of my models.

Recently, I was able to provide a self-contained and non-technical overview and synthesis of some of my scientific discoveries since 1957, as well as explanations of the work of many other scientists, in my 2021 Magnum Opus

Conscious Mind, Resonant Brain: How Each Brain Makes a Mind

https://lnkd.in/eiJh4Ti<https://urldefense.com/v3/__https:/lnkd.in/eiJh4Ti__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLJFgMIrsA$>
++++++++++++++++++++++++++++++++++++++++++++++++++++
THE NOBEL PRIZES IN PHYSICS TO HOPFIELD AND HINTON
FOR MODELS THEY DID NOT DISCOVER: THE CASE OF HINTON

Here I summarize my concerns about the Hinton award.

Many authors developed Back Propagation (BP) before Hinton; e.g., Amari (1967), Werbos (1974), Parker (1982), all before Rumelhart, Hinton, & Williams (1986).

BP has serious computational weaknesses:

It is UNTRUSTWORTHY (because it is UNEXPLAINABLE).

It is UNRELIABLE (because it can experience CATASTROPHIC FORGETTING.

It should thus never be used in financial or medical applications.

BP learning is also SLOW and uses non-biological NONLOCAL WEIGHT TRANSPORT.

See Figure, right column, top.

In 1988, I published 17 computational problems of BP:
https://lnkd.in/erKJvXFA<https://urldefense.com/v3/__https:/lnkd.in/erKJvXFA__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLKBi5XbJA$>

BP gradually grew out of favor because other models were better.

Later, huge online databases and supercomputers enabled Deep Learning to use BP to learn.

My 1988 article contrasted BP with Adaptive Resonance Theory (ART) which I first published in 1976:
https://lnkd.in/evkfq22G<https://urldefense.com/v3/__https:/lnkd.in/evkfq22G__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLIQ1IwXMA$>

See Figure, right column, bottom.

ART never had BP’s problems.

ART is now the most advanced cognitive and neural theory that explains HOW HUMANS LEARN TO ATTEND, RECOGNIZE, and PREDICT events in a changing world.

ART also explains and simulates data from hundreds of psychological and neurobiological experiments.

In 1980, I derived ART from a THOUGHT EXPERIMENT about how ANY system can AUTONOMOUSLY learn to correct predictive errors in a changing world:
https://lnkd.in/eGWE8kJg<https://urldefense.com/v3/__https:/lnkd.in/eGWE8kJg__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLLRHPDd2A$>

The thought experiment derives ART from a few facts of life that do not mention mind or brain.

ART is thus a UNIVERSAL solution of the problem of autonomous error correction in a changing world.

That is why ART models can be used in designs for AUTONOMOUS ADAPTIVE INTELLIGENCE in engineering, technology, and AI.

ART also proposes a solution of the classical MIND-BODY PROBLEM:

HOW, WHERE in our brains, and WHY from a deep computational perspective, we CONSCIOUSLY SEE, HEAR, FEEL, and KNOW about the world, and use our conscious states to PLAN and ACT to realize VALUED GOALS.

For details, see

Conscious Mind, Resonant Brain: How Each Brain Makes a Mind

https://lnkd.in/eiJh4Ti<https://urldefense.com/v3/__https:/lnkd.in/eiJh4Ti__;!!LIr3w8kk_Xxm!v627G1ahi4gLbdDMaVZyydda2T9EiLgzE7QwNyJAfGtYY5CpVsdIUGb5gGOrntuYxLJFgMIrsA$>
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20241021/4491636e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 2016779 bytes
Desc: image001.png
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20241021/4491636e/attachment-0001.png>