<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:Capita;
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
p.MsoHeader, li.MsoHeader, div.MsoHeader
{mso-style-priority:99;
mso-style-link:"Header Char";
margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
p.MsoFooter, li.MsoFooter, div.MsoFooter
{mso-style-priority:99;
mso-style-link:"Footer Char";
margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.gmailsignatureprefix
{mso-style-name:gmailsignatureprefix;}
span.EmailStyle24
{mso-style-type:personal-reply;
font-family:"Arial",sans-serif;
color:windowtext;}
span.HeaderChar
{mso-style-name:"Header Char";
mso-style-priority:99;
mso-style-link:Header;
font-family:"Calibri",sans-serif;}
span.FooterChar
{mso-style-name:"Footer Char";
mso-style-priority:99;
mso-style-link:Footer;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
/* Page Definitions */
@page
{mso-endnote-separator:url("cid:header.htm\@01DB1A28.5B7EEB20") es;
mso-endnote-continuation-separator:url("cid:header.htm\@01DB1A28.5B7EEB20") ecs;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;
mso-footer:url("cid:header.htm\@01DB1A28.5B7EEB20") f1;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="2050" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="2" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="#EDEAC4" lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Dear Steve,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Actually, the biological neural networks that I derived to explain psychological and neurobiological data do have a statistical mechanical interpretation.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">The network activities, or short-term memory (STM) traces, are the statistical means of cell or population activities. The long-term memory (LTM) traces are also population means.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Using deterministic, instead of statistical, differential equations was justified by the hypothesis that the variances were small compared to the means.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">In fact, in 1964, the Rockefeller Institute for Medical Research distributed the monograph that I wrote when I entered there as a graduate student that year. We were asked to
write an essay, but I felt the need to bring together a lot of my discoveries since 1957 into one place.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">I called the monograph:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">The Theory of Embedding Fields with Applications to Psychology and Neurophysiology.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">For those curious about neural network history, you can download it from my web page:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><a href="https://sites.bu.edu/steveg/files/2016/06/Gro1964EmbeddingFields.pdf">https://sites.bu.edu/steveg/files/2016/06/Gro1964EmbeddingFields.pdf</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">In it, you will see my early struggles to deal with this statistical mechanical issue.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">In fact, when I came to Rockefeller, one of my professors was George Uhlenbeck, a distinguished statistical physicist. I went to George to explain to him the sense in which my
neural networks were a form of <i>nonstationary</i> statistical mechanics, nonstationary because they describe a theory of learning.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">George listened patiently until I was finished and then proclaimed: “Grossberg, physics is closed”.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">I write about some of this historical background in a new book that Oxford University Press will soon publish.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Neural network modeling has come a long way since then, including the development of neural networks that are composed of interacting
<i>spiking</i> neurons that can do learning, perception, and spatial navigation. When one averages properly across populations of these spiking neurons, one gets back the familiar deterministic neural networks, and without a loss of functional properties.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">For example:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:"Capita",serif;color:#555555;background:white"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Grossberg, S. and Versace, M. (2008). Spikes, synchrony, and attentive learning by laminar thalamocortical circuits. <i>Brain Research</i>, <b>1218</b>, 278-312. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><a href="https://sites.bu.edu/steveg/files/2016/06/GroVer2008BR.pdf">https://sites.bu.edu/steveg/files/2016/06/GroVer2008BR.pdf</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Cao, Y., and Grossberg, S. (2012). Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: A method of converting neural rate models into spiking
models. <i>Neural Networks</i>, <b>26</b>, 75-98. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><a href="https://sites.bu.edu/steveg/files/2016/06/CaoGroTR2011.pdf">https://sites.bu.edu/steveg/files/2016/06/CaoGroTR2011.pdf</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Pilly, P.K., and Grossberg, S. (2013). Spiking neurons in a hierarchical self-organizing map model can learn to develop spatial and temporal properties of entorhinal grid cells
and hippocampal place cells. <i>PLOS ONE,<o:p></o:p></i></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0060599<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Best again,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Steve<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Stephen José Hanson <jose@rubic.rutgers.edu><br>
<b>Date: </b>Wednesday, October 9, 2024 at 7:44 AM<br>
<b>To: </b>Grossberg, Stephen <steve@bu.edu>, Brad Wyble <bwyble@gmail.com><br>
<b>Cc: </b>Jonathan D. Cohen <jdc@princeton.edu>, Connectionists <connectionists@cs.cmu.edu>, Stephen Grossberg <steve@cns.bu.edu><br>
<b>Subject: </b>Re: Connectionists: 2024 Nobel Prize in Physics goes to Hopfield and Hinton: History of relevant discoveries before Hopfield<o:p></o:p></span></p>
</div>
<p>Steve, we agree on the fact that the "Scientific committee of the Nobel commitee" didn't know the NN history very well. If you look at the Committee's presentation, the "deep expert" seemed to be trying in a thin way to make sure that Hopfield->Hinton could
be characterized as a "statistical physics" approach. But it was clear that the LLM "breakthrough" was what was on Sweedish Academy of Sciences' mind, which one would be hard to make a case for Hopfield. But for dynamical NN systems in the context of
brain function would seem to include you and Terry and some others I can think of.<o:p></o:p></p>
<p>As to who invented BP, again, I simply repeat the obvious. Context matters. Its not about applications (speech recog, object recog etc.), in this case its about brain modeling. I 've read Paul's thesis and as mathematical statistics, it clearly has the
same mechanics, but the innovations to brain modeling and "hidden units" are not the focus, and I am saying that makes the difference.<o:p></o:p></p>
<p>Best,<o:p></o:p></p>
<p>Steve<o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On 10/8/24 9:14 PM, Grossberg, Stephen wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Dear Steve,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Thanks for your detailed reply.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">I believe that what I wrote about the 1974 work of Paul Werbos, which also developed computational examples, gives him modern priority for back propagation.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Mentioning lots of other applications does not diminish Paul’s priority.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">What Paul did not do well was to market his work.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">But good marketing does not imply priority, just as Voltaire’s “marketing” of the discoveries of Newton on the Continent did not diminish Newton’s priority. Just Google to find
the following quote:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">“Yes, Voltaire played a significant role in spreading Isaac Newton's scientific ideas across the European continent, particularly through his writings like "Letters from England"
which highlighted Newton's work and helped popularize Newtonian physics in France and beyond.”</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">I also am concerned about the Hopfield award because, again, the Nobel Prize committee does not seem to know the scientific history
behind that.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">I published several articles in 1967 – 1972 in the
<i>Proceedings of the National Academy of Sciences</i> that introduced the Additive Model that Hopfield used in 1984. This work proved global theorems about the limits and oscillations of my Generalized Additive Models. See sites.bu.edu/steveg for downloadable
versions of these articles. For example:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Grossberg, S. (1971). Pavlovian pattern learning by nonlinear neural networks. <i>Proceedings of the National Academy of Sciences</i>, 68, 828-831. </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"><a href="https://sites.bu.edu/steveg/files/2016/06/Gro1971ProNatAcaSci.pdf">https://sites.bu.edu/steveg/files/2016/06/Gro1971ProNatAcaSci.pdf</a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">You will note from the title of this article that the model arose from my discovery and development of biological neural networks to provide principled mechanistic
explanations of increasingly large psychological and neurobiological data bases. One of my early loves were animal conditioning and associative learning, to whose clarification the above article, one in a series, contributed.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Building on these discoveries, Michael Cohen and I published a Liapunov function that included the Additive Model, Shunting Model, and major generalizations thereof
in 1982 and 1983 before the 1984 Hopfield article appeared. For example,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Cohen, M.A. and Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. <i>IEEE Transactions
on Systems, Man, and Cybernetics</i>, <b>SMC-13</b>, 815-826. </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"><a href="https://sites.bu.edu/steveg/files/2016/06/CohGro1983IEEE.pdf">https://sites.bu.edu/steveg/files/2016/06/CohGro1983IEEE.pdf</a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Significantly, we proved mathematically that our generalized Liapunov function worked in all cases.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">This 1983 article was part of a series of earlier articles in which I proved theorems about the limiting and oscillatory behavior of increasingly general neural
networks that arose in multiple kinds of applications.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">One does not pull such a result out of a hat.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">In this regard, I was told that Hopfield knew about my work before he published his 1984 article.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">I was also alerted before his publication by colleagues of mine that Hopfield had just presented a public lecture about my model, declaring it a major breakthrough.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">At that time in my life, I was trying to manage a storm of creative ideas that thrust me into multiple topics of neural networks. One example, was my 1978 Theory
of Human Memory that laid out a major research program that took several decades to fully develop with multiple collaborators.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Please keep in mind that I started my research in neural networks in 1957 as a Freshman at Dartmouth College when I was taking Introductory Psychology.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">That year, I introduced the paradigm of using nonlinear systems of neural networks, as well as the short-term memory (STM), medium-term memory (MTM), and long-term
memory (LTM) laws that are used in one specialization or another to this day, including in the Additive Model, to computationally explain data about how our brains make our minds. See the historical summary in
<a href="https://en.wikipedia.org/wiki/Stephen_Grossberg">https://en.wikipedia.org/wiki/Stephen_Grossberg</a> .</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">I was not thinking about priority. I was trying to develop the latest in a steady stream of new ideas.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">When I started in 1957, I knew no one else who was doing neural networks. That is why various of my colleagues call me the Father of AI.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Because I was intellectually lonely, and wanted others to enjoy and be able to contribute to what I viewed as a paradigm-shifting breakthrough, I felt strongly
the need to help create a neural networks community. </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">That challenge thrust me into founding infrastructure for the field of neural networks, including a research center, academic department, the International Neural
Network Society, the journal <i>Neural Networks</i>, multiple international conferences on neural networks, and Boston-area research centers, while I began training over 100 gifted PhD students, postdocs, and faculty to do neural network research. See the
Wikipedia page for a summary.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">I did not feel that I had the strength to fight priority battles on top of all my educational, administrative, and leadership responsibilities.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">As some of you know, I was recently able to provide a self-contained and non-technical overview and synthesis of some of my scientific discoveries since 1957, as
well as explanations of the work of hundreds of other scientists, in my 2021 Magnum Opus</span><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span></b><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Conscious Mind, Resonant Brain: How Each Brain Makes a Mind</span></b><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"><a href="https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552">https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552</a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">I was grateful to learn that the book won the 2022 PROSE book award in Neuroscience of the Association of American Publishers.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Best to all,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#212121">Steve</span><o:p></o:p></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Brad Wyble <a href="mailto:bwyble@gmail.com">
<bwyble@gmail.com></a><br>
<b>Date: </b>Tuesday, October 8, 2024 at 7:56 PM<br>
<b>To: </b>Stephen José Hanson <a href="mailto:jose@rubic.rutgers.edu"><jose@rubic.rutgers.edu></a><br>
<b>Cc: </b>Grossberg, Stephen <a href="mailto:steve@bu.edu"><steve@bu.edu></a>, Jonathan D. Cohen
<a href="mailto:jdc@princeton.edu"><jdc@princeton.edu></a>, Connectionists <a href="mailto:connectionists@cs.cmu.edu">
<connectionists@cs.cmu.edu></a><br>
<b>Subject: </b>Re: Connectionists: 2024 Nobel Prize in Physics goes to Hopfield and Hinton</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">We really can trace the current AI boom back to John Carmack who wrote Doom, which ushered in the era of GPU-hungry computing. Credit where it's due please. </span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On Tue, Oct 8, 2024 at 4:10 PM Stephen José Hanson <</span><a href="mailto:jose@rubic.rutgers.edu"><span style="font-size:11.0pt">jose@rubic.rutgers.edu</span></a><span style="font-size:11.0pt">> wrote:</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p>Hi Steve,<o:p></o:p></p>
<p style="margin-bottom:0in">The problem every writer encounters is what can be concluded as resolved knowledge rather then new/novel knowledge. In the law this is of course “legal precedence”, so does the reference refer to a recent precedent, or does the
one for the 17<sup>th</sup> century hold precedence? In the present case, I agree that calculating gradients of functions using the chain rule was invented (Legendre -- Least squares) far before Rumelhart and Hinton applied it to error gradients in acyclic/cyclic
networks, and of course there were others as you say, in the 20<sup>th</sup> century that also applied error gradient to networks (Parker, Le cun et al). Schmidhuber says all that matters is the “math” not the applied context. However, I seriously doubt that
Legendre could have imagined using gradients of function error through succesive application in a acylic network would have produced a hierarchical kinship relationship (distinguishing between an italian and english family mother, fathers, sons, aunts, grandparents
etc.) in the hidden units of a network, simply by observing individuals with fixed feature relations. I think any reasonable person would maintain that this application is completely novel and could not be predicted in or out of context from the “math” and
certainly not from the 18<sup>th</sup> century. Hidden units were new in this context and their representational nature was novel, in this context. Scope of reference is also based on logical or causal proximity to the reference. In this case, referencing
Darwin or Newton in all biological or physics papers should be based on the outcome of the metaphorical test of whether the recent results tie back to original source in some direct line, for example, was Oswald’s grandfather responsible for the death of President
John F. Kennedy? Failing this test, suggests that the older reference may not have scope. But of course this can be subjective.<o:p></o:p></p>
<p style="margin-bottom:0in">Steve<o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On 10/8/24 2:38 PM, Grossberg, Stephen wrote:</span><o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Actually, Paul Werbos developed back propagation into its modern form, and worked out computational examples, for his 1974 Harvard PhD thesis.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Then David Parker rediscovered it in 1982, etc.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">Schmidhuber provides an excellent and wide-ranging history of many contributors to Deep Learning and its antecedents:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.sciencedirect.com/science/article/pii/S0893608014002135?casa_token=k47YCzFwcFEAAAAA:me_ZGF5brDqjRihq5kHyeQBzyUMYBypJ3neSinZ-cPn1pnyi69DGyM9eKSyLsdiRf759I77c7w" target="_blank"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">https://www.sciencedirect.com/science/article/pii/S0893608014002135?casa_token=k47YCzFwcFEAAAAA:me_ZGF5brDqjRihq5kHyeQBzyUMYBypJ3neSinZ-cPn1pnyi69DGyM9eKSyLsdiRf759I77c7w</span></a><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif">This article has been cited over 23,000 times.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:16.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<div id="m_-23520673755399983mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Connectionists </span><a href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu" target="_blank"><span style="font-size:12.0pt"><connectionists-bounces@mailman.srv.cs.cmu.edu></span></a><span style="font-size:12.0pt;color:black">
on behalf of Stephen José Hanson </span><a href="mailto:jose@rubic.rutgers.edu" target="_blank"><span style="font-size:12.0pt"><jose@rubic.rutgers.edu></span></a><span style="font-size:12.0pt;color:black"><br>
<b>Date: </b>Tuesday, October 8, 2024 at 2:25 PM<br>
<b>To: </b>Jonathan D. Cohen </span><a href="mailto:jdc@princeton.edu" target="_blank"><span style="font-size:12.0pt"><jdc@princeton.edu></span></a><span style="font-size:12.0pt;color:black">, Connectionists
</span><a href="mailto:connectionists@cs.cmu.edu" target="_blank"><span style="font-size:12.0pt"><connectionists@cs.cmu.edu></span></a><span style="font-size:12.0pt;color:black"><br>
<b>Subject: </b>Re: Connectionists: 2024 Nobel Prize in Physics goes to Hopfield and Hinton</span><o:p></o:p></p>
</div>
<p>Yes, Jon good point here, and although there is a through line from Hopfield to Hinton and Sejnowski.. Ie boltzmann machines and onto DL and LLMs<o:p></o:p></p>
<p>Dave of course invented BP, Geoff would always say.. his contribution was to try and talk Dave out of it as it had so many computational problems and could be in no way considered biologically plausible.<o:p></o:p></p>
<p>Steve<o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On 10/8/24 8:47 AM, Jonathan D. Cohen wrote:</span><o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<pre>I’d like to add, in this context, a note in memoriam of David Rumelhart, who was an integral contributor to the work honored by today’s Nobel Prize.<o:p></o:p></pre>
<pre> <o:p></o:p></pre>
<pre>jdc<o:p></o:p></pre>
<pre> <o:p></o:p></pre>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>Stephen José Hanson<o:p></o:p></pre>
<pre>Professor, Psychology Department<o:p></o:p></pre>
<pre>Director, RUBIC (Rutgers University Brain Imaging Center)<o:p></o:p></pre>
<pre>Member, Executive Committee, RUCCS<o:p></o:p></pre>
</div>
</div>
</div>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>Stephen José Hanson<o:p></o:p></pre>
<pre>Professor, Psychology Department<o:p></o:p></pre>
<pre>Director, RUBIC (Rutgers University Brain Imaging Center)<o:p></o:p></pre>
<pre>Member, Executive Committee, RUCCS<o:p></o:p></pre>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><br clear="all">
</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix"><span style="font-size:11.0pt">--
</span></span><o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Brad Wyble (he/him)</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>Stephen José Hanson<o:p></o:p></pre>
<pre>Professor, Psychology Department<o:p></o:p></pre>
<pre>Director, RUBIC (Rutgers University Brain Imaging Center)<o:p></o:p></pre>
<pre>Member, Executive Committee, RUCCS<o:p></o:p></pre>
</div>
</div>
</div>
</body>
</html>