
<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>

</head>

<body dir="ltr">

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

Dear Juergen and Connectionists colleagues,</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

In his attached email below, Juergen mentioned a 1972 article of my friend and colleague, Shun-Ichi Amari, about recurrent neural networks that learn.</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

Here are a couple of my own early articles from 1969 and 1971 about such networks. I introduced them to explain paradoxical data about

<b>serial verbal learning</b>, notably the <b>bowed serial position effect</b>:</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted2">Grossberg, S. (1969). On the serial learning of lists.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted2"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted2">Mathematical

 Biosciences,</em></span><em style="box-sizing:border-box;color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255)"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted2"> </span></em><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted2">4,

 201-253.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted2"> </span></span><br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted2 ContentPasted3"><a href="https://sites.bu.edu/steveg/files/2016/06/Gro1969MBLists.pdf" id="LPNoLPOWALinkPreview_1">https://sites.bu.edu/steveg/files/2016/06/Gro1969MBLists.pdf</a><br>

</span></span></div>

<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_1 _EReadonly_1"></div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><br>

</span></div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted0">Grossberg, S. and Pepe, J. (1971). Spiking threshold and overarousal effects in serial learning.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted0"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted0">Journal

 of Statistical Physics</em></span><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted0">, 3, 95-125.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted0"> </span></span><br>

</div>

<div class="elementToProof">

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

<a href="https://sites.bu.edu/steveg/files/2016/06/GroPepe1971JoSP.pdf" id="LPNoLPOWALinkPreview">https://sites.bu.edu/steveg/files/2016/06/GroPepe1971JoSP.pdf</a><br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

Juergen also mentioned that Shun-Ichi's work was a precursor of what some people call the Hopfield model, whose most cited articles were published in 1982 and 1984.</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

I actually started publishing articles on this topic starting in the 1960s. Here are two of them:</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

<span style="font-size:18pt;text-align:left;background-color:rgba(0, 0, 0, 0);display:inline !important" class="ContentPasted5">Grossberg, S. (1969). On learning and energy-entropy dependence in recurrent and nonrecurrent signed networks.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgba(0, 0, 0, 0)" class="ContentPasted5"> </span></span><span style="font-size:18pt;background-color:rgba(0, 0, 0, 0)"><em style="box-sizing:border-box;text-align:left;background-color:rgba(0, 0, 0, 0)" class="ContentPasted5">Journal

 of Statistical Physics,</em></span><em style="box-sizing:border-box;color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255)"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgba(0, 0, 0, 0)" class="ContentPasted5"> </span></em><span style="font-size:18pt;text-align:left;background-color:rgba(0, 0, 0, 0);display:inline !important" class="ContentPasted5">1,

 319-350.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgba(0, 0, 0, 0)" class="ContentPasted5"> </span></span><br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1 ContentPasted4">

<a href="https://sites.bu.edu/steveg/files/2016/06/Gro1969JourStatPhy.pdf" id="LPlnkOWALinkPreview">https://sites.bu.edu/steveg/files/2016/06/Gro1969JourStatPhy.pdf</a><br>

</div>

</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

<br>

</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

<span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted6">Grossberg, S. (1971). Pavlovian pattern learning by nonlinear neural networks.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted6"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted6">Proceedings

 of the National Academy of Sciences</em></span><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted6">, 68, 828-831.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted6"> </span></span><br>

</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

<span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted6 ContentPasted7"><a href="https://sites.bu.edu/steveg/files/2016/06/Gro1971ProNatAcaSci.pdf" id="LPlnkOWALinkPreview">https://sites.bu.edu/steveg/files/2016/06/Gro1971ProNatAcaSci.pdf</a><br>

</span></span></div>

<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1"></div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

An early use of <b>Lyapunov functions</b> to prove global limit theorems in associative recurrent neural networks is found in the following 1980 PNAS article:</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted11">Grossberg, S. (1980). Biological competition: Decision rules, pattern formation, and oscillations.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted11"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted11">Proceedings

 of the</em></span><em style="box-sizing:border-box;color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255)"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted11"> </span></em><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted11">National</em></span><em style="box-sizing:border-box;color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255)"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted11"> </span></em><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted11">Academy</em></span><em style="box-sizing:border-box;color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255)"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted11"> </span></em><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted11">of

 Sciences</em></span><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted11">,</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted11"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><strong style="box-sizing:border-box;font-weight:bold;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted11">77</strong></span><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted11">,

 2338-2342.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted11"> </span></span></div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted10">

<a href="https://sites.bu.edu/steveg/files/2016/06/Gro1980PNAS.pdf" id="LPlnkOWALinkPreview_2">https://sites.bu.edu/steveg/files/2016/06/Gro1980PNAS.pdf</a></div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

Subsequent results culminated in my 1983 article with Michael Cohen, which was in press when the Hopfield (1982) article was published:</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted9">Cohen, M.A. and Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive

 neural networks.</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted9"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><em style="box-sizing:border-box;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted9">IEEE

 Transactions on Systems, Man, and Cybernetics</em></span><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted9">,</span><span style="color:rgb(85, 85, 85);font-family:Capita;font-size:18px;text-align:left;background-color:rgb(255, 255, 255);display:inline !important"><span style="font-family:Arial, Helvetica, sans-serif;font-size:18pt;color:rgb(0, 0, 0);background-color:rgb(255, 255, 255)" class="ContentPasted9"> </span></span><span style="font-size:18pt;background-color:rgb(255, 255, 255)"><strong style="box-sizing:border-box;font-weight:bold;text-align:left;background-color:rgb(255, 255, 255)" class="ContentPasted9">SMC-13</strong></span><span style="font-size:18pt;text-align:left;background-color:rgb(255, 255, 255);display:inline !important" class="ContentPasted9">,

 815-826.</span><br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted8">

 <a href="https://sites.bu.edu/steveg/files/2016/06/CohGro1983IEEE.pdf" id="LPlnk410072">https://sites.bu.edu/steveg/files/2016/06/CohGro1983IEEE.pdf</a></div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

Our article introduced a general class of neural networks for <b>associative spatial pattern learning</b>, which included the

<b>Additive and Shunting neural networks</b> that I had earlier introduced, as well as a<b> Lyapunov function</b> for all of them. </div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

This article proved <b>global limit theorems</b> about all these systems using that Lyapunov function.</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

The Hopfield article describes the special case of the Additive model.</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

<br>

</div>

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">

His article proved no theorems.</div>

<div class="elementToProof">

<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted1">

<br>

</div>

</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

Best to all,</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

<br>

</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

Steve</div>

<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">

<br>

</div>

<div class="elementToProof">

<div id="Signature">

<div>

<div></div>

<div></div>

<div></div>

<div id="divtagdefaultwrapper" dir="ltr" style="font-size:18pt; color:#000000; font-family:Arial,Helvetica,sans-serif">

<p style="margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0"><span id="ms-rterangepaste-start"></span></p>

<div class="" style="color:rgb(0,0,0); font-style:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">

<span class="Apple-style-span" style="font-family:Arial; border-collapse:separate; border-spacing:0px">

<div class="" style=""><span class="Apple-style-span" style="orphans:2; widows:2">

<div class="" style="">

<div class="" style="font-weight:normal; border-collapse:separate; border-spacing:0px">

Stephen Grossberg</div>

<div class="" style=""><a href="http://en.wikipedia.org/wiki/Stephen_Grossberg" class="" style=""><span style="color:rgb(0,0,0)">http://en.wikipedia.org/wiki/Stephen_Grossberg</span></a></div>

<div class="" style=""><a href="http://scholar.google.com/citations?user=3BIV70wAAAAJ&hl=en" class=""><span style="color:rgb(0,0,0)">http://scholar.google.com/citations?user=3BIV70wAAAAJ&hl=en</span></a></div>

</div>

</span><span class="Apple-style-span" style="orphans:2; widows:2; color:rgb(0,0,0)"></span></div>

</span><span class="" style="orphans:2; widows:2"><a href="https://youtu.be/9n5AnvFur7I" class=""><span style="color:rgb(0,0,0)">https://youtu.be/9n5AnvFur7I</span></a></span></div>

<div class="" style="color:rgb(0,0,0); font-style:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">

<span style="color:rgb(0,0,0)"></span><a href="https://www.youtube.com/watch?v=_hBye6JQCh4" class="" style="font-family:ArialMT"><span style="color:rgb(0,0,0)">https://www.youtube.com/watch?v=_hBye6JQCh4</span></a></div>

<div class="" style="color:rgb(0,0,0); font-style:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">

<a href="https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552">https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552</a><br class="">

<span class="Apple-style-span" style="font-weight:normal; border-collapse:separate; border-spacing:0px">

<div class="" style="word-wrap:break-word"><span class="Apple-style-span" style="">

<div class="" style="word-wrap:break-word"><br class="">

Wang Professor of Cognitive and Neural Systems</div>

<div class="" style="word-wrap:break-word"><span style="font-family:Arial,Helvetica,sans-serif; background-color:rgb(255,255,255); display:inline!important">Director, Center for Adaptive Systems</span><br class="">

Professor Emeritus of Mathematics & Statistics, </div>

<div class="" style="word-wrap:break-word">       Psychological & Brain Sciences, and Biomedical Engineering<br>

</div>

<div class="" style="word-wrap:break-word">Boston University<br class="">

sites.bu.edu/steveg<br class="">

steve@bu.edu</div>

</span></div>

</span></div>

<span id="ms-rterangepaste-end"></span><br>

<p style="margin-top:0px; margin-bottom:0px"></p>

</div>

</div>

</div>

</div>

<div id="appendonsend"></div>

<hr style="display:inline-block;width:98%" tabindex="-1">

<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Schmidhuber Juergen <juergen@idsia.ch><br>

<b>Sent:</b> Wednesday, January 25, 2023 8:44 AM<br>

<b>To:</b> connectionists@cs.cmu.edu <connectionists@cs.cmu.edu><br>

<b>Subject:</b> Re: Connectionists: Annotated History of Modern AI and Deep Learning</font>

<div> </div>

</div>

<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">

<div class="PlainText">Some are not aware of this historic tidbit in Sec. 4 of the survey: half a century ago, Shun-Ichi Amari published a learning recurrent neural network (1972) which was later called the Hopfield network.<br>

<br>

<a href="https://people.idsia.ch/~juergen/deep-learning-history.html#rnn">https://people.idsia.ch/~juergen/deep-learning-history.html#rnn</a><br>

<br>

J�rgen<br>

<br>

<br>

<br>

<br>

> On 13. Jan 2023, at 11:13, Schmidhuber Juergen <juergen@idsia.ch> wrote:<br>

> <br>

> Machine learning is the science of credit assignment. My new survey credits the pioneers of deep learning and modern AI (supplementing my award-winning 2015 survey):

<br>

> <br>

> <a href="https://arxiv.org/abs/2212.11279">https://arxiv.org/abs/2212.11279</a><br>

> <br>

> <a href="https://people.idsia.ch/~juergen/deep-learning-history.html">https://people.idsia.ch/~juergen/deep-learning-history.html</a><br>

> <br>

> This was already reviewed by several deep learning pioneers and other experts. Nevertheless, let me know under juergen@idsia.ch if you can spot any remaining error or have suggestions for improvements.<br>

> <br>

> Happy New Year!<br>

> <br>

> J�rgen<br>

> <br>

<br>

<br>

</div>

</span></font></div>

</body>

</html>