<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"></div><div dir="ltr">I am just old enough to appreciate all this and young enough not to have met any of them. My late father took Fortran (on punch cards) from John McCarthy at MIT, and McCarthy (who I did later meet) left in the middle of the semester, having just invented LISP. </div><div dir="ltr"><br><blockquote type="cite">On Jan 17, 2023, at 15:01, Grossberg, Stephen <steve@bu.edu> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Dear Gary, Michael, and other Connectionists colleagues,</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Michael's reminiscences remind me of related memories.</div>
<div class="elementToProof">
<div id="Signature">
<div>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:18pt; color:#000000; font-family:Arial,Helvetica,sans-serif">
<div class="" style="color:rgb(0,0,0); font-style:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<span class="Apple-style-span" style="font-weight:normal; border-collapse:separate; border-spacing:0px">
<div class="" style="word-wrap:break-word"><span class="Apple-style-span" style=""></span></div>
</span></div>
<span id="ms-rterangepaste-end"></span><br>
<p style="margin-top:0px; margin-bottom:0px"></p>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
I was traveling from Stanford in 1964 to MIT to study with Norbert Wiener when I heard that Wiener had just died. I was a PhD student at Stanford then and was hoping to find a more congenial place at MIT to continue my work on neural networks.</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
<br>
</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
Instead, I got my PhD at the Rockefeller Institute for Medical Research in New York, where Abe Pais, who worked for Niels Bohr and was a colleague of Albert Einstein at the Institute for Advanced Study, was on the faculty. Abe regaled us with rather colorful
stories about both Bohr and Einstein.</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
<br>
</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
Later, when I was an assistant professor at MIT, I got to know Norman Levinson and his wife, Fagi, very well. Norman was Wiener's most famous student:</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Norman-5FLevinson&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=Y1JYLzdcDoSmpHQwk-rqybKXqInvvLKYY1kOvYoMiok&e=" id="LPNoLPOWALinkPreview_1" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt;">https://en.wikipedia.org/wiki/Norman_Levinson</a><br>
</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof ContentPasted1">
<span style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 18pt;">Norman and Fagi told me lots of stories about Wiener's many famous idiosyncrasies. </span><br>
</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
<br>
</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
While I was a young professor at MIT, Norman and Fagi generously treated me as their scientific godson and took me under their wing both at their home and at many scientific conferences. After Norman's death, Fagi became our daughter's god grandmother and shared
many happy family celebrations with us.</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
<br>
</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
For an interesting and heartwarming story about Fagi's impact on the mathematics community to which she was connected through Norman, and thus Wiener, see:</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof ContentPasted0">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__news.mit.edu_2010_obit-2Dlevinson&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=uKihsMN0Ynd0WkGoAO4bDdn5Qe7EoUmsYKFVX5FUTFg&e=" id="LPNoLPOWALinkPreview">https://news.mit.edu/2010/obit-levinson</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1"></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3">
I also had the good luck to meet Warren McCulloch</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Warren-5FSturgis-5FMcCulloch&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=rwamaAhUn_ITk-YlmIZximRviDFml0IDrxBGDmRKPBU&e=" id="LPNoLPOWALinkPreview_2">https://en.wikipedia.org/wiki/Warren_Sturgis_McCulloch</a></div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_2 _EReadonly_1"></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted2">
and Jerry Lettvin <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Jerome-5FLettvin&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=iAYkX1gbtU0YrayVwSmog1bCLtdLWUiiGl2N-3qiIYE&e=" id="LPlnkOWALinkPreview_1">https://en.wikipedia.org/wiki/Jerome_Lettvin</a></div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_1 _EReadonly_1"></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
when I got to MIT. </div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I remember wandering about Warren's lab at the Research Lab of Electronics with my friend, Stu Kauffman, who was then working with <span style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 18pt;">McCulloch: </span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Stuart-5FKauffman&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=b_0FxVN_5eo_InE_82lk47ENvvj3pUqPuNHIPO0SLyo&e=" id="LPNoLPOWALinkPreview_4" style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt;">https://en.wikipedia.org/wiki/Stuart_Kauffman</a>.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted5">
I met Stu at Dartmouth, where I began my work in neural networks as a freshman in 1957. We have remained close friends to the present time.</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_4 _EReadonly_1"></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Jerry and his wife, Maggie, also looked after me and invited me to dinner parties at their home. Maggie became quite a famous person in her own right and was a major role model for exercise, health, and women in general:</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted4">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Maggie-5FLettvin&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=aE4ZPhrItmznYh2VFtJ75ODBFxICFXj4tf_eLQ11F3A&e=" id="LPNoLPOWALinkPreview_3">https://en.wikipedia.org/wiki/Maggie_Lettvin</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_3 _EReadonly_1"></div>
<br>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I will never forget the generosity and kindness of these incredibly talented people.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
My Magnum Opus includes discussions of Bohr, Einstein, and McCulloch, among other great scientists:</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amazon.com_Conscious-2DMind-2DResonant-2DBrain-2DMakes_dp_0190070552&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=ou936izPpFpChNwRZ7hsXjS2h1WcDB3Zn1SvpbrBqQJ2UGafMEGcvLbbCYXBrlLo&s=M2rv3-OFfWi0TDKoQG0GaMUZg6yERYtvaKnpqLmDal0&e=" id="LPlnk623948">https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552</a><br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Best, </div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 18pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Steve</div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:18pt; color:rgb(0,0,0)" class="elementToProof">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Gary Marcus <gary.marcus@nyu.edu><br>
<b>Sent:</b> Tuesday, January 17, 2023 1:35 PM<br>
<b>To:</b> Michael Arbib <arbib@usc.edu><br>
<b>Cc:</b> connectionists@cs.cmu.edu <connectionists@cs.cmu.edu><br>
<b>Subject:</b> Re: Connectionists: Annotated History of Modern AI and Deep Learning</font>
<div> </div>
</div>
<div dir="auto">
<div dir="ltr"></div>
<div dir="ltr">Wow. Chills down spine, in a good way. I did not know that and look forward to reading! </div>
<div dir="ltr"><br>
<blockquote type="cite">On Jan 17, 2023, at 10:32, Michael Arbib <arbib@usc.edu> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<div class="x_WordSection1">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<span style="color:#00B0F0">Now that Cybernetics has been brought into the conversation, and since I may be the only person who was both a PhD student of Norbert Wiener (for a while) and an RA for Warren McCulloch, I take the liberty of drawing attention to
a memoir I wrote:</span></p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<span style="color:#00B0F0"> </span></p>
<p class="x_EndNoteBibliography" style="margin: 0in 0in 3pt; text-align: justify; font-size: 10pt; font-family: "Times New Roman", serif; color: windowtext;margin-left:.5in; text-indent:-.5in">
<span style="color:#00B0F0">Arbib, M. A. (2018). From cybernetics to brain theory, and more: A memoir.
<i>Cognitive Systems Research</i>,<i> 50</i>, 83-145. </span><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#00B0F0"></span></p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<span style="color:#00B0F0"> </span></p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<span style="color:#00B0F0">A preprint is available on ResearchGate – just enter “Arbib ResearchGate Memoir” in your browser. There are ideas in there whose solution I still await ….</span></p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
************************************<span style="color:windowtext"></span></p>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<span style="color:windowtext"> </span></p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<b><span style="color:windowtext">From:</span></b><span style="color:windowtext"> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu>
<b>On Behalf Of </b>Stephen José Hanson<br>
<b>Sent:</b> Tuesday, January 17, 2023 5:13 AM<br>
<b>To:</b> Sean Manion <stmanion@gmail.com>; Gary Marcus <gary.marcus@nyu.edu><br>
<b>Cc:</b> connectionists@cs.cmu.edu<br>
<b>Subject:</b> Re: Connectionists: Annotated History of Modern AI and Deep Learning</span></p>
</div>
</div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
<p><span style="font-size:13.5pt; font-family:"Courier New"">Sean,</span></p>
<p><span style="font-size:13.5pt; font-family:"Courier New"">What a wonderfult find!</span></p>
<p><span style="font-size:13.5pt; font-family:"Courier New"">I believe this is most likely a precursor to the Macy meetings 1946-1953, which was run by McCulloch.<br>
It included the wonderful list from Wiener of the areas that "have obtained a degree of intimacy".</span></p>
<p><span style="font-size:13.5pt; font-family:"Courier New"">These meetings became called by the attendees-- CYBERNETICS.. and of course is the precursor to AI and Neural Networks, computational neuroscience etc..</span></p>
<p><span style="font-size:13.5pt; font-family:"Courier New""><a href="https://urldefense.com/v3/__https:/press.uchicago.edu/ucp/books/book/distributed/C/bo23348570.html__;!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOuilVTjVXw$" data-auth="NotApplicable" data-loopstyle="link">https://press.uchicago.edu/ucp/books/book/distributed/C/bo23348570.html</a></span></p>
<p><span style="font-size:13.5pt; font-family:"Courier New"">thanks for sharing!</span></p>
<p><span style="font-size:13.5pt; font-family:"Courier New"">Steve</span></p>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
On 1/17/23 00:28, Sean Manion wrote:</p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Thank you all for a great discussion, and of course Jürgen for your work on the annotated history that has kicked it off.
</p>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
For reasons tangential to all of this, I have been recently reviewing some of the MIT Archives and found this invitation from Wiener, von Neumann, and Aiken to several individuals for a sometimes historically overlooked 2 day meeting that was held at Princeton
in January 1945 on a "...field of effort, which as yet is not even named."</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
I thought some might find this of interest.</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Cheers!</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sean</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
</div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
<div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
On Mon, Jan 16, 2023 at 11:51 PM Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" data-auth="NotApplicable" data-loopstyle="link">gary.marcus@nyu.edu</a>> wrote:</p>
</div>
<blockquote style="border:none; border-left:solid #CCCCCC 1.0pt; padding:0in 0in 0in 6.0pt; margin-left:4.8pt; margin-right:0in">
<div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Hi, Juergen,</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Thanks for your reply. Restricting your title to “modern” AI as you did is a start, but I think still not enough. For example, from what I understand about NNAISANCE, through talking with you and Bas Steunebrink, there’s quite a bit of hybrid AI in what you
are doing at your company, not well represented in the review. The related open-access book certainly draws heavily on both traditions (<a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Flink.springer.com*2Fbook*2F10.1007*2F978-3-031-08020-3&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=nzjxkTj5pTpbFiPk8QGz0HvRgVoDGlRJnyo9vGsuALU*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOui_qF6Ftw$" data-auth="NotApplicable" data-loopstyle="link">https://link.springer.com/book/10.1007/978-3-031-08020-3</a>).</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Likewise, there is plenty of eg symbolic planning in modern navigation systems, most robots etc; still plenty of use of symbolic trees in game playing; lots of people still use taxonomies and inheritance, etc., an AFAIK nobody has built a trustworthy virtual
assistant, even in a narrow domain, with only deep learning. And so on. </p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
In the end, it’s really a question about balance, which is what I think Andrzej was getting at; you go miles deep on the history of deep learning, which I respect, but just give relatively superficial pointers (not none!) outside that tradition. Definitely
better, to be sure, in having at least a few pointers than in having none, and I would agree that the future is uncertain. I think you strike the right note there!</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
As an aside, saying that everything can be formulated as RL is maybe no more helpful than saying that everything we (currently) know how to do can be formulated in terms of Turing machine. True, but doesn’t carry you far enough in most real world applications.
I personally see RL as part of an answer, but most useful in (and here we might partly agree) the context of systems with rich internal models of the world. </p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
My own view is that we will get to more reliable AI only once the field more fully embraces the project of articulating how such models work and how they are developed. </p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Which is maybe the one place where you (eg <a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Farxiv.org*2Fpdf*2F1803.10122.pdf&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=ZyJcnldLGH1FWLfLy*2B1OaQogSmVjZdOKu05SVnmQgyo*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOujbUxHYPQ$" data-auth="NotApplicable" data-loopstyle="link">https://arxiv.org/pdf/1803.10122.pdf</a>),
Yann LeCun (eg <a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Fopenreview.net*2Fforum*3Fid*3DBZ5a1r-kVsf&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=NciJImyaIct5w804QCbLcBdvC56Tb8s8oyS9Rn*2BGxhY*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOugZ5NfTNg$" data-auth="NotApplicable" data-loopstyle="link">https://openreview.net/forum?id=BZ5a1r-kVsf</a>),
and I (eg <a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Farxiv.org*2Fabs*2F2002.06177&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=xk1Vn44bpzaJS*2Bgxa2RrYNREA4i1O*2Bmf2vInHJc76KE*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOugdXMd64Q$" data-auth="NotApplicable" data-loopstyle="link">https://arxiv.org/abs/2002.06177</a>)
are most in agreement.</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Best,</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Gary</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<br>
<br>
</p>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;margin-bottom:12.0pt">
On Jan 15, 2023, at 23:04, Schmidhuber Juergen <<a href="mailto:juergen@idsia.ch" data-auth="NotApplicable" data-loopstyle="link">juergen@idsia.ch</a>> wrote:</p>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<div>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Thanks for these thoughts, Gary! <br>
<br>
1. Well, the survey is about the roots of “modern AI” (as opposed to all of AI) which is mostly driven by “deep learning.” Hence the focus on the latter and the URL "deep-learning-history.html.” On the other hand, many of the most famous modern AI applications
actually combine deep learning and other cited techniques (more on this below).<br>
<br>
Any problem of computer science can be formulated in the general reinforcement learning (RL) framework, and the survey points to ancient relevant techniques for search & planning, now often combined with NNs:<br>
<br>
"Certain RL problems can be addressed through non-neural techniques invented long before the 1980s: Monte Carlo (tree) search (MC, 1949) [MOC1-5], dynamic programming (DP, 1953) [BEL53], artificial evolution (1954) [EVO1-7][TUR1] (unpublished), alpha-beta-pruning
(1959) [S59], control theory and system identification (1950s) [KAL59][GLA85], stochastic gradient descent (SGD, 1951) [STO51-52], and universal search techniques (1973) [AIT7].<br>
<br>
Deep FNNs and RNNs, however, are useful tools for _improving_ certain types of RL. In the 1980s, concepts of function approximation and NNs were combined with system identification [WER87-89][MUN87][NGU89], DP and its online variant called Temporal Differences
[TD1-3], artificial evolution [EVONN1-3] and policy gradients [GD1][PG1-3]. Many additional references on this can be found in Sec. 6 of the 2015 survey [DL1].
<br>
<br>
When there is a Markovian interface [PLAN3] to the environment such that the current input to the RL machine conveys all the information required to determine a next optimal action, RL with DP/TD/MC-based FNNs can be very successful, as shown in 1994 [TD2]
(master-level backgammon player) and the 2010s [DM1-2a] (superhuman players for Go, chess, and other games). For more complex cases without Markovian interfaces, …”<br>
<br>
Theoretically optimal planners/problem solvers based on algorithmic information theory are mentioned in Sec. 19.<br>
<br>
2. Here a few relevant paragraphs from the intro:<br>
<br>
"A history of AI written in the 1980s would have emphasized topics such as theorem proving [GOD][GOD34][ZU48][NS56], logic programming, expert systems, and heuristic search [FEI63,83][LEN83]. This would be in line with topics of a 1956 conference in Dartmouth,
where the term "AI" was coined by John McCarthy as a way of describing an old area of research seeing renewed interest.
<br>
<br>
Practical AI dates back at least to 1914, when Leonardo Torres y Quevedo built the first working chess end game player [BRU1-4] (back then chess was considered as an activity restricted to the realms of intelligent creatures). AI theory dates back at least
to 1931-34 when Kurt Gödel identified fundamental limits of any type of computation-based AI [GOD][BIB3][GOD21,a,b].<br>
<br>
A history of AI written in the early 2000s would have put more emphasis on topics such as support vector machines and kernel methods [SVM1-4], Bayesian (actually Laplacian or possibly Saundersonian [STI83-85]) reasoning [BAY1-8][FI22] and other concepts of
probability theory and statistics [MM1-5][NIL98][RUS95], decision trees, e.g. [MIT97], ensemble methods [ENS1-4], swarm intelligence [SW1], and evolutionary computation [EVO1-7][TUR1]. Why? Because back then such techniques drove many successful AI applications.<br>
<br>
A history of AI written in the 2020s must emphasize concepts such as the even older chain rule [LEI07] and deep nonlinear artificial neural networks (NNs) trained by gradient descent [GD’], in particular, feedback-based recurrent networks, which are general
computers whose programs are weight matrices [AC90]. Why? Because many of the most famous and most commercial recent AI applications depend on them [DL4]."<br>
<br>
3. Regarding the future, you mentioned your hunch on neurosymbolic integration. While the survey speculates a bit about the future, it also says: "But who knows what kind of AI history will prevail 20 years from now?”
<br>
<br>
Juergen<br>
<br>
<br>
<br>
</p>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
On 14. Jan 2023, at 15:04, Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" data-auth="NotApplicable" data-loopstyle="link">gary.marcus@nyu.edu</a>> wrote:</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Dear Juergen,</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
You have made a good case that the history of deep learning is often misrepresented. But, by parity of reasoning, a few pointers to a tiny fraction of the work done in symbolic AI does not in any way make this a thorough and balanced exercise with respect to
the field as a whole.</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
I am 100% with Andrzej Wichert, in thinking that vast areas of AI such as planning, reasoning, natural language understanding, robotics and knowledge representation are treated very superficially here. A few pointers to theorem proving and the like does not
solve that. </p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Your essay is a fine if opinionated history of deep learning, with a special emphasis on your own work, but of somewhat limited value beyond a few terse references in explicating other approaches to AI. This would be ok if the title and aspiration didn’t aim
for as a whole; if you really want the paper to reflect the field as a whole, and the ambitions of the title, you have more work to do.
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
My own hunch is that in a decade, maybe much sooner, a major emphasis of the field will be on neurosymbolic integration. Your own startup is heading in that direction, and the commericial desire to make LLMs reliable and truthful will also push in that direction.
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Historians looking back on this paper will see too little about that roots of that trend documented here.</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Gary </p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
On Jan 14, 2023, at 12:42 AM, Schmidhuber Juergen <<a href="mailto:juergen@idsia.ch" data-auth="NotApplicable" data-loopstyle="link">juergen@idsia.ch</a>> wrote:</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Dear Andrzej, thanks, but come on, the report cites lots of “symbolic” AI from theorem proving (e.g., Zuse 1948) to later surveys of expert systems and “traditional" AI. Note that Sec. 18 and Sec. 19 go back even much further in time (not even speaking of
Sec. 20). The survey also explains why AI histories written in the 1980s/2000s/2020s differ. Here again the table of contents:</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 1: Introduction</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 2: 1676: The Chain Rule For Backward Credit Assignment</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 3: Circa 1800: First Neural Net (NN) / Linear Regression / Shallow Learning</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 4: 1920-1925: First Recurrent NN (RNN) Architecture. ~1972: First Learning RNNs</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 5: 1958: Multilayer Feedforward NN (without Deep Learning)</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 6: 1965: First Deep Learning</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 7: 1967-68: Deep Learning by Stochastic Gradient Descent </p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 8: 1970: Backpropagation. 1982: For NNs. 1960: Precursor. </p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 9: 1979: First Deep Convolutional NN (1969: Rectified Linear Units) </p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 10: 1980s-90s: Graph NNs / Stochastic Delta Rule (Dropout) / More RNNs / Etc</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 11: Feb 1990: Generative Adversarial Networks / Artificial Curiosity / NN Online Planners</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 12: April 1990: NNs Learn to Generate Subgoals / Work on Command </p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 13: March 1991: NNs Learn to Program NNs. Transformers with Linearized Self-Attention</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 14: April 1991: Deep Learning by Self-Supervised Pre-Training. Distilling NNs</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 15: June 1991: Fundamental Deep Learning Problem: Vanishing/Exploding Gradients</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 16: June 1991: Roots of Long Short-Term Memory / Highway Nets / ResNets</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 17: 1980s-: NNs for Learning to Act Without a Teacher </p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 18: It's the Hardware, Stupid!</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 19: But Don't Neglect the Theory of AI (Since 1931) and Computer Science</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 20: The Broader Historic Context from Big Bang to Far Future</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 21: Acknowledgments</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Sec. 22: 555+ Partially Annotated References (many more in the award-winning survey [DL1])</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Tweet: <a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__twitter.com_SchmidhuberAI_status_1606333832956973060-3Fcxt-3DHHwWiMC8gYiH7MosAAAA*26d*3DDwIDaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DoGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5*26s*3DnWCXLKazOjmixYrJVR0CMlR12PasGbAd8bsS6VZ10bk*26e*3D&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=bZfxmr1oj5alCicvKzzhXDN4q6855smPHdbBVbvcqSw*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOugHyPKUQw$" data-auth="NotApplicable" data-loopstyle="link">
https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_SchmidhuberAI_status_1606333832956973060-3Fcxt-3DHHwWiMC8gYiH7MosAAAA&d=DwIDaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=oGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5&s=nWCXLKazOjmixYrJVR0CMlR12PasGbAd8bsS6VZ10bk&e=</a>
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Jürgen</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
</p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
On 13. Jan 2023, at 14:40, Andrzej Wichert <<a href="mailto:andreas.wichert@tecnico.ulisboa.pt" data-auth="NotApplicable" data-loopstyle="link">andreas.wichert@tecnico.ulisboa.pt</a>> wrote:</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Dear Juergen,</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
You make the same mistake at it was done in the earlier 1970. You identify deep learning with modern AI, the paper should be called instead "Annotated History of Deep Learning”</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Otherwise, you ignore symbolical AI, like search, production systems, knowledge representation, search, planning etc., as if is not part of AI anymore (suggested by your title).</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Best,</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Andreas</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
--------------------------------------------------------------------------------------------------</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Prof. Auxiliar Andreas Wichert </p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttp-3A__web.tecnico.ulisboa.pt_andreas.wichert_*26d*3DDwIDaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DoGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5*26s*3Dh5Zy9Hk2IoWPt7me1mLhcYHEuJ55mmNOAppZKcivxAk*26e*3D&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=AceluKx*2B8KWGPMO06FCjnpyFyTKpTMbig7tM*2BhOG*2FUg*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOugJDXOm1w$" data-auth="NotApplicable" data-loopstyle="link">https://urldefense.proofpoint.com/v2/url?u=http-3A__web.tecnico.ulisboa.pt_andreas.wichert_&d=DwIDaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=oGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5&s=h5Zy9Hk2IoWPt7me1mLhcYHEuJ55mmNOAppZKcivxAk&e=</a></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
-</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__www.amazon.com_author_andreaswichert*26d*3DDwIDaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DoGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5*26s*3Dw1RtYvs8dwtfvlTkHqP_P-74ITvUW2IiHLSai7br25U*26e*3D&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=*2F*2FDhQhTwI7XGRKbBkGIi8*2BuOWxknWbBpspj5YPwUbTU*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOugvtzNUdg$" data-auth="NotApplicable" data-loopstyle="link">https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amazon.com_author_andreaswichert&d=DwIDaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=oGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5&s=w1RtYvs8dwtfvlTkHqP_P-74ITvUW2IiHLSai7br25U&e=</a></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Instituto Superior Técnico - Universidade de Lisboa</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Campus IST-Taguspark</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Avenida Professor Cavaco Silva Phone: +351 214233231</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
2744-016 Porto Salvo, Portugal</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
On 13 Jan 2023, at 08:13, Schmidhuber Juergen <<a href="mailto:juergen@idsia.ch" data-auth="NotApplicable" data-loopstyle="link">juergen@idsia.ch</a>> wrote:</p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Machine learning is the science of credit assignment. My new survey credits the pioneers of deep learning and modern AI (supplementing my award-winning 2015 survey):</p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__arxiv.org_abs_2212.11279*26d*3DDwIDaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DoGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5*26s*3D6E5_tonSfNtoMPw1fvFOm8UFm7tDVH7un_kbogNG_1w*26e*3D&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=e*2Bf09tNf3tO8tY2jANTGEUnAipIJum92za3z*2FNZMbaw*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOuhyBWAubA$" data-auth="NotApplicable" data-loopstyle="link">https://urldefense.proofpoint.com/v2/url?u=https-3A__arxiv.org_abs_2212.11279&d=DwIDaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=oGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5&s=6E5_tonSfNtoMPw1fvFOm8UFm7tDVH7un_kbogNG_1w&e=</a></p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
<a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__people.idsia.ch_-7Ejuergen_deep-2Dlearning-2Dhistory.html*26d*3DDwIDaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DoGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5*26s*3DXPnftI8leeqoElbWQIApFNQ2L4gDcrGy_eiJv2ZPYYk*26e*3D&data=05*7C01*7Cjose*40rubic.rutgers.edu*7Cb3db76bd41ff4a59071908daf85da95a*7Cb92d2b234d35447093ff69aca6632ffe*7C1*7C0*7C638095378420227407*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*7C*7C*7C&sdata=cGUbD8*2Fc31LU5nO7Oc1I7b3UERXBn2GArYhEO1IE1Bc*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!LIr3w8kk_Xxm!rhOV1_I3SVtEWzbHAOD-b63GOlQ_CnsGhyQR9CixHIagtitUuZEV-Lr8y549an4wJVQrYiLaOuiTz43jzQ$" data-auth="NotApplicable" data-loopstyle="link">https://urldefense.proofpoint.com/v2/url?u=https-3A__people.idsia.ch_-7Ejuergen_deep-2Dlearning-2Dhistory.html&d=DwIDaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=oGn-OID5YOewbgo3j_HjFjI3I2N3hx-w0hoIfLR_JJsn8q5UZDYAl5HOHPY-87N5&s=XPnftI8leeqoElbWQIApFNQ2L4gDcrGy_eiJv2ZPYYk&e=</a></p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
This was already reviewed by several deep learning pioneers and other experts. Nevertheless, let me know under
<a href="mailto:juergen@idsia.ch" data-auth="NotApplicable" data-loopstyle="link">
juergen@idsia.ch</a> if you can spot any remaining error or have suggestions for improvements.</p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Happy New Year!</p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;">
Jürgen</p>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<p class="x_MsoNormal" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; color: black;margin-bottom:12.0pt">
</p>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<pre style="margin: 0in; font-size: 10pt; font-family: "Courier New"; color: black;">-- </pre>
<pre style="margin: 0in; font-size: 10pt; font-family: "Courier New"; color: black;">Stephen José Hanson</pre>
<pre style="margin: 0in; font-size: 10pt; font-family: "Courier New"; color: black;">Professor, Psychology Department</pre>
<pre style="margin: 0in; font-size: 10pt; font-family: "Courier New"; color: black;">Director, RUBIC (Rutgers University Brain Imaging Center)</pre>
<pre style="margin: 0in; font-size: 10pt; font-family: "Courier New"; color: black;">Member, Executive Committee, RUCCS</pre>
</div>
</div>
</blockquote>
</div>
</div></blockquote></body></html>