<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
@font-face
{font-family:"Apple Color Emoji";
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Roboto;
panose-1:2 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
span.xapple-converted-space
{mso-style-name:x_apple-converted-space;}
span.EmailStyle25
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Dear John,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I reply below <i>in italics</i>, among your questions:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="color:black">From:
</span></b><span style="color:black">Weng, Juyang <weng@msu.edu><br>
<b>Date: </b>Saturday, February 24, 2024 at 4:44</span><span style="font-family:"Arial",sans-serif;color:black"> </span><span style="color:black">PM<br>
<b>To: </b>Jeffrey Bowers <J.Bowers@bristol.ac.uk>, Grossberg, Stephen <steve@bu.edu>, KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk>, Gary Marcus <gary.marcus@nyu.edu>, Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222">Dear Steve,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222"> I have had pleasure to listen and follow your various ART models. With your suggestions, I also bought your Book "Conscious Mind, Resonant Brain: How Each
Brain Makes a Mind" and browsed it. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222"> Let me ask some questions that will be useful for many people on this list:<br>
(1) Do you agree that the grandmother cell does not explain how to learn invariances, such as location invariance, scale invariance, and orientational invariance? Of course, those invariances are not perfect as explained in my Cresceptron paper (IJCV 197),
arguably the first Deep Learning network for 3D?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">The grandmother cell concept is nothing more than a verbal term. It is not a computational model, so does not learn anything.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">I glanced at your Cresceptron paper. It says:
<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">“</span></i><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white"> the user manually draws a polygon outlining the region of interest and types in the label of its class….”<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white">It seems that this is not a self-organizing model that learns in real time through incremental learning. Our model is, and explains challenging neurobiological
data along the way.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white">You also write that your model may be “the first Deep Learning network for 3D”. Our work does not use Deep Learning, which has 17 serious computational
problems in addition to not being biologically plausible. None of these problems of back propagation and Deep Learning have been a problem for Adaptive Resonance Theory since I introduced it in 1976.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white">In particular, Deep Learning is both untrustworthy (because it is not explainable) and unreliable (because it can experience catastrophic forgetting).
<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white">I review these 17 problems in my 2021 Magnum Opus. You can also find them discussed in Section 17 of the following 1988 article that was published in the
first issue of Neural Networks:<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:10.5pt;font-family:Roboto;color:#131314;background:white"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Grossberg, S. (1988) Nonlinear neural networks: Principles, mechanisms, and architectures. <o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Neural Networks, <b>1 </b>, 17-61. <o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><a href="https://sites.bu.edu/steveg/files/2016/06/Gro1988NN.pdf">https://sites.bu.edu/steveg/files/2016/06/Gro1988NN.pdf</a><o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222"><br>
(2) Your model discussed your book is not developmental, namely, does not start from a single cell like the human brain and does not learn incrementally. Could you point me to an incremental learning algorithm in your book if what I write is incorrect?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">I am bewildered by your comment above, since it is obviously not true about ANY of my neural models of brain development and learning, all of which self-organize and work in an incremental learning setting.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Such models are described in a self-contained and non-technical way in my book.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Scores of my articles about self-organizing brain development and learning are described with all technical details on my web page sites.bu.edu/steveg.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">I am unclear what you mean by the phrase: “does not start from a single cell like the human brain” since you clearly do not mean that the human brain is composed of a single cell.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">On the other hand, Chapter 17 of my 2021 Magnum Opus clarifies that principles of complementarity, uncertainty, and resonance that are embodied in Adaptive Resonance Theory, as well as in various of my
other neural network models, also have precursors in cellular organisms that existed long before human brains did, including slime molds and Hydras.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">These design principles thus seem to have been conserved for a very long time during the evolutionary process.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Principles of uncertainty, complementarity, and resonance also have analogs in the laws of physics with which our brains have ceaselessly interacted for eons during their evolution. Quantum mechanics is
one example of these principles in physics.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Explaining in detail how our brains were shaped during evolution to also embody these physical principles is a long-term project worthy of a great deal of additional research.</span></i><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222"> (3) Your model does not explain how to learn any Turing machines. </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Human brains self-organize using analog signals and parallel computations, and do so in real time. Turing machines do not have these properties.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">So, yes, no biological neural network model of how brains make minds learn Turing machines, except in the sense that our cognitive systems, that are parts of our brains, have learned to generate emergent
properties that invented and can mathematically analyze Turing machines.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Is that what you mean?<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">To explain how our brains can mathematically understand Turing machines, you first need to explain how our brains have learned to represent and use numerical representations in the first place.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">The question “where do numbers come from” is a fundamental one.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Some progress has been made in modeling “where numbers come from” and how our brains can learn to use numerical representations and mathematical symbols. A LOT more work needs to be done on this fundamental
problem.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Perhaps the following article may be helpful:<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Grossberg, S. and Repin, D. (2003) A neural model of how the brain represents and compares multi-digit numbers: Spatial and categorical processes. Neural Networks,<b> 16</b>, 1107-1140. <o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><a href="https://sites.bu.edu/steveg/files/2016/06/GroRep2003NN.pdf">https://sites.bu.edu/steveg/files/2016/06/GroRep2003NN.pdf</a><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Best,<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">Steve<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt">P.S. Since I became an Emeritus Professor, I no longer serve on any journal editorial boards or advisory committees. Please remove me from yours.<o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222"> Do you have a plan for your model to handle (1) to (3) above, while my DN model does them? </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222">
<span style="background:white">Thank you very much for being on the Advisory Committee of Brain-Mind Institute. </span><br>
Best regards,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222">-John</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#222222">On Sat, Feb 24, 2024 at 4:03 PM Grossberg, Stephen <steve@bu.edu> wrote:</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p style="background:white"><span style="font-size:11.0pt;color:#222222">Dear Jeff,</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">Thanks for your supportive remark.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">One thing to keep in mind is that, even if a recognition category has a compressed representation using a small, compact population of cells, a much larger population of cells is needed
for that category to work.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">For starters, even a compact category representation is activated by a distributed pattern of activation across the network of feature-selective cells with which the category resonates
via excitatory feedback signals when it is chosen.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">In the case of invariant object categories, a widespread neural architecture is needed to learn it, including modulatory signals from the dorsal, or Where, cortical stream to the ventral,
or What, cortical stream where the category is being learned.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">These modulatory signals are needed to ensure that the invariant object category binds together only views that belong to that object, and not irrelevant features that may be distributed
across the scene.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">These modulatory signals also maintain spatial attention on the invariant category as it is being learned. I call the resonance that accomplishes this a surface-shroud resonance. I propose
that it occurs between cortical areas V4 and PPC and triggers a system-wide resonance at earlier and later cortical areas.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">Acting in space on the object that is recognized by the invariant category requires reciprocal What-to-Where stream interactions. These interactions embody a proposed solution of the Where’s
Waldo Problem.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">I have attached a couple of the figures that summarize the ARTSCAN Search architecture that tries to explain and simulate these interactions.</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">This neural architecture is far from being a “grandmother cell”!</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">My Magnum Opus provides a lot more modeling explanations and data about these issues:</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#1155CC"><a href="https://urldefense.com/v3/__https:/www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552__;!!HXCxUKc!x8_h_sDc7CZsLdA-pZYnpLJ-xI4kdhCIL5T22Qj55OahMhzyWMqNHgQ6ubkD81gqDIlov9vVzA$" target="_blank"><span style="color:#1155CC">https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552</span></a></span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">Best again,</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222">Steve</span><o:p></o:p></p>
<p style="background:white"><span style="font-size:11.0pt;color:#222222"> </span><o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on
behalf of Jeffrey Bowers <J.Bowers@bristol.ac.uk><br>
<b>Sent:</b> Saturday, February 24, 2024 4:38 AM<br>
<b>To:</b> Grossberg, Stephen <steve@bu.edu>; KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk>; Gary Marcus <gary.marcus@nyu.edu>; Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc:</b> connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject:</b> Re: Connectionists: Early history of symbolic and neural network approaches to AI</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Dear Steve, I agree, the grandmother cell theory is ill defined, and it is often defined in such a way that it is false. But then people conclude from that that the brain encodes information
in a distributed manner, with each unit (neuron) coding for multiple different things. That conclusion is unjustified. I think your ART models provide an excellent example of one way to implement grandmother cell theories. ART can learn localist codes where
a single unit encodes an object in an abstract way. The Jennifer Aniston neuron results are entirely consistent with your models, even though a given neuron might respond above baseline to other inputs (at least prior to settling into a resonance). Jeff</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<div id="x_mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Grossberg, Stephen <steve@bu.edu><br>
<b>Date: </b>Friday, 23 February 2024 at 18:12<br>
<b>To: </b>Jeffrey Bowers <J.Bowers@bristol.ac.uk>, KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk>, Gary Marcus <gary.marcus@nyu.edu>, Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu>, Grossberg, Stephen <steve@bu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Dear Jeff et al.,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">The term Grandmother Cell was a good heuristic but, as has been noted in this email thread, is also ill-defined.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">It is known that there are cells in anterior Inferotemporal Cortex (ITa) that may be called invariant object recognition categories because they respond to a visually perceived
object from multiple views, sizes, and positions.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">There are also view-specific categories in posterior Inferotemporal Cortex (ITp) that do not have such broad invariance.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">I list below several of our articles that model how invariant object categories and view-specific categories may be learned. We also use the modeling results to explain a lot
of data.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Just a scan of the article titles illustrates that there has been a lot of work on this topic.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Fazl, A., Grossberg, S., and Mingolla, E. (2009). View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based
attentional shrouds. <i>Cognitive Psychology</i>, <b>58</b>, 1-48. </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><a href="https://urldefense.com/v3/__https:/sites.bu.edu/steveg/files/2016/06/FazGroMin2008.pdf__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95NVs-tmLM$">https://sites.bu.edu/steveg/files/2016/06/FazGroMin2008.pdf</a></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Cao, Y., Grossberg, S., and Markowitz, J. (2011). How does the brain rapidly learn and reorganize view- and positionally-invariant object representations in inferior temporal
cortex? <i>Neural Networks</i>, <b>24</b>, 1050-1061.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><a href="https://urldefense.com/v3/__https:/sites.bu.edu/steveg/files/2016/06/NN2853.pdf__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95N4VqUiQ0$">https://sites.bu.edu/steveg/files/2016/06/NN2853.pdf</a></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Grossberg, S., Markowitz, J., and Cao, Y. (2011). On the road to invariant recognition: Explaining tradeoff and morph properties of cells in inferotemporal cortex using multiple-scale
task-sensitive attentive learning. <i>Neural Networks</i>, <b>24</b>, 1036-1049.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><a href="https://urldefense.com/v3/__https:/sites.bu.edu/steveg/files/2016/06/GroMarCao2011TR.pdf__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95NmVjlCCA$">https://sites.bu.edu/steveg/files/2016/06/GroMarCao2011TR.pdf</a></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Grossberg, S., Srinivasan, K., and Yazdabakhsh, A. (2011). On the road to invariant object recognition: How cortical area V2 transforms absolute to relative disparity during
3D vision. <i>Neural Networks</i>, <b>24</b>, 686-692. </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><a href="https://urldefense.com/v3/__https:/sites.bu.edu/steveg/files/2016/06/GroSriYaz2011TR.pdf__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95NR1uvfJY$">https://sites.bu.edu/steveg/files/2016/06/GroSriYaz2011TR.pdf</a></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">
</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Foley, N.C., Grossberg, S. and Mingolla, E. (2012). Neural dynamics of object-based multifocal visual spatial attention and priming: Object cueing, useful-field-of-view, and
crowding. <i>Cognitive Psychology</i>, <b>65</b>, 77-117.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><a href="https://urldefense.com/v3/__https:/sites.bu.edu/steveg/files/2016/06/FolGroMin2012.pdf__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95N7ByOAj4$">https://sites.bu.edu/steveg/files/2016/06/FolGroMin2012.pdf</a></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Grossberg, S., Srinivasan, K., and Yazdanbakhsh, A. (2014). Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene
with eye movements. <i>Frontiers in Psychology: Perception Science, </i>doi: 10.3389/fpsyg.2014.01457</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><a href="https://urldefense.com/v3/__https:/www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2014.01457/full__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95NY-PYhTY$">https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2014.01457/full</a></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">More articles on related topics can be found on my web page sites.bu.edu/steveg, including how humans can search for an object at an expected position in space, even though
its invariant object category representation cannot be used to do so.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Best,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Steve</span><span lang="EN-GB"><o:p></o:p></span></p>
<div id="x_mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Jeffrey Bowers <J.Bowers@bristol.ac.uk><br>
<b>Date: </b>Thursday, February 22, 2024 at 11:11</span><span lang="EN-GB" style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">AM<br>
<b>To: </b>KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk>, Gary Marcus <gary.marcus@nyu.edu>, Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Good point, I should not have used simple cells as an example of grandmother cells. In fact, I agree that some sort of population coding is likely supporting our perception of orientation.
For example, simple cells are oriented in steps of about 5 degrees, but we can perceive orientations at a much finer granularity, so it must be a combination of cells driving our perception.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">The other reason I should have not used simple cells is that grandmother cells are a theory about how we identify familiar categories of objects (my grandmother, or a dog or a cat). Orientation
is a continuous dimension where distributed coding may be more suitable. The better example I gave is the word representation DOG in the IA model. The fact that the DOG detector is partly activated by the input CAT does not falsify the hypothesis that DOG
is locally coded. Indeed, it has hand-wired to be localist. In the same way, the fact that a Jennifer Aniston neuron might be weakly activated by another face does not rule out the hypothesis that the neuron selectively codes for Jennifer Aniston. I agree
it is not strong evidence for a grandmother cell – there may be other images that drive the neuron even more, we just don’t know given the limited number of images presented to the patient. But it is interesting that there are various demonstrations that
artificial networks learn grandmother cells under some conditions – when you can test the model on all the familiar categories it has seen. So, I would not rule out grandmother cells out of hand.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Jeff </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<div id="x_mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk><br>
<b>Date: </b>Wednesday, 21 February 2024 at 20:56<br>
<b>To: </b>Jeffrey Bowers <J.Bowers@bristol.ac.uk>, Gary Marcus <gary.marcus@nyu.edu>, Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Again, it is great to be examining the relationship between ‘real’ neural coding and the ins and outs of representation in ANNs. I’m really pleased to be able to make a few contributions to a
list which I’ve lurked on since the late 1980s!</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">I feel I should add an alternative interpretation of orientation coding in primary visual cortex to that so clearly explained by Jeffrey. It is, indeed, tempting to think of orientation tuned
cells as labelled lines or grandmother cells where we read off activity in individual cells as conveying the presence of a line segment with a specific orientation at a particular location in the visual field. As neuroscientists we can certainly do this. The
key question is whether brain areas outside primary visual cortex, which are consumers of information coded in primary visual cortex, also do this. The alternative view of orientation coding is that orientation is represented by a population code where orientation
is represented as the vector sum of orientation preferences in cells with many different orientation tunings, weighted by their levels of activity, and that it is this population code that is read by areas that are consumers of orientation information. The
notion of neural population coding of orientation was first tested electrophysiologically by Georgopoulos in 1982, examining population coding of the direction of arm movements in primary motor cortex. There is more recent psychophysical evidence that people’s
confidence in their judgements of the orientation of a visual stimulus can be predicted on the basis of a population coding scheme (Bays, 2016, A signature of neural coding at human perceptual limits. Journal of Vision,
</span><span lang="EN-GB"><a href="https://urldefense.com/v3/__https:/jov.arvojournals.org/article.aspx?articleid=2552242__;!!HXCxUKc!xerjmqrW_Xex4VBT9aigO8X0lTvUAuaU_sdXOYMi8HHngPHq-UAZiqk-r6N9KCDg7kcmi7rqo95NslADdm8$"><span style="font-size:11.0pt">https://jov.arvojournals.org/article.aspx?articleid=2552242</span></a></span><span lang="EN-GB" style="font-size:11.0pt">),
where a person’s judgment is indicative of the state of a high level consumer of orientation information.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">So again, I’d err on the side of suggesting that although we can conceive of single neurons in primary visual cortex as encoding information (maybe not really symbols in this case anyway), it
isn’t our ability to interpret things like this that matters, rather, it is the way the rest of the brain interprets information delivered by primary visual cortex.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">cheers,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Bob</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"><img border="0" width="57" height="57" style="width:.5937in;height:.5937in" id="x_Picture_x0020_12" src="cid:image001.jpg@01DA64AF.5BBBA710" alt="Image result for university of durham logo">
<img border="0" width="118" height="55" style="width:1.2291in;height:.5729in" id="x_Picture_x0020_11" src="cid:image002.png@01DA64AF.5BBBA710" alt="signature_2025328812"> <img border="0" width="94" height="55" style="width:.9791in;height:.5729in" id="x_Picture_x0020_10" src="cid:image003.png@01DA64AF.5BBBA710" alt="signature_824875734"> <img border="0" width="46" height="56" style="width:.4791in;height:.5833in" id="x_Picture_x0020_9" src="cid:image004.jpg@01DA64AF.5BBBA710" alt="Image result for durham cvac"></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Professor of Psychology, University of Durham.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham PaleoPsychology Group.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham Centre for Vision and Visual Cognition.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham Centre for Visual Arts and Culture.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"><img border="0" width="49" height="49" style="width:.5104in;height:.5104in" id="x_Picture_x0020_8" src="cid:image005.jpg@01DA64AF.5BBBA710" alt="9k="></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Fellow. </span>
<span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Canadian Institute for Advanced Research,
</span><span lang="EN-GB"><o:p></o:p></span></p>
<div style="border:none;border-bottom:solid windowtext 1.0pt;padding:0in 0in 1.0pt 0in">
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Brain, Mind & Consciousness Programme.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Department of Psychology,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">University of Durham,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham DH1 3LE, UK.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">p: +44 191 334 3261</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">f: +44 191 334 3434</span><span lang="EN-GB"><o:p></o:p></span></p>
<div style="border:none;border-bottom:solid windowtext 1.0pt;padding:0in 0in 1.0pt 0in">
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<div id="x_mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Jeffrey Bowers <J.Bowers@bristol.ac.uk><br>
<b>Date: </b>Wednesday, 21 February 2024 at 12:31<br>
<b>To: </b>KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk>, Gary Marcus <gary.marcus@nyu.edu>, Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><strong><span lang="EN-GB" style="font-size:12.0pt;font-family:"Calibri",sans-serif;color:black;background:#FFFECF">[EXTERNAL EMAIL]</span></strong><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="xmsonormal"><span lang="EN-GB">It is possible to define a grandmother cell in a way that falsifies them. For instance, defining grandmother cells as single neurons that only *respond* to inputs from one category. Another definition that is more
plausible is single neurons that only *represent* one category. In psychology there are “localist” models that have single units that represent one category (e.g., there is a unit in the Interactive Activation Model that codes for the word DOG). And a feature
of localist codes is that they are partly activated by similar inputs. So a DOG detector is partly activated by the input HOG by virtue of sharing two letters. But that partial activation of the DOG unit from HOG is no evidence against a localist or grandmother
cell representation of the word DOG in the IA model. Just as a simple cell of a vertical line is partly activated by a line 5 degrees off vertical – that does not undermine the hypothesis that the simple cell *represents* vertical lines. I talk about the
plausibility of Grandmother cells and discuss the Aniston cells in a paper I wrote sometime back:<o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB"> <o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-family:"Arial",sans-serif;color:#222222;background:white">Bowers, J. S. (2009). On the biological plausibility of grandmother cells: implications for neural network theories in psychology and neuroscience.<span class="xapple-converted-space"> </span></span><i><span lang="EN-GB" style="font-family:"Arial",sans-serif;color:#222222">Psychological
review</span></i><span lang="EN-GB" style="font-family:"Arial",sans-serif;color:#222222;background:white">,<span class="xapple-converted-space"> </span></span><i><span lang="EN-GB" style="font-family:"Arial",sans-serif;color:#222222">116</span></i><span lang="EN-GB" style="font-family:"Arial",sans-serif;color:#222222;background:white">(1),
220.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<div id="x_mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of KENTRIDGE, ROBERT W. <robert.kentridge@durham.ac.uk><br>
<b>Date: </b>Wednesday, 21 February 2024 at 11:48<br>
<b>To: </b>Gary Marcus <gary.marcus@nyu.edu>, Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">I agree – empirical evidence is just what we need in this super-interesting discussion.
</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">I should point out a few things about the Quiroga et al 2005 ‘Jennifer Aniston cell’ finding (<i>Nature</i>, <b>435</b>. 1102 - 1107 ).
</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Quiroga et al themselves are at pains to point out that whilst the cells they found responded to a wide variety of depictions of specific individuals they were not ‘Grandmother cells’ as defined
by Jerry Lettvin – that is, specific cells that respond to a broad range of depictions of an individual and *<b>only</b>* of that individual, meaning that one can infer that this individual is being perceived, thought of, etc. whenever that cell is active.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">The cells Quiroga found do, indeed, respond to remarkably diverse ranges of stimuli depicting individuals, including not just photos in different poses, at different ages, in different costumes
(including Hale Berry as Catwoman for the Hale Berry cell), but also names presented as text (e.g. ‘HALE BERRY’). Quiroga et al only presented stimuli representing a relatively small range of individuals and so it is unsafe to conclude that the cells they
found respond *<b>only</b>* to the specific individuals they found. Indeed, they report that the Jennifer Aniston cell also responded strongly to an image of a different actress, Lisa Kudrow, who appeared in ‘Friends’ along with Jennifer Aniston.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">So, the empirical evidence is still on the side of activity in sets of neurons as representing specific symbols (including those standing for specific individuals) rather than individual cells
standing for specific symbols.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">cheers</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Bob</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"><img border="0" width="57" height="57" style="width:.5937in;height:.5937in" id="x_Picture_x0020_5" src="cid:image001.jpg@01DA64AF.5BBBA710" alt="Image result for university of durham logo">
<img border="0" width="118" height="55" style="width:1.2291in;height:.5729in" id="x_Picture_x0020_4" src="cid:image002.png@01DA64AF.5BBBA710" alt="signature_2975123418"> <img border="0" width="94" height="55" style="width:.9791in;height:.5729in" id="x_Picture_x0020_3" src="cid:image003.png@01DA64AF.5BBBA710" alt="signature_2364801924"> <img border="0" width="46" height="56" style="width:.4791in;height:.5833in" id="x_Picture_x0020_2" src="cid:image004.jpg@01DA64AF.5BBBA710" alt="Image result for durham cvac"></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Professor of Psychology, University of Durham.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham PaleoPsychology Group.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham Centre for Vision and Visual Cognition.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham Centre for Visual Arts and Culture.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"><img border="0" width="49" height="49" style="width:.5104in;height:.5104in" id="x_Picture_x0020_1" src="cid:image005.jpg@01DA64AF.5BBBA710" alt="9k="></span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Fellow. </span>
<span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Canadian Institute for Advanced Research,
</span><span lang="EN-GB"><o:p></o:p></span></p>
<div style="border:none;border-bottom:solid windowtext 1.0pt;padding:0in 0in 1.0pt 0in">
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Brain, Mind & Consciousness Programme.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Department of Psychology,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">University of Durham,</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Durham DH1 3LE, UK.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">p: +44 191 334 3261</span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">f: +44 191 334 3434</span><span lang="EN-GB"><o:p></o:p></span></p>
<div style="border:none;border-bottom:solid windowtext 1.0pt;padding:0in 0in 1.0pt 0in">
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<div id="x_mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Gary Marcus <gary.marcus@nyu.edu><br>
<b>Date: </b>Wednesday, 21 February 2024 at 05:49<br>
<b>To: </b>Laurent Mertens <laurent.mertens@kuleuven.be><br>
<b>Cc: </b>connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><strong><span lang="EN-GB" style="font-size:12.0pt;font-family:"Calibri",sans-serif;color:black;background:#FFFECF">[EXTERNAL EMAIL]</span></strong><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt">Deeply disappointing that someone would try to inject actual empirical evidence into this discussion.
</span><span lang="EN-GB" style="font-size:11.0pt;font-family:"Apple Color Emoji"">😂</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal" style="margin-bottom:12.0pt"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal" style="margin-bottom:12.0pt"><span lang="EN-GB" style="font-size:11.0pt">On Feb 20, 2024, at 08:41, Laurent Mertens <laurent.mertens@kuleuven.be> wrote:</span><span lang="EN-GB"><o:p></o:p></span></p>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Reacting to your statement:</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">"However, inside the skull of my brain, there are not any neurons that have a one-to-one correspondence to the symbol."</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">What about the Grandmother/Jennifer Aniston/Halle Berry neuron?</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">(See, e.g.,
</span><span lang="EN-GB"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.caltech.edu_about_news_single-2Dcell-2Drecognition-2Dhalle-2Dberry-2Dbrain-2Dcell-2D1013&d=DwMFAw&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=it3XOFrc2yBru1bmF9dud4UoT60mjmur8mR3zGu365JPKmtWSuFnJTxRJOV4WSpa&s=kh-rqxQw6qcxbM8bhUYTHNaJHN5jtc3SLI5RXC5XgWA&e="><span style="font-size:12.0pt;font-family:"Aptos",sans-serif">https://www.caltech.edu/about/news/single-cell-recognition-halle-berry-brain-cell-1013</span></a></span><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">)</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">KR,</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Laurent</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif">
<hr size="0" width="69%" align="center">
</span></div>
<div id="x_divRplyFwdMsg">
<p class="xmsonormal"><b><span lang="EN-GB" style="font-size:11.0pt;color:black">From:</span></b><span lang="EN-GB" style="font-size:11.0pt;color:black"> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Weng, Juyang <weng@msu.edu><br>
<b>Sent:</b> Monday, February 19, 2024 11:11 PM<br>
<b>To:</b> Michael Arbib <arbib@usc.edu>; connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject:</b> Re: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB" style="font-size:11.0pt">
</span><span lang="EN-GB"><o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Dear Michael,</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> You wrote, "Your brain did not deal with symbols?"</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> I have my Conscious Learning (DN-3) model that tells me:<br>
My brain "deals with symbols" that are sensed from the extra-body world by the brain's sensors and effecters.</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> However, inside the skull of my brain, there are not any neurons that have a one-to-one correspondence to the symbol. In this sense, the brain
does not have any symbol in the skull.</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> This is my educated hypothesis. The DN-3 brain does not need any symbol inside the skull.</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> In this sense, almost all neural network models are flawed about the brain, as long as they have a block diagram where each block corresponds to
a function concept in the extra-body world. I am sorry to say that, which may make many enemies. </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black"> Best regards,</span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">-John </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif">
<hr size="0" width="69%" align="center">
</span></div>
<div id="x_x_divRplyFwdMsg">
<p class="xmsonormal"><b><span lang="EN-GB" style="font-size:11.0pt;color:black">From:</span></b><span lang="EN-GB" style="font-size:11.0pt;color:black"> Michael Arbib <arbib@usc.edu><br>
<b>Sent:</b> Monday, February 19, 2024 1:28 PM<br>
<b>To:</b> Weng, Juyang <weng@msu.edu>; connectionists@mailman.srv.cs.cmu.edu <connectionists@mailman.srv.cs.cmu.edu><br>
<b>Subject:</b> RE: Connectionists: Early history of symbolic and neural network approaches to AI</span><span lang="EN-GB" style="font-size:11.0pt">
</span><span lang="EN-GB"><o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span lang="EN-GB" style="font-size:11.0pt"> </span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
</div>
<p><span lang="EN-GB">So you believe that, as you wrote out these words, the neural networks in your brain did not deal with symbols?<o:p></o:p></span></p>
<p><span lang="EN-GB"> <o:p></o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p><b><span lang="EN-GB">From:</span></b><span lang="EN-GB"> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu>
<b>On Behalf Of </b>Weng, Juyang<br>
<b>Sent:</b> Monday, February 19, 2024 8:07 AM<br>
<b>To:</b> connectionists@mailman.srv.cs.cmu.edu<br>
<b>Subject:</b> Connectionists: Early history of symbolic and neural network approaches to AI<o:p></o:p></span></p>
</div>
<p><span lang="EN-GB"> <o:p></o:p></span></p>
<p><span lang="EN-GB" style="color:black">I do not agree with <span style="background:white">
Newell and Simon</span> if they wrote that. Otherwise, images and video are also symbols. They probably were not sophisticated enough in 1976 to realize why neural networks in the brain should not contain or deal with symbols.</span><span lang="EN-GB"><o:p></o:p></span></p>
<p style="margin-bottom:12.0pt"><span lang="EN-GB"> <o:p></o:p></span></p>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>