<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:958296370;
mso-list-template-ids:-593845046;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Dear John,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If I understand correctly, all learning systems do something along the lines of maximum likelihood learning or error minimization, like your DN. What’s your point?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="background:lime;mso-highlight:lime">JOHN: </span>
<i><span style="background:yellow;mso-highlight:yellow">“Of course, the brain network does not remember all shapes and all configurations of parts. That is why our DN must do maximum likelihood optimality, using a limited number of resources to best estimate
such a huge space of cluttered scenes.”</span><o:p></o:p></i></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So, can your DN model identify the parts of objects in the cluttered images below? Here was my note:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="background:lime;mso-highlight:lime">ASIM:</span><span style="background:yellow;mso-highlight:yellow">
<i>“And we can also identify parts of wholes in these scenes. Here are some example scenes. In the first two scenes, we can identify the huskies along with the ears, eyes, legs, faces and so on. In the satellite image below, we can identify parts of the planes
like the fuselage, tail, wing and so on. That’s the fundamental part of DARPA’s XAI model – to be able to
</i></span><i><span style="background:red;mso-highlight:red">identify the parts to confirm the whole object</span><span style="background:yellow;mso-highlight:yellow">. And if you can identify the parts, a school bus will never become an ostrich with change
of a few pixels. So you get a lot of things with Explainable models of this form –
</span><span style="background:red;mso-highlight:red">a symbolic XAI model, robustness against adversarial attacks, and a model that you can trust</span><span style="background:yellow;mso-highlight:yellow">. Explainable AI of this form can become the best defense
against adversarial attacks. You may not need any adversarial training of any kind.</span>”<o:p></o:p></i></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best,<o:p></o:p></p>
<p class="MsoNormal">Asim Roy<o:p></o:p></p>
<p class="MsoNormal">Professor, Information Systems<o:p></o:p></p>
<p class="MsoNormal">Arizona State University<o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lifeboat.com_ex_bios.asim.roy&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=oDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw&e=" target="_blank">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__isearch.asu.edu_profile_9973&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=jCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro&e=" target="_blank">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><img border="0" width="480" height="360" style="width:5.0in;height:3.75in" id="_x0000_i1030" src="cid:image001.jpg@01D81F7F.B4F08980" alt="A dog and a cat lying on a bed
Description automatically generated with low confidence">
<img border="0" width="308" height="231" style="width:3.2083in;height:2.4062in" id="_x0000_i1029" src="cid:image002.jpg@01D81F7F.B4F08980" alt="A wolf walking in the snow
Description automatically generated with medium confidence"> <img border="0" width="1443" height="1452" style="width:15.0312in;height:15.125in" id="_x0000_i1028" src="cid:image003.jpg@01D81F7F.B4F08980" alt="An aerial view of a city
Description automatically generated with medium confidence"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Juyang Weng <juyang.weng@gmail.com> <br>
<b>Sent:</b> Friday, February 11, 2022 3:40 PM<br>
<b>To:</b> Asim Roy <ASIM.ROY@asu.edu><br>
<b>Cc:</b> John K Tsotsos <tsotsos@cse.yorku.ca>; connectionists@mailman.srv.cs.cmu.edu; Gary Marcus <gary.marcus@nyu.edu><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in conversation with Geoff Hinton<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Dear Asim,<o:p></o:p></p>
<div>
<p class="MsoNormal">We should not just assume your 1, a dog, since there are many articulated objects and each articulated object looks very different under many situations. There are further many other articulated objects other than the dog. You will
see below.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Since you want explainable AI, you must not start with a single object symbol like "dog". A symbol is "abstract" already. That is way Michael Jordan complained that neural networks do not abstract well.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Let us do your 2. How many possible combinations?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The number of shapes of a part: Suppose a part of a uniform color. It has m pixels along its boundary; each pixel has n possible positions, the number of shapes of a part is O(m^n), already exponential in n. We assumed that each pixel
has the same color, which is not true.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Suppose that each part is centered at location l, the number of combinations of p parts of your object (dog) is O(p^l), another exponential complexity.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Thus the number of combinations of your object (dog) in a clean background, like Jeffe Hinton proposed to do, is already a product two two exponential complexities:<br>
O(m^n)O(p^l)=O(m^{n}p^{l}).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Suppose that there are b objects in a cluttered scene, the number of combination of objects in a cluttered scene is
<br>
[O(m^{n}p^{l})]^b=O(m^{nb}p^{lb}).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Of course, the brain network does not remember all shapes and all configurations of parts. That is why our DN must do maximum likelihood optimality, using a limited number of resources to best estimate such
a huge space of cluttered scenes. <br>
I am not talking about abstraction yet, which is another subject about "pop up" in the brain.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I guess that many people on this list are not familiar with such a complexity analysis. Gary Marcus, sorry to overload you with this.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">That is what I said in several talks that the brain is an elephant and all disciplines are blind men. <br>
Even people with a PhD in computer science may not be skillful in such an exponential complexity in vision, since many computer science programs have dropped automata theory from their required course lists. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I asked Gary Marcus to suggest how to solve this huge problem. But instead, he asked me to try a data set which is a dead end. Many have tried and are dead, like ImageNet.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Best regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-Joh<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Feb 11, 2022 at 4:59 PM Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear John,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
Let’s start with a simple case, say a dog, and enumerate how many possible parts and objects a dog would need to remember or recognize.<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
How many possible combinations did you use in your calculation for the DN?<o:p></o:p></li></ol>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Best,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Asim<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>>
<br>
<b>Sent:</b> Friday, February 11, 2022 12:33 PM<br>
<b>To:</b> Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>>; John K Tsotsos <<a href="mailto:tsotsos@cse.yorku.ca" target="_blank">tsotsos@cse.yorku.ca</a>><br>
<b>Cc:</b> <a href="mailto:connectionists@mailman.srv.cs.cmu.edu" target="_blank">
connectionists@mailman.srv.cs.cmu.edu</a>; Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" target="_blank">gary.marcus@nyu.edu</a>><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in conversation with Geoff Hinton<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear Asim,<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thank you for saying "we can". <br>
Please provide: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(1) a neural network that does all you said "we can" and<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">(2) the complexity analysis for all possible combinations among all possible parts and all possible objects<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">This chain of conversations is very useful for those who are not yet familiar with the "complexity of vision" (NP hard) that John Tsotso wrote papers argued about.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">John Tsotso: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Our DN solves this problem like a brain in a constant time (frame time)! The solution simply pops up.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Best regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-John<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Feb 10, 2022 at 3:01 AM Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear John,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We can deal with cluttered scenes. And we can also identify parts of wholes in these scenes. Here are some example scenes. In the first two scenes, we can identify the huskies along
with the ears, eyes, legs, faces and so on. In the satellite image below, we can identify parts of the planes like the fuselage, tail, wing and so on. That’s the fundamental part of DARPA’s XAI model – to be able to identify the parts to confirm the whole
object. And if you can identify the parts, a school bus will never become an ostrich with change of a few pixels. So you get a lot of things with Explainable models of this form – a symbolic XAI model, robustness against adversarial attacks, and a model that
you can trust. Explainable AI of this form can become the best defense against adversarial attacks. You may not need any adversarial training of any kind.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Best,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Asim Roy<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Professor, Information Systems<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Arizona State University<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lifeboat.com_ex_bios.asim.roy&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=oDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw&e=" target="_blank">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__isearch.asu.edu_profile_9973&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=jCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro&e=" target="_blank">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<img border="0" width="480" height="360" style="width:5.0in;height:3.75in" id="m_6087161075607456294gmail-m_2055946567400815365gmail-m_5005686710730097958gmail-m_4384369736864902407Picture_x0020_1" src="cid:image001.jpg@01D81F7F.B4F08980" alt="A dog and a cat lying on a bed
Description automatically generated with low confidence"> <img border="0" width="308" height="231" style="width:3.2083in;height:2.4062in" id="m_6087161075607456294gmail-m_2055946567400815365gmail-m_5005686710730097958gmail-m_4384369736864902407Picture_x0020_3" src="cid:image002.jpg@01D81F7F.B4F08980" alt="A wolf walking in the snow
Description automatically generated with medium confidence"> <img border="0" width="1443" height="1452" style="width:15.0312in;height:15.125in" id="m_6087161075607456294gmail-m_2055946567400815365gmail-m_5005686710730097958gmail-m_4384369736864902407Picture_x0020_6" src="cid:image003.jpg@01D81F7F.B4F08980" alt="An aerial view of a city
Description automatically generated with medium confidence"><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> Connectionists <<a href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu" target="_blank">connectionists-bounces@mailman.srv.cs.cmu.edu</a>>
<b>On Behalf Of </b>Juyang Weng<br>
<b>Sent:</b> Wednesday, February 9, 2022 3:19 PM<br>
<b>To:</b> Post Connectionists <<a href="mailto:connectionists@mailman.srv.cs.cmu.edu" target="_blank">connectionists@mailman.srv.cs.cmu.edu</a>><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in conversation with Geoff Hinton<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear Gary,<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As my reply to Asim Roy indicated, the parts and whole problem that Geoff Hinton considered is ill-posed since it bypasses how a brain network segments the "whole" from 1000 parts
in the cluttered scene. Only 10 parts belong to the whole.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The relation problem has also been solved and mathematically proven if one understands emergent universal Turing machines using a Developmental Network (DN). The solution to relation
is a special case of the solution to the compositionality problem which is a special case of the emergent universal Turing machine.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am not telling you "a son looks like his father because the father makes money to feed the son". The solution is supported by biology and a mathematical proof.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
Best regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-John<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Date: Mon, 7 Feb 2022 07:57:34 -0800<br>
From: Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" target="_blank">gary.marcus@nyu.edu</a>><br>
To: Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>><br>
Cc: Post Connectionists <<a href="mailto:connectionists@mailman.srv.cs.cmu.edu" target="_blank">connectionists@mailman.srv.cs.cmu.edu</a>><br>
Subject: Re: Connectionists: Stephen Hanson in conversation with Geoff<br>
Hinton<br>
Message-ID: <<a href="mailto:D0E77E54-78C0-4605-B40C-434E2B8F1E7C@nyu.edu" target="_blank">D0E77E54-78C0-4605-B40C-434E2B8F1E7C@nyu.edu</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Dear John,<br>
<br>
I agree with you that cluttered scenes are critical, but Geoff?s GLOM paper [<a href="https://urldefense.com/v3/__https:/www.cs.toronto.edu/*hinton/absps/glomfinal.pdf__;fg!!IKRxdwAv5BmarQ!Nz1SeTUV0HTHgPjgQgoT1IgAHrVhxdw8HVVMwgs83QlxthT1NyY5hgDxKe34wLc$" target="_blank">https://www.cs.toronto.edu/~hinton/absps/glomfinal.pdf</a>]
might actually have some relevance. It may well be that we need to do a better job with parts and whole before we can fully address clutter, and Geoff is certainly taking that question seriously.<br>
<br>
Geoff?s ?Stable islands of identical vectors? do sound suspiciously like symbols to me (in a good way!), but regardless, they seem to me to be a plausible candidate as a foundation for coping with clutter.<br>
<br>
And not just cluttered scenes, but also relations between multiple objects in a scene, which is another example of the broader issue you raise, challenging for pure MLPs but critical for deeper AI.<br>
<br>
Gary<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Juyang (John) Weng<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Juyang (John) Weng<o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Juyang (John) Weng<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>