<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#ecca99">
<p><font size="+1">Tom, understanding is a theorem?</font></p>
<p><font size="+1">you mean it should be a theorem?</font></p>
<p><font size="+1">and yes, if you are having brain surgery.. you
hope your surgeon, "understands" what they are doing..</font></p>
<p><font size="+1">Steve<br>
</font></p>
<div class="moz-cite-prefix">On 2/3/22 12:31 PM, Dietterich, Thomas
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:PH0P222MB0026CD1AB01077BAA6B104EFBE289@PH0P222MB0026.NAMP222.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:UICTFontTextStyleBody;
panose-1:0 0 0 0 0 0 0 0 0 0;}@font-face
{font-family:UICTFontTextStyleItalicBody;
panose-1:0 0 0 0 0 0 0 0 0 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">“Understanding” is not a Boolean. It is a
theorem that no system can enumerate all of the consequences
of a state of affairs in the world.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For low-stakes application work, we can be
satisfied by a system that “does the right thing”. If the
system draws a good picture, that’s sufficient. It
“understood” the request.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But for higher-stakes applications---and
for advancing the science---we seek a causal account of how
the components of a system cause it to do the right thing. We
are hoping that a small set of mechanisms can produce broad
coverage of intelligent behavior. This gives us confidence
that the system will respond correctly outside of the narrow
tasks on which we have tested it.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">--Tom <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">Thomas G. Dietterich,
Distinguished Professor Emeritus<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">School of Electrical Engineering
and Computer Science
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">US Mail: 1148 Kelley Engineering
Center
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">Office: 2067 Kelley Engineering
Center
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">Oregon State Univ., Corvallis, OR
97331-5501<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">Voice: 541-737-5559; FAX:
541-737-1300<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier
New";color:#1F497D">URL: <a
href="http://web.engr.oregonstate.edu/~tgd/"
moz-do-not-send="true">
http://web.engr.oregonstate.edu/~tgd/</a><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Connectionists
<a class="moz-txt-link-rfc2396E" href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu"><connectionists-bounces@mailman.srv.cs.cmu.edu></a>
<b>On Behalf Of </b>Gary Marcus<br>
<b>Sent:</b> Thursday, February 3, 2022 8:26 AM<br>
<b>To:</b> Danko Nikolic <a class="moz-txt-link-rfc2396E" href="mailto:danko.nikolic@gmail.com"><danko.nikolic@gmail.com></a><br>
<b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:connectionists@mailman.srv.cs.cmu.edu">connectionists@mailman.srv.cs.cmu.edu</a>; AIhub
<a class="moz-txt-link-rfc2396E" href="mailto:aihuborg@gmail.com"><aihuborg@gmail.com></a><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in
conversation with Geoff Hinton<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="color:#D73F09">[This email originated from
outside of OSU. Use caution with links and attachments.]</span><o:p></o:p></p>
<div>
<p class="MsoNormal">Dear Danko, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Well said. I had a somewhat similar
response to Jeff Dean’s 2021 TED talk, in which he said
(paraphrasing from memory, because I don’t remember the
precise words) that the famous 200 Quoc Le unsupervised
model [<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatic.googleusercontent.com%2Fmedia%2Fresearch.google.com%2Fen%2F%2Farchive%2Funsupervised_icml2012.pdf&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=l1WwVtsu%2BBMn0UfnWdN7tHCdsWTdIi9P%2Ffd50ThMgEs%3D&reserved=0"
moz-do-not-send="true">https://static.googleusercontent.com/media/research.google.com/en//archive/unsupervised_icml2012.pdf</a>]
had learned the concept of a ca. In reality the model had
clustered together some catlike images based on the image
statistics that it had extracted, but it was a long way
from a full, counterfactual-supporting concept of a cat,
much as you describe below. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I fully agree with you that the reason
for even having a semantics is as you put it, "to 1) learn
with a few examples and 2) apply the knowledge to a broad
set of situations.” GPT-3 sometimes gives the appearance
of having done so, but it falls apart under close
inspection, so the problem remains unsolved.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Gary<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Feb 3, 2022, at 3:19 AM,
Danko Nikolic <<a
href="mailto:danko.nikolic@gmail.com"
moz-do-not-send="true">danko.nikolic@gmail.com</a>>
wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">G. Hinton wrote: "I believe
that any reasonable person would admit that if
you ask a neural net to draw a picture of a
hamster wearing a red hat and it draws such a
picture, it understood the request."
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I would like to suggest why
drawing a hamster with a red hat does not
necessarily imply understanding of the
statement "hamster wearing a red hat".<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">To understand that "hamster
wearing a red hat" would mean inferring, in
newly emerging situations of this hamster, all
the real-life implications that the red hat
brings to the little animal.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">What would happen to the
hat if the hamster rolls on its back? (Would
the hat fall off?)<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">What would happen to the
red hat when the hamster enters its lair?
(Would the hat fall off?)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">What would happen to that
hamster when it goes foraging? (Would the red
hat have an influence on finding food?)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">What would happen in a
situation of being chased by a predator?
(Would it be easier for predators to spot the
hamster?)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">...and so on.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Countless many questions
can be asked. One has understood "hamster
wearing a red hat" only if one can answer
reasonably well many of such real-life
relevant questions. Similarly, a student
has understood materias in a class only if
they can apply the materials in real-life
situations (e.g., applying Pythagora's
theorem). If a student gives a correct answer
to a multiple choice question, we don't know
whether the student understood the material or
whether this was just rote learning (often, it
is rote learning). <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I also suggest that
understanding also comes together with
effective learning: We store new information
in such a way that we can recall it later and
use it effectively i.e., make good inferences
in newly emerging situations based on this
knowledge.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In short: Understanding
makes us humans able to 1) learn with a few
examples and 2) apply the knowledge to a broad
set of situations. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">No neural network today has
such capabilities and we don't know how to
give them such capabilities. Neural networks
need large amounts of training examples that
cover a large variety of situations and then
the networks can only deal with what the
training examples have already covered. Neural
networks cannot extrapolate in that
'understanding' sense.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I suggest that
understanding truly extrapolates from a piece
of knowledge. It is not about satisfying a
task such as translation between languages or
drawing hamsters with hats. It is how you got
the capability to complete the task: Did you
only have a few examples that covered
something different but related and then you
extrapolated from that knowledge? If yes, this
is going in the direction of understanding.
Have you seen countless examples and then
interpolated among them? Then perhaps it is
not understanding.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">So, for the case of drawing
a hamster wearing a red hat, understanding
perhaps would have taken place if the
following happened before that:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">1) first, the network
learned about hamsters (not many examples)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">2) after that the network
learned about red hats (outside the context of
hamsters and without many examples) <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">3) finally the network
learned about drawing (outside of the context
of hats and hamsters, not many examples)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">After that, the network is
asked to draw a hamster with a red hat. If it
does it successfully, maybe we have started
cracking the problem of understanding.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Note also that this
requires the network to learn sequentially
without exhibiting catastrophic forgetting of
the previous knowledge, which is possibly also
a consequence of human learning by
understanding.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Danko<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Dr. Danko Nikolić<br>
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__www.danko-2Dnikolic.com%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DHwOLDw6UCRzU5-FPSceKjtpNm7C6sZQU5kuGAMVbPaI%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4tckqFTDoD5PoPbncU5nuSw1jRQ5yHbl1rEZExji4iQ%3D&reserved=0"
target="_blank" moz-do-not-send="true">www.danko-nikolic.com</a><br>
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.linkedin.com_in_danko-2Dnikolic_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3Db70c8lokmxM3Kz66OfMIM4pROgAhTJOAlp205vOmCQ8%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=NJoyEx8ZVu82e0Tw5u%2BNCEQtcJ8tW%2BnOpgD%2FwkVTZ%2FM%3D&reserved=0"
target="_blank" moz-do-not-send="true">https://www.linkedin.com/in/danko-nikolic/</a>
<o:p></o:p></p>
<div>
<p class="MsoNormal">--- A progress
usually starts with an insight ---<o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2">
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable"
style="border:none;border-top:solid #D3D4DE
1.0pt" cellpadding="0" border="1">
<tbody>
<tr>
<td
style="width:41.25pt;border:none;padding:9.75pt
.75pt .75pt .75pt" width="55">
<p class="MsoNormal"><a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.avast.com_sig-2Demail-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dlink-26utm-5Fcampaign-3Dsig-2Demail-26utm-5Fcontent-3Dwebmail%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DAo9QQWtO62go0hx1tb3NU6xw2FNBadjj8q64-hl5Sx4%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gRfN5WuMr4%2BLwZMRqPOPaTJ97VoUB%2FVRMELwu5lQeiU%3D&reserved=0"
target="_blank" moz-do-not-send="true"><span
style="text-decoration:none"><img
style="width:.4791in;height:.302in"
id="_x0000_i1025"
src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif"
moz-do-not-send="true" width="46"
height="29" border="0"></span></a><o:p></o:p></p>
</td>
<td
style="width:352.5pt;border:none;padding:9.0pt
.75pt .75pt .75pt" width="470">
<p class="MsoNormal"
style="line-height:13.5pt"><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#41424E">Virus-free.
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.avast.com_sig-2Demail-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dlink-26utm-5Fcampaign-3Dsig-2Demail-26utm-5Fcontent-3Dwebmail%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DAo9QQWtO62go0hx1tb3NU6xw2FNBadjj8q64-hl5Sx4%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gRfN5WuMr4%2BLwZMRqPOPaTJ97VoUB%2FVRMELwu5lQeiU%3D&reserved=0"
target="_blank"
moz-do-not-send="true">
<span style="color:#4453EA">www.avast.com</span></a>
<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Feb 3, 2022 at 9:55
AM Asim Roy <<a
href="mailto:ASIM.ROY@asu.edu"
moz-do-not-send="true">ASIM.ROY@asu.edu</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid
#CCCCCC 1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Without
getting into the specific dispute between
Gary and Geoff, I think with approaches
similar to GLOM, we are finally headed in
the right direction. There’s plenty of
neurophysiological evidence for
single-cell abstractions and multisensory
neurons in the brain, which one might
claim correspond to symbols. And I think
we can finally reconcile the decades old
dispute between Symbolic AI and
Connectionism.<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="background:yellow">GARY: (Your
GLOM, which as you know I praised
publicly, is in many ways an effort to
wind up with encodings that effectively
serve as symbols in exactly that way,
guaranteed to serve as consistent
representations of specific concepts.)</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="background:yellow">GARY: I have
<i>never</i> called for dismissal of
neural networks, but rather for some
hybrid between the two (as you yourself
contemplated in 1991); the point of the
2001 book was to characterize exactly
where multilayer perceptrons succeeded
and broke down, and where symbols could
complement them.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Asim
Roy<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Professor,
Information Systems<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Arizona
State University<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__lifeboat.com_ex_bios.asim.roy%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DoDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZfFq3zPrJN82BL7HTseBtLgleW3A9INVS1X09McIIMM%3D&reserved=0"
target="_blank" moz-do-not-send="true">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__isearch.asu.edu_profile_9973%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DjCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aYIb70CDbZx%2BLtTyEHog2F06H%2F2W9QT1sF8W2VmBWJg%3D&reserved=0"
target="_blank" moz-do-not-send="true">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid
#E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b>
Connectionists <<a
href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu"
target="_blank"
moz-do-not-send="true">connectionists-bounces@mailman.srv.cs.cmu.edu</a>>
<b>On Behalf Of </b>Gary Marcus<br>
<b>Sent:</b> Wednesday, February 2,
2022 1:26 PM<br>
<b>To:</b> Geoffrey Hinton <<a
href="mailto:geoffrey.hinton@gmail.com"
target="_blank"
moz-do-not-send="true">geoffrey.hinton@gmail.com</a>><br>
<b>Cc:</b> AIhub <<a
href="mailto:aihuborg@gmail.com"
target="_blank"
moz-do-not-send="true">aihuborg@gmail.com</a>>;
<a
href="mailto:connectionists@mailman.srv.cs.cmu.edu"
target="_blank"
moz-do-not-send="true">connectionists@mailman.srv.cs.cmu.edu</a><br>
<b>Subject:</b> Re: Connectionists:
Stephen Hanson in conversation with
Geoff Hinton<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear
Geoff, and interested others,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">What,
for example, would you make of a
system that often drew the red-hatted
hamster you requested, and perhaps a
fifth of the time gave you utter
nonsense? Or say one that you trained
to create birds but sometimes output
stuff like this:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><image001.png><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">One
could <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">a.
avert one’s eyes and deem the
anomalous outputs irrelevant<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">or<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">b.
wonder if it might be possible that
sometimes the system gets the right
answer for the wrong reasons (eg
partial historical contingency), and
wonder whether another approach might
be indicated.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Benchmarks
are harder than they look; most of the
field has come to recognize that. The
Turing Test has turned out to be a
lousy measure of intelligence, easily
gamed. It has turned out empirically
that the Winograd Schema Challenge did
not measure common sense as well as
Hector might have thought. (As it
happens, I am a minor coauthor of a
very recent review on this very
topic: <a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Farxiv.org%2Fabs%2F2201.02387__%3B!!IKRxdwAv5BmarQ!INA0AMmG3iD1B8MDtLfjWCwcBjxO-e-eM2Ci9KEO_XYOiIEgiywK-G_8j6L3bHA%24&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=muB9%2FpE63uI65lV8LwulXTZQWRRVVsH89PCIcp6TAcA%3D&reserved=0"
target="_blank"
moz-do-not-send="true">https://arxiv.org/abs/2201.02387</a>)
But its conquest in no way means
machines now have common sense; many
people from many different
perspectives recognize that
(including, e.g., Yann LeCun, who
generally tends to be more aligned
with you than with me).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">So:
on the goalpost of the Winograd
schema, I was wrong, and you can quote
me; but what you said about me and
machine translation remains your
invention, and it is inexcusable that
you simply ignored my 2019
clarification. On the essential goal
of trying to reach meaning and
understanding, I remain unmoved; the
problem remains unsolved. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">All
of the problems LLMs have with
coherence, reliability, truthfulness,
misinformation, etc stand witness to
that fact. (Their persistent inability
to filter out toxic and insulting
remarks stems from the same.) I am
hardly the only person in the field to
see that progress on any given
benchmark does not inherently mean
that the deep underlying problems have
solved. You, yourself, in fact, have
occasionally made that point. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">With
respect to embeddings: Embeddings are
very good for natural language
<i>processing</i>; but NLP is not the
same as NL<i>U</i> – when it comes to
<i>understanding</i>, their worth is
still an open question. Perhaps they
will turn out to be necessary; they
clearly aren’t sufficient. In their
extreme, they might even collapse into
being symbols, in the sense of
uniquely identifiable encodings, akin
to the ASCII code, in which a specific
set of numbers stands for a specific
word or concept. (Wouldn’t that be
ironic?)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(Your
GLOM, which as you know I praised
publicly, is in many ways an effort to
wind up with encodings that
effectively serve as symbols in
exactly that way, guaranteed to serve
as consistent representations of
specific concepts.)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Notably
absent from your email is any kind of
apology for misrepresenting my
position. It’s fine to say that “many
people thirty years ago once thought
X” and another to say “Gary Marcus
said X in 2015”, when I didn’t. I have
consistently felt throughout our
interactions that you have mistaken me
for Zenon Pylyshyn; indeed, you once
(at NeurIPS 2014) apologized to me for
having made that error. I am still not
he. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Which
maybe connects to the last point; if
you read my work, you would see thirty
years of arguments
<i>for</i> neural networks, just not
in the way that you want them to
exist. I have ALWAYS argued that there
is a role for them; characterizing me
as a person “strongly opposed to
neural networks” misses the whole
point of my 2001 book, which was
subtitled “Integrating Connectionism
and Cognitive Science.”<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">In
the last two decades or so you have
insisted (for reasons you have never
fully clarified, so far as I know) on
abandoning symbol-manipulation, but
the reverse is not the case: I have <i>never</i>
called for dismissal of neural
networks, but rather for some hybrid
between the two (as you yourself
contemplated in 1991); the point of
the 2001 book was to characterize
exactly where multilayer perceptrons
succeeded and broke down, and where
symbols could complement them. It’s a
rhetorical trick (which is what the
previous thread was about) to pretend
otherwise.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Gary<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;margin-bottom:12.0pt">On
Feb 2, 2022, at 11:22, Geoffrey
Hinton <<a
href="mailto:geoffrey.hinton@gmail.com"
target="_blank"
moz-do-not-send="true">geoffrey.hinton@gmail.com</a>>
wrote:<o:p></o:p></p>
</blockquote>
</div>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p></o:p></p>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Embeddings
are just vectors of soft feature
detectors and they are very good
for NLP. The quote on my webpage
from Gary's 2015 chapter implies
the opposite.<o:p></o:p></p>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">A
few decades ago, everyone I knew
then would have agreed that the
ability to translate a sentence
into many different languages
was strong evidence that you
understood it.<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><o:p> </o:p></p>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">But
once neural networks could do
that, their critics moved the
goalposts. An exception is
Hector Levesque who defined the
goalposts more sharply by saying
that the ability to get pronoun
references correct in Winograd
sentences is a crucial test.
Neural nets are improving at
that but still have some way to
go. Will Gary agree that when
they can get pronoun
references correct in Winograd
sentences they really do
understand? Or does he want to
reserve the right to weasel out
of that too?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Some
people, like Gary, appear to be
strongly opposed to neural
networks because they do not fit
their preconceived notions of
how the mind should work.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I
believe that any reasonable
person would admit that if you
ask a neural net to draw a
picture of a hamster wearing a
red hat and it draws such a
picture, it understood the
request.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Geoff<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On
Wed, Feb 2, 2022 at 1:38 PM Gary
Marcus <<a
href="mailto:gary.marcus@nyu.edu"
target="_blank"
moz-do-not-send="true">gary.marcus@nyu.edu</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote
style="border:none;border-left:solid
#CCCCCC 1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Dear
AI Hub, cc: Steven
Hanson and Geoffrey
Hinton, and the larger
neural network
community,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">There
has been a lot of
recent discussion on
this list about
framing and scientific
integrity. Often the
first step in
restructuring
narratives is to bully
and dehumanize
critics. The second is
to misrepresent their
position. People in
positions of power are
sometimes tempted to
do this.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">The
Hinton-Hanson
interview that you
just published is a
real-time example of
just that. It opens
with a needless and
largely content-free
personal attack on a
single scholar (me),
with the explicit
intention of
discrediting that
person. Worse, the
only substantive thing
it says is false.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Hinton
says “In 2015 he
[Marcus] made a
prediction that
computers wouldn’t be
able to do machine
translation.”</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">I
never said any such
thing. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">What
I predicted, rather,
was that multilayer
perceptrons, as they
existed then, would
not (on their own,
absent other
mechanisms) </span><i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleItalicBody",serif">understand</span></i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif"> language.
Seven years later,
they still haven’t,
except in the most
superficial way. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">I
made no comment
whatsoever about
machine translation,
which I view as a
separate problem,
solvable to a certain
degree by
correspondance without
semantics. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">I
specifically tried to
clarify Hinton’s
confusion in 2019,
but, disappointingly,
he has continued to
purvey misinformation
despite that
clarification. Here is
what I wrote privately
to him then, which
should have put the
matter to rest:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">You
have taken a single
out of context quote
[from 2015] and
misrepresented it. The
quote, which you have
prominently displayed
at the bottom on your
own web page, says:</span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal;min-height:22.9px">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div
style="margin-left:.75in;font-stretch:normal">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Hierarchies
of features are less
suited to challenges
such as language,
inference, and
high-level planning.
For example, as Noam
Chomsky famously
pointed out, language
is filled with
sentences you haven't
seen before. Pure
classifier systems
don't know what to do
with such sentences.
The talent of feature
detectors --
in identifying which
member of some
category something
belongs to -- doesn't
translate into
understanding
novel sentences, in
which each sentence
has its own unique
meaning. </span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal;min-height:22.9px">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">It
does </span><i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleItalicBody",serif">not</span></i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif"> say
"neural nets would not
be able to deal with
novel sentences"; it
says that hierachies
of features detectors
(on their own, if you
read the context of
the essay) would have
trouble </span><i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleItalicBody",serif">understanding </span></i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">novel sentences.
</span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal;min-height:22.9px">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Google
Translate does yet
not </span><i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleItalicBody",serif">understand</span></i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif"> the
content of the
sentences is
translates. It cannot
reliably answer
questions about who
did what to whom, or
why, it cannot infer
the order of the
events in paragraphs,
it can't determine the
internal consistency
of those events, and
so forth.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Since
then, a number of
scholars, such as the
the computational
linguist Emily Bender,
have made similar
points, and indeed
current LLM
difficulties with
misinformation,
incoherence and
fabrication all follow
from these concerns.
Quoting from Bender’s
prizewinning 2020 ACL
article on the matter
with Alexander
Koller, <a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__aclanthology.org_2020.acl-2Dmain.463.pdf%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DxnFSVUARkfmiXtiTP_uXfFKv4uNEGgEeTluRFR7dnUpay2BM5EiLz-XYCkBNJLlL%26s%3DK-Vl6vSvzuYtRMi-s4j7mzPkNRTb-I6Zmf7rbuKEBpk%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mhs4l94QkmKDeNH2BalecsEBsKbCIlOYa7BX4TkXS4U%3D&reserved=0"
target="_blank"
moz-do-not-send="true">https://aclanthology.org/2020.acl-main.463.pdf</a>,
also emphasizing
issues of
understanding and
meaning:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div
style="margin-left:27.0pt;font-stretch:normal">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><i><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleItalicBody",serif">The
success of the large
neural language
models on many NLP
tasks is exciting.
However, we find
that these successes
sometimes lead to
hype in which these
models are being
described as
“understanding”
language or
capturing “meaning”.
In this position
paper, we argue that
a system trained
only on form has a
priori no way to
learn meaning. .. a
clear understanding
of the distinction
between form and
meaning will help
guide the field
towards better
science around
natural language
understanding. </span></i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Her
later article with
Gebru on language
models “stochastic
parrots” is in some
ways an extension of
this point; machine
translation requires
mimicry, true
understanding (which
is what I was
discussing in 2015)
requires something
deeper than that. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Hinton’s
intellectual error
here is in equating
machine translation
with the deeper
comprehension that
robust natural
language understanding
will require; as
Bender and Koller
observed, the two
appear not to be the
same. (There is a
longer discussion of
the relation between
language understanding
and machine
translation, and why
the latter has turned
out to be more
approachable than the
former, in my 2019
book with Ernest
Davis).</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">More
broadly, Hinton’s
ongoing dismissiveness
of research from
perspectives other
than his own (e.g.
linguistics) have done
the field a
disservice. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">As
Herb Simon once
observed, science does
not have to be
zero-sum.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Sincerely,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Gary
Marcus</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">Professor
Emeritus</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:13.0pt;font-family:"UICTFontTextStyleBody",serif">New
York University</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><o:p> </o:p></p>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;margin-bottom:12.0pt">On
Feb 2, 2022, at 06:12,
AIhub <<a
href="mailto:aihuborg@gmail.com"
target="_blank"
moz-do-not-send="true">aihuborg@gmail.com</a>>
wrote:<o:p></o:p></p>
</blockquote>
</div>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Stephen
Hanson in
conversation with
Geoff Hinton<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">In
the latest episode
of this video series
for
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__AIhub.org%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DxnFSVUARkfmiXtiTP_uXfFKv4uNEGgEeTluRFR7dnUpay2BM5EiLz-XYCkBNJLlL%26s%3DeOtzMh8ILIH5EF7K20Ks4Fr27XfNV_F24bkj-SPk-2A%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2QAoxDbJIG9ogvM30Xq42Qdm4y8Cx7iOy3HpDpph0sM%3D&reserved=0"
target="_blank"
moz-do-not-send="true">
AIhub.org</a>,
Stephen Hanson talks
to Geoff
Hinton about neural
networks,
backpropagation,
overparameterization,
digit recognition,
voxel cells, syntax
and semantics,
Winograd sentences,
and more.<o:p></o:p></p>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">You can watch
the discussion,
and read the
transcript, here:<br
clear="all">
<o:p></o:p></p>
<div>
<p
class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__aihub.org_2022_02_02_what-2Dis-2Dai-2Dstephen-2Dhanson-2Din-2Dconversation-2Dwith-2Dgeoff-2Dhinton_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DOY_RYGrfxOqV7XeNJDHuzE--aEtmNRaEyQ0VJkqFCWw%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wNqUnQ5%2BASwQHX76s9dIWeiL5cQWSeKSCQBHd6yhQ6U%3D&reserved=0"
target="_blank" moz-do-not-send="true">https://aihub.org/2022/02/02/what-is-ai-stephen-hanson-in-conversation-with-geoff-hinton/</a><o:p></o:p></p>
</div>
<div>
<p
class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p
class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-family:"Arial",sans-serif">About
AIhub: </span><o:p></o:p></p>
</div>
<div>
<p
class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-family:"Arial",sans-serif">AIhub
is a
non-profit
dedicated to
connecting the
AI community
to the public
by providing
free,
high-quality
information
through
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__AIhub.org%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DxnFSVUARkfmiXtiTP_uXfFKv4uNEGgEeTluRFR7dnUpay2BM5EiLz-XYCkBNJLlL%26s%3DeOtzMh8ILIH5EF7K20Ks4Fr27XfNV_F24bkj-SPk-2A%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2QAoxDbJIG9ogvM30Xq42Qdm4y8Cx7iOy3HpDpph0sM%3D&reserved=0"
target="_blank" moz-do-not-send="true">
AIhub.org</a>
(<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__aihub.org_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DIKFanqeMi73gOiS7yD-X_vRx_OqDAwv1Il5psrxnhIA%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BDcjgXKucBci9yelMlpE1%2B79V8%2FfD9O8YFE4gdU2igA%3D&reserved=0"
target="_blank" moz-do-not-send="true">https://aihub.org/</a>). We help
researchers
publish the
latest AI
news,
summaries of
their work,
opinion
pieces,
tutorials and
more. We are
supported by
many leading
scientific
organizations
in AI, namely
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__aaai.org_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DwBvjOWTzEkbfFAGNj9wOaiJlXMODmHNcoWO5JYHugS0%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Rg%2FxVcf%2BIbFFXsCXesENBfEmQGoB2xZoy1S89jKpUFU%3D&reserved=0"
target="_blank" moz-do-not-send="true">
AAAI</a>, <a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__neurips.cc_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3D3-lOHXyu8171pT_UE9hYWwK6ft4I-cvYkuX7shC00w0%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=W2%2B7kXY%2FmTHCRZpViZh%2BYacUOAAvCv3KMvPc644p49o%3D&reserved=0"
target="_blank" moz-do-not-send="true">
NeurIPS</a>, <a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__icml.cc_imls_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DJJyjwIpPy9gtKrZzBMbW3sRMh3P3Kcw-SvtxG35EiP0%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=CRrxgRU83vcFOwZnfZgVhVyfXdW4vUbiDwuWIwk4YlA%3D&reserved=0"
target="_blank" moz-do-not-send="true">
ICML</a>, <a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.journals.elsevier.com_artificial-2Dintelligence%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DeWrRCVWlcbySaH3XgacPpi0iR0-NDQYCLJ1x5yyMr8U%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jOWhI2qz7lbf5aWrLuNVSFYyCHi01YJORlh68SVy0II%3D&reserved=0"
target="_blank" moz-do-not-send="true">
AIJ</a>/<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.journals.elsevier.com_artificial-2Dintelligence%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DeWrRCVWlcbySaH3XgacPpi0iR0-NDQYCLJ1x5yyMr8U%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jOWhI2qz7lbf5aWrLuNVSFYyCHi01YJORlh68SVy0II%3D&reserved=0"
target="_blank" moz-do-not-send="true">IJCAI</a>,
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__sigai.acm.org_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3D7rC6MJFaMqOms10EYDQwfnmX-zuVNhu9fz8cwUwiLGQ%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026944990348%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=QxnYLVcxShLNdKPdB57xr%2BcdrDVCQfvTNlnuXKwYqEY%3D&reserved=0"
target="_blank" moz-do-not-send="true">
ACM SIGAI</a>,
EurAI/AICOMM,
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__claire-2Dai.org_%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3D66ZofDIhuDba6Fb0LhlMGD3XbBhU7ez7dc3HD5-pXec%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026945146559%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=sYOQKbnEAQffgoZtGazCbFZ8OAoX71NvIBXpGnEu2u0%3D&reserved=0"
target="_blank" moz-do-not-send="true">
CLAIRE</a> and
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.robocup.org__%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3Dyl7-VPSvMrHWYKZFtKdFpThQ9UTb2jW14grhVOlAwV21R4FwPri0ROJ-uFdMqHy1%26s%3DbBI6GRq--MHLpIIahwoVN8iyXXc7JAeH3kegNKcFJc0%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026945146559%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WQh%2FcGXz1%2FHoK1xhfEHTapj53xAYnVRRb7CyY32PKsI%3D&reserved=0"
target="_blank" moz-do-not-send="true">
RoboCup</a>.</span><o:p></o:p></p>
</div>
<div>
<p
class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-family:"Arial",sans-serif">Twitter:
@aihuborg</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2">
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable"
style="border:none;border-top:solid #D3D4DE
1.0pt" cellpadding="0" border="1">
<tbody>
<tr>
<td
style="width:41.25pt;border:none;padding:9.75pt
.75pt .75pt .75pt" width="55">
<p class="MsoNormal"><a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.avast.com_sig-2Demail-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dlink-26utm-5Fcampaign-3Dsig-2Demail-26utm-5Fcontent-3Dwebmail%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DAo9QQWtO62go0hx1tb3NU6xw2FNBadjj8q64-hl5Sx4%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026945146559%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rX%2BCbKaTKFHYkmcbdVhkWDOMpvvGDPozGwRsjsYFAuU%3D&reserved=0"
target="_blank" moz-do-not-send="true"><span
style="text-decoration:none"><img
style="width:.4791in;height:.302in"
id="_x0000_i1026"
src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif"
moz-do-not-send="true" width="46"
height="29" border="0"></span></a><o:p></o:p></p>
</td>
<td
style="width:352.5pt;border:none;padding:9.0pt
.75pt .75pt .75pt" width="470">
<p class="MsoNormal"
style="line-height:13.5pt"><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#41424E">Virus-free.
<a
href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.avast.com_sig-2Demail-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dlink-26utm-5Fcampaign-3Dsig-2Demail-26utm-5Fcontent-3Dwebmail%26d%3DDwMFaQ%26c%3DslrrB7dE8n7gBJbeO0g-IQ%26r%3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ%26m%3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj%26s%3DAo9QQWtO62go0hx1tb3NU6xw2FNBadjj8q64-hl5Sx4%26e%3D&data=04%7C01%7Ctgd%40oregonstate.edu%7C3db6ca275cc748415eaa08d9e732a318%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637795026945146559%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rX%2BCbKaTKFHYkmcbdVhkWDOMpvvGDPozGwRsjsYFAuU%3D&reserved=0"
target="_blank"
moz-do-not-send="true">
<span style="color:#4453EA">www.avast.com</span></a>
<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<div class="moz-signature">-- <br>
<img src="cid:part36.E7AEC02E.59B4D37A@rubic.rutgers.edu"
border="0"></div>
</body>
</html>