<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Dear John,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We don’t use a segmentation approach to finding parts. For high-speed image processing, an engineering problem of course, you cannot afford to have a human looking at segmented parts for verification. With symbolic outputs for the parts,
as in the DARPA figure, the verification for parts can be done at an extremely high speed by a separate computer program. We can also output “Don’t Know” if we can’t verify the parts and, in that case, a human can take a look to verify objects.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I kind of explained what we do in my response to Geoffrey Hinton’s comment “<i><span style="background:yellow;mso-highlight:yellow">But I am not convinced there
</span><span style="background:lime;mso-highlight:lime">will ever be </span><span style="background:yellow;mso-highlight:yellow">a simple causal explanation for how you recognize that a handwritten 2 is a 2</span>.”</i> I am adding that response one more time.
<u>We teach the system composition of objects from parts and also the connectivity between the parts</u>. It’s similar to how we teach humans about parts of objects.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Here are responses to some additional comments of yours:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="background:lime;mso-highlight:lime">John</span>: “<i><span style="background:yellow;mso-highlight:yellow">I am not saying that we solved the NP completeness problem</span>.</i>”
<i> </i>I have not made any claim about training these systems in polynomial time. When we have a large number of objects and parts to deal with, we can break up the problem to consist of several smaller sets of objects and parts. Simple divide and conquer
principle.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="background:lime;mso-highlight:lime">John</span>: “<i><span style="background:yellow;mso-highlight:yellow">The problem of the brain is not symbolic (e.g., pixels)</span>.</i>” I am not sure what you mean here. There’s plenty
of neurophysiological evidence that the brain uses multimodal abstractions (multisensory neurons).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Again, to reiterate, you get a lot with Explainable models of the type envisioned by DARPA – a symbolic XAI model, robustness against adversarial attacks, a model you can trust, and high-speed processing.
<u>XAI of this form can become the best defense against adversarial attacks. You may not need any adversarial training of any kind.<o:p></o:p></u></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hope to see you at the IEEE World Congress on Computational Intelligence in Padua, Italy, July 2022 (<a href="https://wcci2022.org/">WCCI2022 – WCCI2022 Padua, Italy 18-23 July</a>) where I have a tutorial on this subject.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Asim Roy<o:p></o:p></p>
<p class="MsoNormal">Professor, Information Systems<o:p></o:p></p>
<p class="MsoNormal">Arizona State University<o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__lifeboat.com_ex_bios.asim.roy*26d*3DDwMFaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj*26s*3DoDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw*26e*3D*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C2174db997d41416570df08d9e7977df4*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795460131639617*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DQ2onlybx5B4kjGT7utxrdgtNYcAF9Xdo*2FPKbLmc054w*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!JbWNkdLmRRWC-fJ56HQ2rP8WJjq2HHjkPvblQHY33hbprAWylUhDSJy9dOjXJJc*24*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C089bbef47dda414f4b1308d9e81186cb*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795984264413927*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DupBnYQI8bCc*2By2PTAWajTl*2FN5lfR2KIMuDKP23FdyHs*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKiolJSoqKioqKioqKioqKiUlKiolJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!PE_a193Wp530h-nER7kk18HR4eB_1q0kEbJMDDBWj1sZLsYflejZY_BM7KWIWRM*24&data=04*7C01*7Cnathaniel.bastian*40westpoint.edu*7Cdc262a74048a4989fd8e08d9e81236ab*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795987200498624*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=5qyi2UXXrDV1SB53alp3QlOUxDgUfv01HfvbcG2gGEM*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiUlKioqKioqKioqKioqJSUqKiolJSUlJSUlJSUlJSUlJSUlJQ!!IKRxdwAv5BmarQ!LkQuyj42ldEMFEwGffdQMsrhfMh1Z_t8yiF60b9pImqpgUXQNNwNbXzwB9oxZTQ$" target="_blank">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__isearch.asu.edu_profile_9973*26d*3DDwMFaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj*26s*3DjCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro*26e*3D*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C2174db997d41416570df08d9e7977df4*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795460131639617*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3Dt0ejzRVeYNk43LgxyJgFikJNAU*2BtVZVxc*2BYQ6sUreXo*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!IKRxdwAv5BmarQ!JbWNkdLmRRWC-fJ56HQ2rP8WJjq2HHjkPvblQHY33hbprAWylUhDSJy9G0xM_xk*24*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C089bbef47dda414f4b1308d9e81186cb*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795984264413927*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DGQjP*2BbllN8ow1od3fIQ0njaGm7MnH06FdYkhNHf7lFQ*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKiolJSoqKioqKioqKioqKiUlKioqJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!PE_a193Wp530h-nER7kk18HR4eB_1q0kEbJMDDBWj1sZLsYflejZY_BMyzOniUo*24&data=04*7C01*7Cnathaniel.bastian*40westpoint.edu*7Cdc262a74048a4989fd8e08d9e81236ab*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795987200498624*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=oNzZzfqt5nI*2BO1*2FiPrtmIRCeuzzE*2BwzEmngfUGR*2Bg8w*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiolJSoqKioqKioqKioqKiUlKiolJSUlJSUlJSUlJSUlJSUlJSUlJSU!!IKRxdwAv5BmarQ!LkQuyj42ldEMFEwGffdQMsrhfMh1Z_t8yiF60b9pImqpgUXQNNwNbXzwu3" target="_blank">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttp*3A*2F*2Fwww.teuvonet.com*2F*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C2174db997d41416570df08d9e7977df4*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795460131639617*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DWJ7*2FN8Y0*2BOkdBZpEh6ts2ApoRBcFzBML*2FVoiB9h*2BBz0*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!JbWNkdLmRRWC-fJ56HQ2rP8WJjq2HHjkPvblQHY33hbprAWylUhDSJy9Zxkcv7w*24*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C089bbef47dda414f4b1308d9e81186cb*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795984264413927*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DqeXjvz5Y3AhVYcGw8*2BRqyh1YpjxL*2BRIseU5u5yVzqBw*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSoqKiolJSoqKioqKioqKioqKiUlKioqKiolJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!PE_a193Wp530h-nER7kk18HR4eB_1q0kEbJMDDBWj1sZLsYflejZY_BMpJCsBgw*24&data=04*7C01*7Cnathaniel.bastian*40westpoint.edu*7Cdc262a74048a4989fd8e08d9e81236ab*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795987200498624*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=u5BFd9WTCBdNKgPGhIu6L3vb6fDtp*2Bodb3Jo9h3Llcg*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiUlKioqKioqKioqKioqJSUqKiolJSUlJSUlJSUlJSUlJSUlJSU!!IKRxdwAv5BmarQ!LkQuyj42ldEMFEwGffdQMsrhfMh1Z_t8yiF60b9pImqpgUXQNNwNbXzw3GwLgxw$">www.teuvonet.com</a>
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">==================================================================================================================================================<o:p></o:p></p>
<p class="MsoNormal"><span style="background:fuchsia;mso-highlight:fuchsia">I am responding to this part of Geoffrey Hinton’s note:</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><i><span style="background:yellow;mso-highlight:yellow">“I agree that it's nice to have a causal explanations. But I am not convinced there
</span><span style="background:lime;mso-highlight:lime">will ever be </span><span style="background:yellow;mso-highlight:yellow">a simple causal explanation for how you recognize that a handwritten 2 is a 2. We can introspect on how we do it and this may or
may not give some insight into how we check our answer, but the immediate sense that a handwritten 2 is a 2 is computed by a neural net that is not functionally equivalent to any simple and easily explainable procedure.</span>”<o:p></o:p></i></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The causal explanation is actually done quite simply, and we are doing it currently. I can talk about this now because Arizona State University (ASU) has filed a provisional patent application on the technology. The basic idea was laid
out by DARPA in their Explainable AI (XAI) program (<a href="https://www.darpa.mil/program/explainable-artificial-intelligence">Explainable Artificial Intelligence (darpa.mil)</a>) and illustrated in the figure below. The idea is to identify objects based
on its parts. So, the figure below says that it’s a cat because it has fur, whiskers, and claws plus an unlabeled visual feature. I am not sure if DARPA got anything close to this from its funding of various entities. What this means is that you need a parts
model. And we do that. In the case of MNIST handwritten digits that Geoff mentions, we “teach” this parts model what the top part of a digit “3” looks like, what the bottom part looks like and so on. And we also teach connectivity between parts and the composition
of objects from parts. And we do that for all digits. And we get a symbolic model sitting on top of a CNN model that provides the explanation that Geoff is referring to as the causal explanation. This “teaching” is similar to the way you would teach a kid
to recognize different digits. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">An advantage of this parts model, in addition to being in an explainable symbolic form, is robustness to adversarial attack. We recently tested on the MNIST data. Where a regular CNN model’s accuracy was reduced by a fast gradient method
to 27%, our XAI model maintained an accuracy of 90%, probably higher. In general, it would be hard to make a school bus look like an ostrich, with a few pixel changes, if you can identify the parts of a school bus and an ostrich.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">A parts model that DARPA wanted provides both a symbolic explanation and adversarial protection. The problem that Geoffrey is referring to is solved.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am doing a tutorial on this at IEEE World Congress on Computational Intelligence in Padua, Italy, July 2022 (<a href="https://wcci2022.org/">WCCI2022 – WCCI2022 Padua, Italy 18-23 July</a>). I am copying the organizers and want to thank
them for accepting the tutorial proposal. The only other presentation I have done on this is at a Military Operations Research Society (MORS) meeting last December.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So, back to the future. Hybrid models might indeed save deep learning models and let us deploy these models without concern. We might not even need adversarial training of any kind.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Asim Roy<o:p></o:p></p>
<p class="MsoNormal">Professor, Information Systems<o:p></o:p></p>
<p class="MsoNormal">Arizona State University<o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__lifeboat.com_ex_bios.asim.roy*26d*3DDwMFaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj*26s*3DoDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw*26e*3D*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C2174db997d41416570df08d9e7977df4*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795460131639617*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DQ2onlybx5B4kjGT7utxrdgtNYcAF9Xdo*2FPKbLmc054w*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!JbWNkdLmRRWC-fJ56HQ2rP8WJjq2HHjkPvblQHY33hbprAWylUhDSJy9dOjXJJc*24*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C089bbef47dda414f4b1308d9e81186cb*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795984264413927*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DupBnYQI8bCc*2By2PTAWajTl*2FN5lfR2KIMuDKP23FdyHs*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKiolJSoqKioqKioqKioqKiUlKiolJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!PE_a193Wp530h-nER7kk18HR4eB_1q0kEbJMDDBWj1sZLsYflejZY_BM7KWIWRM*24&data=04*7C01*7Cnathaniel.bastian*40westpoint.edu*7Cdc262a74048a4989fd8e08d9e81236ab*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795987200498624*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=5qyi2UXXrDV1SB53alp3QlOUxDgUfv01HfvbcG2gGEM*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiUlKioqKioqKioqKioqJSUqKiolJSUlJSUlJSUlJSUlJSUlJQ!!IKRxdwAv5BmarQ!LkQuyj42ldEMFEwGffdQMsrhfMh1Z_t8yiF60b9pImqpgUXQNNwNbXzwB9oxZTQ$" target="_blank">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.proofpoint.com*2Fv2*2Furl*3Fu*3Dhttps-3A__isearch.asu.edu_profile_9973*26d*3DDwMFaQ*26c*3DslrrB7dE8n7gBJbeO0g-IQ*26r*3DwQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ*26m*3DwaSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj*26s*3DjCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro*26e*3D*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C2174db997d41416570df08d9e7977df4*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795460131639617*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3Dt0ejzRVeYNk43LgxyJgFikJNAU*2BtVZVxc*2BYQ6sUreXo*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!IKRxdwAv5BmarQ!JbWNkdLmRRWC-fJ56HQ2rP8WJjq2HHjkPvblQHY33hbprAWylUhDSJy9G0xM_xk*24*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C089bbef47dda414f4b1308d9e81186cb*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795984264413927*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DGQjP*2BbllN8ow1od3fIQ0njaGm7MnH06FdYkhNHf7lFQ*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKiolJSoqKioqKioqKioqKiUlKioqJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!PE_a193Wp530h-nER7kk18HR4eB_1q0kEbJMDDBWj1sZLsYflejZY_BMyzOniUo*24&data=04*7C01*7Cnathaniel.bastian*40westpoint.edu*7Cdc262a74048a4989fd8e08d9e81236ab*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795987200498624*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=oNzZzfqt5nI*2BO1*2FiPrtmIRCeuzzE*2BwzEmngfUGR*2Bg8w*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiolJSoqKioqKioqKioqKiUlKiolJSUlJSUlJSUlJSUlJSUlJSUlJSU!!IKRxdwAv5BmarQ!LkQuyj42ldEMFEwGffdQMsrhfMh1Z_t8yiF60b9pImqpgUXQNNwNbXzwu3" target="_blank">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/nam02.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fnam02.safelinks.protection.outlook.com*2F*3Furl*3Dhttp*3A*2F*2Fwww.teuvonet.com*2F*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C2174db997d41416570df08d9e7977df4*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795460131639617*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DWJ7*2FN8Y0*2BOkdBZpEh6ts2ApoRBcFzBML*2FVoiB9h*2BBz0*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!JbWNkdLmRRWC-fJ56HQ2rP8WJjq2HHjkPvblQHY33hbprAWylUhDSJy9Zxkcv7w*24*26data*3D04*7C01*7Cnathaniel.bastian*40westpoint.edu*7C089bbef47dda414f4b1308d9e81186cb*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795984264413927*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000*26sdata*3DqeXjvz5Y3AhVYcGw8*2BRqyh1YpjxL*2BRIseU5u5yVzqBw*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSoqKiolJSoqKioqKioqKioqKiUlKioqKiolJSUlJSUlJSUlJSUlJSUlJSUl!!IKRxdwAv5BmarQ!PE_a193Wp530h-nER7kk18HR4eB_1q0kEbJMDDBWj1sZLsYflejZY_BMpJCsBgw*24&data=04*7C01*7Cnathaniel.bastian*40westpoint.edu*7Cdc262a74048a4989fd8e08d9e81236ab*7C99ff8811351740a9bf1045ea0a321f0b*7C0*7C1*7C637795987200498624*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=u5BFd9WTCBdNKgPGhIu6L3vb6fDtp*2Bodb3Jo9h3Llcg*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiUlKioqKioqKioqKioqJSUqKiolJSUlJSUlJSUlJSUlJSUlJSU!!IKRxdwAv5BmarQ!LkQuyj42ldEMFEwGffdQMsrhfMh1Z_t8yiF60b9pImqpgUXQNNwNbXzw3GwLgxw$">www.teuvonet.com</a>
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Juyang Weng <juyang.weng@gmail.com> <br>
<b>Sent:</b> Sunday, February 13, 2022 12:59 PM<br>
<b>To:</b> Asim Roy <ASIM.ROY@asu.edu><br>
<b>Cc:</b> John K Tsotsos <tsotsos@cse.yorku.ca>; connectionists@mailman.srv.cs.cmu.edu; Gary Marcus <gary.marcus@nyu.edu><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in conversation with Geoff Hinton<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Dear Asim,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The following information might be very useful to Gary Marcus and many others on this list. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I try to make my complexity analysis a little more complete, as John Tsotsos seems to have considered (2) below (?), if my memory serves me correctly.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(1) The exponential complexity of recognizing and segmenting a part: each pixel has c colors, a part with e pixel-elements has O(c^e) complexity.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">(2) Group "parts" into an attended object:<br>
"Suppose that each part is centered at location l, the number of combinations of p parts of your object (dog) is O(p^l), another exponential complexity."<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">This exponential O(p^l) has never been addressed by any neural networks other than our DN. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(3) Segment an object from a cluttered background. Suppose a cluttered scene has m parts, m>> p. Segmenting an object from a cluttered scene (many parts!)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">has a complexity O(2^m) where 2 is belonging or not-belonging to the object.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The real complexity is at least a product of above three exponential complexities. O(c^e p^l 2^m).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">In other words, what you wrote "<i>we can also identify parts of wholes in these scenes" is an illusion, since you have not discussed how your network deals with</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i>NP hard problems. Just three examples are an illusion. It is a toy illusion. </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i>Of course, our DN can do all above and more, with a constant (ML) frame complexity, but the network size is of a brain-size. </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i>I am not saying that we solved the NP completeness problem. The NO completeness problem is pure symbolic. The problem of the brain is not symbolic (e.g., pixels).<br>
We should not expect to do better than humans, unlike Li Fei-Fei incorrectly claimed. </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i>Best regards,</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i>-John</i><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Feb 11, 2022, 9:49 PM Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear John,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If I understand correctly, all learning systems do something along the lines of maximum likelihood learning or error minimization, like your DN. What’s your point?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black;background:lime">JOHN:
</span><i><span style="color:black;background:yellow">“Of course, the brain network does not remember all shapes and all configurations of parts. That is why our DN must do maximum likelihood optimality, using a limited number of resources to best estimate
such a huge space of cluttered scenes.”</span></i><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">So, can your DN model identify the parts of objects in the cluttered images below? Here was my note:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black;background:lime">ASIM:</span><span style="color:black;background:yellow">
<i>“And we can also identify parts of wholes in these scenes. Here are some example scenes. In the first two scenes, we can identify the huskies along with the ears, eyes, legs, faces and so on. In the satellite image below, we can identify parts of the planes
like the fuselage, tail, wing and so on. That’s the fundamental part of DARPA’s XAI model – to be able to
</i></span><i><span style="color:black;background:red">identify the parts to confirm the whole object</span><span style="color:black;background:yellow">. And if you can identify the parts, a school bus will never become an ostrich with change of a few pixels.
So you get a lot of things with Explainable models of this form – </span><span style="color:black;background:red">a symbolic XAI model, robustness against adversarial attacks, and a model that you can trust</span><span style="color:black;background:yellow">.
Explainable AI of this form can become the best defense against adversarial attacks. You may not need any adversarial training of any kind.</span>”</i><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Best,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Asim Roy<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Professor, Information Systems<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Arizona State University<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lifeboat.com_ex_bios.asim.roy&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=oDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw&e=" target="_blank">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__isearch.asu.edu_profile_9973&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=jCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro&e=" target="_blank">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</body>
</html>