<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"></div><div dir="ltr">Agreed. An in depth oral history about the rise and fall of IBM’s Watson—including interviews with David Ferrucci, Ken Jennings, and a pair of medical AI experts--is coming soon, in a new podcast that I am hosting, called Humans versus Machines. </div><div dir="ltr"><br></div><div dir="ltr"><div style="display: block;" class=""><div style="-webkit-user-select: all; -webkit-user-drag: element; display: inline-block;" class="apple-rich-link" draggable="true" role="link" data-url="https://podcasts.apple.com/us/podcast/humans-vs-machines-with-gary-marcus/id1532110146"><a style="border-radius:10px;font-family:-apple-system, Helvetica, Arial, sans-serif;display:block;-webkit-user-select:none;width:228px;user-select:none;-webkit-user-modify:read-only;user-modify:read-only;overflow:hidden;text-decoration:none;" class="lp-rich-link" rel="nofollow" href="https://podcasts.apple.com/us/podcast/humans-vs-machines-with-gary-marcus/id1532110146" dir="ltr" role="button" draggable="false" width="228"><table style="table-layout:fixed;border-collapse:collapse;width:228px;background-color:#E9E9EB;font-family:-apple-system, Helvetica, Arial, sans-serif;" class="lp-rich-link-emailBaseTable" cellpadding="0" cellspacing="0" border="0" width="228"><tbody><tr><td vertical-align="center" align="center"><img style="width:172px;filter:;height:172px;" width="172" height="172" draggable="false" class="lp-rich-link-mediaImage" alt="preview.png" src="cid:14AFD071-9181-4E96-8A46-4E1997805CF3"></td></tr><tr><td vertical-align="center"><table bgcolor="#E9E9EB" cellpadding="0" cellspacing="0" width="228" style="font-family:-apple-system, Helvetica, Arial, sans-serif;table-layout:fixed;background-color:rgba(233, 233, 235, 1);" class="lp-rich-link-captionBar"><tbody><tr><td style="padding:8px 0px 8px 0px;" class="lp-rich-link-captionBar-textStackItem"><div style="max-width:100%;margin:0px 16px 0px 16px;overflow:hidden;" class="lp-rich-link-captionBar-textStack"><div style="word-wrap:break-word;font-weight:500;font-size:12px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-topCaption-leading"><a rel="nofollow" href="https://podcasts.apple.com/us/podcast/humans-vs-machines-with-gary-marcus/id1532110146" style="text-decoration: none" draggable="false"><font color="#000000" style="color: rgba(0, 0, 0, 1);">Humans vs. Machines with Gary Marcus</font></a></div><div style="word-wrap:break-word;font-weight:400;font-size:11px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-bottomCaption-leading"><a rel="nofollow" href="https://podcasts.apple.com/us/podcast/humans-vs-machines-with-gary-marcus/id1532110146" style="text-decoration: none" draggable="false"><font color="#000000" style="color: rgba(0, 0, 0, 1);">Aventine Research Institute</font></a></div></div></td></tr></tbody></table></td></tr></tbody></table></a></div></div><br></div><div dir="ltr"><br><blockquote type="cite">On Mar 17, 2023, at 08:40, Asim Roy <ASIM.ROY@asu.edu> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr">
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style>@font-face { font-family: "Cambria Math"; }
@font-face { font-family: Calibri; }
@font-face { font-family: Georgia; }
p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; }
a:link, span.MsoHyperlink { color: rgb(5, 99, 193); text-decoration: underline; }
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText { margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; }
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph { margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif; }
span.EmailStyle17 { font-family: Calibri, sans-serif; color: windowtext; }
span.PlainTextChar { font-family: Calibri, sans-serif; }
.MsoChpDefault { font-family: Calibri, sans-serif; }
@page WordSection1 { size: 8.5in 11in; margin: 1in; }
div.WordSection1 { page: WordSection1; }
ol { margin-bottom: 0in; }
ul { margin-bottom: 0in; }</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoPlainText">IBM Watson Health's failure might provide some insight on "understanding." Here are two articles on Watson Health and some quotes from them.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoListParagraph" style="color:black;margin-left:0in;mso-list:l0 level1 lfo1">
<span style="color:windowtext"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.theregister.com_2022_01_31_machine-5Flearning-5Fthe-5Fhard-5Fway_-23-3A-7E-3Atext-3DIBM-2527s-2520Watson-2520Health-2520failed-2520at-2520the-2520time-252C-2520like-2Cexcelled-2520is-2520all-2520about-2520deducing-2520questions-2520from-2520data.&d=DwMFAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=4OrpEHe_Ckm9_d5iJ3YKKVtZzKuWsnCONlxXVzSiTqFqRSPhvf6i6UFYtdne-l7G&s=brTjcaDR_UgkheRSjuTF5GDvKbydLt5YOkeGWuEPACk&e=">Machine learning
the hard way: Watson's fatal misdiagnosis • The Register</a></span><span style="background:white"><o:p></o:p></span></li></ol>
<p class="MsoPlainText"><i><span style="background:yellow;mso-highlight:yellow">"You can learn the rules for Jeopardy in a minute. Becoming a doctor takes 10 years. Becoming the best doctor you can be takes a lifetime. Medical data, whether in the literature
or in test results, is meaningless or misleading without a lot of implied context. Watson Health couldn't work across lots of fields at once; that needs a general intelligence which AI currently does not have.<o:p></o:p></span></i></p>
<p class="MsoPlainText"><i><span style="background:yellow;mso-highlight:yellow"><o:p> </o:p></span></i></p>
<p class="MsoPlainText"><i><span style="background:yellow;mso-highlight:yellow">IBM's Watson Health failed at the time, like so much AI/ML,
<b><span style="color:#C00000">because it didn't know what the question was</span></b><span style="color:#C00000">
</span>– ironic, since the game of Jeopardy at which it excelled is all about deducing questions from data. It wanted to automate the highest skilled aspects of healthcare, diagnosis and treatment, but the problem wasn't one of getting the most data and the
best algorithm. <b><span style="color:#C00000">Rather, the problem was one of meaning</span></b>."</span></i><i><o:p></o:p></i></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<ol style="margin-top:0in" start="2" type="1">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__webcubatortechnologies.medium.com_what-2Dis-2Dthe-2Dreason-2Dof-2Dibm-2Dwatsons-2Dfailure-2Din-2Dhealthcare-2Dad2043968a28&d=DwMFAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=4OrpEHe_Ckm9_d5iJ3YKKVtZzKuWsnCONlxXVzSiTqFqRSPhvf6i6UFYtdne-l7G&s=a7Ii3bLu15zh_5B9kl0MS8IpmkaL0lEYLMHU88X34fc&e=">What Is The Reason Of IBM Watson’s Failure In Healthcare? |
by Webcubator Technologies | Medium</a><o:p></o:p></li></ol>
<p class="MsoNormal"><i><span style="color:#292929;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">“A leading AI researcher, Yoshua Bengio, said that AI systems
</span></i><b><i><span style="color:#C00000;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">could not understand uncertainty</span></i></b><i><span style="color:#C00000;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">
</span></i><i><span style="color:#292929;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">and pick up on underlying faint signs that doctors generally can observe.<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:.25in"><i><span style="color:#292929;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">It was an incredibly huge task to create an AI with the same understanding and insight as a human doctor.<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:.25in"><i><span style="color:#292929;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">It received intense criticism and a few even alleged that Watson of oncology providing
</span></i><b><i><span style="color:#C00000;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">irrelevant and terrible suggestions</span></i></b><i><span style="color:#292929;letter-spacing:-.05pt;background:yellow;mso-highlight:yellow">.”</span></i><i><span style="color:#292929;letter-spacing:-.05pt;background:white"><o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="color:#292929;letter-spacing:-.05pt;background:white"><o:p> </o:p></span></p>
<p class="MsoNormal">Asim Roy<o:p></o:p></p>
<p class="MsoNormal">Professor, Information Systems<o:p></o:p></p>
<p class="MsoNormal">Arizona State University<o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lifeboat.com_ex_bios.asim.roy&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=oDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw&e=" target="_blank">Lifeboat
Foundation Bios: Professor Asim Roy</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__isearch.asu.edu_profile_9973&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=jCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro&e=" target="_blank">Asim
Roy | iSearch (asu.edu)</a><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:15.0pt;font-family:"Georgia",serif;color:#292929;letter-spacing:-.05pt;background:white"><o:p> </o:p></span></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">-----Original Message-----<br>
From: Connectionists <a href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu">
connectionists-bounces@mailman.srv.cs.cmu.edu</a> On Behalf Of Kagan Tumer<br>
Sent: Wednesday, March 15, 2023 9:09 AM<br>
To: <a href="mailto:connectionists@mailman.srv.cs.cmu.edu">connectionists@mailman.srv.cs.cmu.edu</a><br>
Subject: Re: Connectionists: Chomsky's apple<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">I'm very reluctant to use the word "understand" beyond perhaps ChatGPT understanding the structure of language (statistically). Here's an example of a ChatGPT response to a simple arithmetic operation where:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">1- chatGPT was wrong;<o:p></o:p></p>
<p class="MsoPlainText">2- worked out an example that showed it was wrong, but it didn't register that and double down on its wrong conclusion;<o:p></o:p></p>
<p class="MsoPlainText">3- gave a high level explanation (also wrong) of why it was right even though it was wrong.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">you can forgive 1, but 2 and 3 clearly show ChatGPT does not actually understand what it is saying.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Kagan<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">On 3/14/23 9:54 AM, Natsuki Oka wrote:<o:p></o:p></p>
<p class="MsoPlainText">> [This email originated from outside of OSU. Use caution with links and
<o:p></o:p></p>
<p class="MsoPlainText">> attachments.]<o:p></o:p></p>
<p class="MsoPlainText">> <o:p></o:p></p>
<p class="MsoPlainText">> Judging from the responses below, ChatGPT understands counting and
<o:p></o:p></p>
<p class="MsoPlainText">> sorting to the extent that it can write a correct Python program but
<o:p></o:p></p>
<p class="MsoPlainText">> does not have the understanding to bring the proper execution results.<o:p></o:p></p>
<p class="MsoPlainText">> count_and_sort.png<o:p></o:p></p>
<p class="MsoPlainText">> Here's the program that ChatGPT created:<o:p></o:p></p>
<p class="MsoPlainText">> ---<o:p></o:p></p>
<p class="MsoPlainText">> sentences = [<o:p></o:p></p>
<p class="MsoPlainText">> "The quick brown fox jumps over the lazy dog",<o:p></o:p></p>
<p class="MsoPlainText">> "Python is a popular programming language",<o:p></o:p></p>
<p class="MsoPlainText">> "I like to eat pizza for dinner",<o:p></o:p></p>
<p class="MsoPlainText">> "The capital of France is Paris",<o:p></o:p></p>
<p class="MsoPlainText">> "The cat in the hat wears a red and white striped hat",<o:p></o:p></p>
<p class="MsoPlainText">> "My favorite color is blue",<o:p></o:p></p>
<p class="MsoPlainText">> "The United States has fifty states",<o:p></o:p></p>
<p class="MsoPlainText">> "There are seven days in a week",<o:p></o:p></p>
<p class="MsoPlainText">> "I enjoy listening to music while I work",<o:p></o:p></p>
<p class="MsoPlainText">> "Mount Everest is the highest mountain in the world"<o:p></o:p></p>
<p class="MsoPlainText">> ]<o:p></o:p></p>
<p class="MsoPlainText">> <o:p></o:p></p>
<p class="MsoPlainText">> # sort the sentences by number of words sentences.sort(key=lambda x:
<o:p></o:p></p>
<p class="MsoPlainText">> len(x.split()))<o:p></o:p></p>
<p class="MsoPlainText">> <o:p></o:p></p>
<p class="MsoPlainText">> # print the sorted sentences with the number of words in parentheses
<o:p></o:p></p>
<p class="MsoPlainText">> for sentence in sentences:<o:p></o:p></p>
<p class="MsoPlainText">> num_words = len(sentence.split())<o:p></o:p></p>
<p class="MsoPlainText">> print(f"{sentence} ({num_words})")<o:p></o:p></p>
<p class="MsoPlainText">> ---<o:p></o:p></p>
<p class="MsoPlainText">> <o:p></o:p></p>
<p class="MsoPlainText">> The execution of this program yields the following correct results:<o:p></o:p></p>
<p class="MsoPlainText">> ---<o:p></o:p></p>
<p class="MsoPlainText">> My favorite color is blue (5)<o:p></o:p></p>
<p class="MsoPlainText">> Python is a popular programming language (6) The capital of France is
<o:p></o:p></p>
<p class="MsoPlainText">> Paris (6) The United States has fifty states (6) I like to eat pizza
<o:p></o:p></p>
<p class="MsoPlainText">> for dinner (7) There are seven days in a week (7) I enjoy listening to
<o:p></o:p></p>
<p class="MsoPlainText">> music while I work (8) The quick brown fox jumps over the lazy dog (9)
<o:p></o:p></p>
<p class="MsoPlainText">> Mount Everest is the highest mountain in the world (9) The cat in the
<o:p></o:p></p>
<p class="MsoPlainText">> hat wears a red and white striped hat (12)<o:p></o:p></p>
<p class="MsoPlainText">> ---<o:p></o:p></p>
<p class="MsoPlainText">> <o:p></o:p></p>
<p class="MsoPlainText">> Oka Natsuki<o:p></o:p></p>
<p class="MsoPlainText">> Miyazaki Sangyo-keiei University<o:p></o:p></p>
<p class="MsoPlainText">> <o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">--<o:p></o:p></p>
<p class="MsoPlainText">Kagan Tumer<o:p></o:p></p>
<p class="MsoPlainText">Director, Collaborative Robotics and Intelligent Systems Institute Professor, School of MIME Oregon State University
<a href="https://urldefense.com/v3/__http:/engr.oregonstate.edu/*ktumer__;fg!!IKRxdwAv5BmarQ!aKzaiQllAsrcYabgwa4CW8lljg8MRiJlagYFw1B6WJfkVkN8gVVEAVKksSlL389oBE8tQJwcVKshbRv7JATYTqX9HJw$">
<span style="color:windowtext;text-decoration:none">https://urldefense.com/v3/__http://engr.oregonstate.edu/*ktumer__;fg!!IKRxdwAv5BmarQ!aKzaiQllAsrcYabgwa4CW8lljg8MRiJlagYFw1B6WJfkVkN8gVVEAVKksSlL389oBE8tQJwcVKshbRv7JATYTqX9HJw$</span></a><o:p></o:p></p>
<p class="MsoPlainText"><a href="https://urldefense.com/v3/__https:/kagantumer.com__;!!IKRxdwAv5BmarQ!aKzaiQllAsrcYabgwa4CW8lljg8MRiJlagYFw1B6WJfkVkN8gVVEAVKksSlL389oBE8tQJwcVKshbRv7JATYIbo90Qs$"><span style="color:windowtext;text-decoration:none">https://urldefense.com/v3/__https://kagantumer.com__;!!IKRxdwAv5BmarQ!aKzaiQllAsrcYabgwa4CW8lljg8MRiJlagYFw1B6WJfkVkN8gVVEAVKksSlL389oBE8tQJwcVKshbRv7JATYIbo90Qs$</span></a>
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div></blockquote></body></html>