<div dir="auto">Text is generated by the real world, and therefore one can infer structures of the real world simply by the structural relationships within the text itself. Even our human experience of the world is heavily mediated - no one has "direct" experience of "referents", it is all mediate by casual relationships which we assume preserve meaningful structure from the world which allows our experiences and thoughts to be useful. But that is still mediated and indirect as it has to be.</div><div dir="auto"><br></div><div dir="auto">The upshot is that it is obvious that LLMs may have a better ability to form conceptual models of time than of space (since they operate more directly on time series of symbols and their concepts of space have to be inferred from far less data) - we have experiments showing this is in fact the case (LLM temporal reasoning is much better than their spatial reasoning).</div><div dir="auto"><br></div><div dir="auto">But the fact these concepts are imperfect is not a good argument in favor of the idea they have no "understanding" whatsoever. For instance, an LLM can correctly "interpret" English language descriptions of code transformations in coding problems and refactor code, which involves conceptual mapping far beyond mere parroting of preexisting training data: the space of possible code transforms is exponentially larger than training sets. Even given the fact that LLMs are not perfect at this, if they had zero conceptual understanding one would predict they could only do coding problems that mimicked samples they had ingested during training, not apply generalized principles to generate code even somewhat correctly.</div><div dir="auto"><br></div><div dir="auto">Another bit of evidence for some form of understanding is you can combine LLMs with other modalities and tools and they are able to "use" these new modalities correctly. If the abstractions in LLMs had some relationship to what we might call "understanding", even if it is very limited, one would predict that they could be extended and used in this way, adding multimodal capabilities and so on. And that is what we observe.</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 13, 2024 at 21:56 <a href="mailto:gabriele.scheler@yahoo.com">gabriele.scheler@yahoo.com</a> <<a href="mailto:gabriele.scheler@yahoo.com">gabriele.scheler@yahoo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)"><div><div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16px"><div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16px"></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><font face="Helvetica Neue, Helvetica, Arial, sans-serif" style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;color:rgb(0,0,0)"><span style="font-size:16px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><br></span></font></div><div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><font face="Helvetica Neue, Helvetica, Arial, sans-serif" style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;color:rgb(0,0,0)"><span style="font-size:16px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">Simply put, LLMs represent words by their contexts. They have no referents for the words. That is not understanding. You can follow many tests people have performed to show that LLMs can reproduce, but make errors from a lack of knowing about the referential meaning of pieces of text. They mimic understanding, aka known as statistical parrot.</span></font></div></div></div></div><div><div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16px"><div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16px"><br></div>
<div id="m_-5024836922292093971ydpfa1ce488yahoo_quoted_8579042944" style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16px">
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;color:rgb(38,40,42)">
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">
On Tuesday, February 13, 2024 at 07:15:07 PM GMT+1, Weng, Juyang <<a href="mailto:weng@msu.edu" target="_blank" style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">weng@msu.edu</a>> wrote:
</div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><br></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><br></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><div id="m_-5024836922292093971ydpfa1ce488yiv5569946801" style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">
<div dir="ltr" style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">Dear Gary, </span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> You wrote, "LLMs do not really understand what they are saying".</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> Those LLMs generated text in a natural language, didn't they?</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> Why do you say that LLMs do not understand such text? </span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> The truly understandable answer to this question is not as simple as you
</span><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px;background-color:rgb(255,255,255);color:rgb(34,34,34)">believe</span><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">!
What you "believe" is not convincing and intuitive to many laymen and media! </span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> That is why Jeffery Hinton can simply give you potshots without valid analysis.</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> Best regards,</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">-John Weng</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">Brain-Mind Institute</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"> </span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">On Tue, Feb 13, 2024 at 12:49 AM Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" target="_blank" style="font-family:Arial,Helvetica,sans-serif">gary.marcus@nyu.edu</a>> wrote:</span></div>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-width:1px;border-left-style:solid;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;border-left-color:rgb(204,204,204)">
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">Geoff Hinton recently asked:</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<blockquote style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;background-color:rgb(255,255,255);color:rgb(34,34,34)">Which part of this is a misrepresentation: Gary Marcus
believes that LLMs do not really understand what they are saying and Gary Marcus also believes that they are an existential threat.</span></div>
</blockquote>
<div style="direction:ltr;text-align:left;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">That’s an easy one. The second, regarding existential threat. I</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">
<i style="font-family:Arial,Helvetica,sans-serif">do</i> believe that LLMs do not really understand what they are saying. But I do
<i style="font-family:Arial,Helvetica,sans-serif">not</i> believe that LLMs as such pose a (literally) existential threat, nor have I ever said such a thing, not in the Senate, not in my Substack, not here, and not anywhere else. (Anyone with evidence otherwise should step forward.)</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">I
<i style="font-family:Arial,Helvetica,sans-serif">have</i> in fact said the opposite; e.g., I have said that the human species is hard to extinguish, because we are genetically and geographic diverse, capable of building vaccines, etc. E.g. in interview with AFP, posted at TechExplore I said that I thought
the extinction threat 'overblown', </span><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(17,85,204)"><a href="https://urldefense.com/v3/__https://techxplore.com/news/2023-06-human-extinction-threat-overblown-ai.html__;!!HXCxUKc!1aqYa3ceLXbEbMlwq37LKL7XidWbHppPrJ38-1eZiwfUUxqbMdizbH3eXbTmdYfzgAxFOpN3rN-a808jNKY$" id="m_-5024836922292093971ydpfa1ce488yiv5569946801OWA991bc947-b367-eab1-2ead-1b1571af0d92" style="font-family:Arial,Helvetica,sans-serif;color:rgb(17,85,204)" rel="nofollow" target="_blank">https:// techxplore.com/news/2023-06- human-extinction-threat- overblown-ai.html</a></span><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">.</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">My actual view is captured here ,
</span><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(17,85,204)"><a href="https://urldefense.com/v3/__https://garymarcus.substack.com/p/ai-risk-agi-risk__;!!HXCxUKc!1aqYa3ceLXbEbMlwq37LKL7XidWbHppPrJ38-1eZiwfUUxqbMdizbH3eXbTmdYfzgAxFOpN3rN-aP5MUOq0$" id="m_-5024836922292093971ydpfa1ce488yiv5569946801OWA7fae189c-47c2-32e4-ed0a-92de0f401146" style="font-family:Arial,Helvetica,sans-serif;color:rgb(17,85,204)" rel="nofollow" target="_blank">https://garymarcus.substack. com/p/ai-risk-agi-risk</a></span><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">. </span></div>
<p style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><i style="font-family:Arial,Helvetica,sans-serif">although a lot of the literature equates artificial intelligence risk with the risk of superintelligence
or artificial general intelligence, you don’t have to be superintelligent to create serious problems. I am not worried, immediately, about “AGI risk” (the risk of superintelligent machines beyond our control), in the near term I am worried about what I will
call “MAI risk”—Mediocre AI that is unreliable (a la Bing and GPT-4) but widely deployed—both in terms of the sheer number of people using it, and in terms of the access that the software has to the world. ..</i></span></p>
<p style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><i style="font-family:Arial,Helvetica,sans-serif">Lots of ordinary humans, perhaps of above average intelligence but not necessarily genius-level,
have created all kinds of problems throughout history; in many ways, the critical variable is not intelligence but power, which often caches out as access. In principle, a single idiot with the nuclear codes could destroy the world, with only a modest amount
of intelligence and a surplus of ill-deserved access. …</i></span></p>
<p style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><i style="font-family:Arial,Helvetica,sans-serif">We need to stop worrying (just) about Skynet and robots taking over the world, and think a
lot more about what criminals, including terrorists, might do with LLMs, and what, if anything, we might do to stop them.</i></span></p>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">LLMs may well pose an existential threat to democracy, because (despite their limited capacity
for understanding and fact checking) they are excellent mimics, and their inability to fact-check is to the advantage of bad actors that which to exploit them. </span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">But that doesn’t mean they are remotely likely to literally end the species. </span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">I hope that clarifies my view.</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">Gary</span></div>
</blockquote>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Aptos,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</span></div>
<div style="text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)"><br>
</span></div>
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">--</span></div>
<div style="direction:ltr;text-align:left;text-indent:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif">
<span style="font-family:Arial,Helvetica,sans-serif;font-size:9.75pt;color:rgb(34,34,34)">Juyang (John) Weng</span></div>
</div>
</div></div>
</div>
</div></div></div></blockquote></div></div>