<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">Dear Gary, </span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> You wrote, "LLMs do not really understand what they are saying".</span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> Those LLMs generated text in a natural language, didn't they?</span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> Why do you say that LLMs do not understand such text? </span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> The truly understandable answer to this question is not as simple as you
</span><span style="font-family: Arial, Helvetica, sans-serif; font-size: 13px; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">believe</span><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">!
What you "believe" is not convincing and intuitive to many laymen and media! </span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> That is why Jeffery Hinton can simply give you potshots without valid analysis.</span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> Best regards,</span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">-John Weng</span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">Brain-Mind Institute</span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"> </span></div>
<div class="elementToProof"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">On Tue, Feb 13, 2024 at 12:49 AM Gary Marcus <gary.marcus@nyu.edu> wrote:</span></div>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left: 1px solid rgb(204, 204, 204);">
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">Geoff Hinton recently asked:</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<blockquote>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">Which part of this is a misrepresentation: Gary Marcus
believes that LLMs do not really understand what they are saying and Gary Marcus also believes that they are an existential threat.</span></div>
</blockquote>
<div style="direction: ltr; text-align: left;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">That’s an easy one. The second, regarding existential threat. I</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">
<i>do</i> believe that LLMs do not really understand what they are saying. But I do
<i>not</i> believe that LLMs as such pose a (literally) existential threat, nor have I ever said such a thing, not in the Senate, not in my Substack, not here, and not anywhere else. (Anyone with evidence otherwise should step forward.)</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">I
<i>have</i> in fact said the opposite; e.g., I have said that the human species is hard to extinguish, because we are genetically and geographic diverse, capable of building vaccines, etc. E.g. in interview with AFP, posted at TechExplore I said that I thought
the extinction threat 'overblown', </span><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(17, 85, 204);"><a href="https://urldefense.com/v3/__https://techxplore.com/news/2023-06-human-extinction-threat-overblown-ai.html__;!!HXCxUKc!1aqYa3ceLXbEbMlwq37LKL7XidWbHppPrJ38-1eZiwfUUxqbMdizbH3eXbTmdYfzgAxFOpN3rN-a808jNKY$" target="_blank" id="OWA991bc947-b367-eab1-2ead-1b1571af0d92" class="OWAAutoLink" style="color: rgb(17, 85, 204);">https://<wbr style="scroll-behavior:auto !important">techxplore.com/news/2023-06-<wbr style="scroll-behavior:auto !important">human-extinction-threat-<wbr style="scroll-behavior:auto !important">overblown-ai.html</a></span><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">.</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">My actual view is captured here ,
</span><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(17, 85, 204);"><a href="https://urldefense.com/v3/__https://garymarcus.substack.com/p/ai-risk-agi-risk__;!!HXCxUKc!1aqYa3ceLXbEbMlwq37LKL7XidWbHppPrJ38-1eZiwfUUxqbMdizbH3eXbTmdYfzgAxFOpN3rN-aP5MUOq0$" target="_blank" id="OWA7fae189c-47c2-32e4-ed0a-92de0f401146" class="OWAAutoLink" style="color: rgb(17, 85, 204);">https://garymarcus.substack.<wbr style="scroll-behavior:auto !important">com/p/ai-risk-agi-risk</a></span><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">. </span></div>
<p style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><i>although a lot of the literature equates artificial intelligence risk with the risk of superintelligence
or artificial general intelligence, you don’t have to be superintelligent to create serious problems. I am not worried, immediately, about “AGI risk” (the risk of superintelligent machines beyond our control), in the near term I am worried about what I will
call “MAI risk”—Mediocre AI that is unreliable (a la Bing and GPT-4) but widely deployed—both in terms of the sheer number of people using it, and in terms of the access that the software has to the world. ..</i></span></p>
<p style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><i>Lots of ordinary humans, perhaps of above average intelligence but not necessarily genius-level,
have created all kinds of problems throughout history; in many ways, the critical variable is not intelligence but power, which often caches out as access. In principle, a single idiot with the nuclear codes could destroy the world, with only a modest amount
of intelligence and a surplus of ill-deserved access. …</i></span></p>
<p style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><i>We need to stop worrying (just) about Skynet and robots taking over the world, and think a
lot more about what criminals, including terrorists, might do with LLMs, and what, if anything, we might do to stop them.</i></span></p>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">LLMs may well pose an existential threat to democracy, because (despite their limited capacity
for understanding and fact checking) they are excellent mimics, and their inability to fact-check is to the advantage of bad actors that which to exploit them. </span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">But that doesn’t mean they are remotely likely to literally end the species. </span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">I hope that clarifies my view.</span></div>
<div style="direction: ltr; text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">Gary</span></div>
</blockquote>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="text-align: left; text-indent: 0px;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);"><br>
</span></div>
<div><span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">--</span></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);">Juyang (John) Weng</span></div>
</body>
</html>