Connectionists: An open letter to Geoffrey Hinton: A call for civilized, moderated debate

[Weng, Juyang]

Dear Gary,
    You wrote, "LLMs do not really understand what they are saying".
    Those LLMs generated text in a natural language, didn't they?
    Why do you say that LLMs do not understand such text?
    The truly understandable answer to this question is not as simple as you believe!  What you "believe" is not convincing and intuitive to many laymen and media!
    That is why Jeffery Hinton can simply give you potshots without valid analysis.
    Best regards,
-John Weng
Brain-Mind Institute

On Tue, Feb 13, 2024 at 12:49 AM Gary Marcus <gary.marcus at nyu.edu> wrote:
Geoff Hinton recently asked:

Which part of this is a misrepresentation: Gary Marcus believes that LLMs do not really understand what they are saying and Gary Marcus also believes that they are an existential threat.

That’s an easy one. The second, regarding existential threat. I

  do believe that LLMs do not really understand what they are saying. But I do not believe that LLMs as such pose a (literally) existential threat, nor have I ever said such a thing, not in the Senate, not in my Substack, not here, and not anywhere else. (Anyone with evidence otherwise should step forward.)

I have in fact said the opposite; e.g., I have said that the human species is hard to extinguish, because we are genetically and geographic diverse, capable of building vaccines, etc.  E.g. in interview with AFP, posted at TechExplore  I said that I thought the extinction threat 'overblown', https://techxplore.com/news/2023-06-human-extinction-threat-overblown-ai.html<https://urldefense.com/v3/__https://techxplore.com/news/2023-06-human-extinction-threat-overblown-ai.html__;!!HXCxUKc!1aqYa3ceLXbEbMlwq37LKL7XidWbHppPrJ38-1eZiwfUUxqbMdizbH3eXbTmdYfzgAxFOpN3rN-a808jNKY$>.

My actual view is captured here , https://garymarcus.substack.com/p/ai-risk-agi-risk<https://urldefense.com/v3/__https://garymarcus.substack.com/p/ai-risk-agi-risk__;!!HXCxUKc!1aqYa3ceLXbEbMlwq37LKL7XidWbHppPrJ38-1eZiwfUUxqbMdizbH3eXbTmdYfzgAxFOpN3rN-aP5MUOq0$>.

although a lot of the literature equates artificial intelligence risk with the risk of superintelligence or artificial general intelligence, you don’t have to be superintelligent to create serious problems. I am not worried, immediately, about “AGI risk” (the risk of superintelligent machines beyond our control), in the near term I am worried about what I will call “MAI risk”—Mediocre AI that is unreliable (a la Bing and GPT-4) but widely deployed—both in terms of the sheer number of people using it, and in terms of the access that the software has to the world. ..

Lots of ordinary humans, perhaps of above average intelligence but not necessarily genius-level, have created all kinds of problems throughout history; in many ways, the critical variable is not intelligence but power, which often caches out as access. In principle, a single idiot with the nuclear codes could destroy the world, with only a modest amount of intelligence and a surplus of ill-deserved access. …

We need to stop worrying (just) about Skynet and robots taking over the world, and think a lot more about what criminals, including terrorists, might do with LLMs, and what, if anything, we might do to stop them.

LLMs may well pose an existential threat to democracy, because (despite their limited capacity for understanding and fact checking) they are excellent mimics, and their inability to fact-check is to the advantage of bad actors that which to exploit them.

But that doesn’t mean they are remotely likely to literally end the species.

I hope that clarifies my view.
Gary


--
Juyang (John) Weng
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20240213/9b5fc665/attachment.html>