Connectionists: Statistics versus “Understanding” in Generative AI.

[Weng, Juyang]

Dear Dave,
    We have heard many expressive examples and failure examples about ChatGPT.  They are intuitive but not systematic.
    If my writing in the last CDS Newsletter applies to ChatGPT, ChatGPT (along with all other LLMs) is the luckiest fitter for its training set F, among many similar fitters.   The luckiest fitter has not been systematically tested on a new test T yet.   Its human developers are keeping improving ChatGPT, by hand-tuning its ever-increasing number of hyperparameters.
    The outputs from ChatGPT are from humans assisted by computers, not really from computers.
    Best regards,
-John
________________________________
From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu> on behalf of Dave Touretzky <dst at cs.cmu.edu>
Sent: Monday, February 19, 2024 11:21 AM
To: Connectionists <connectionists at cs.cmu.edu>
Subject: Re: Connectionists: Statistics versus “Understanding” in Generative AI.

Even GPT 3.5 can answer many questions about the spelling of words, so I
don't think the problem is that the tokenization has removed this
information.  I think the problem is that a stack of attention heads
isn't good at applying a novel, arbitrary rule consistently to a long
list of items.

Also, while the average human would be forgiven if they missed a few
entries in the list of US states that do not contain "a" in their name,
no normal human would claim that "Alaska" or "Texas" belongs in this
list, nor would they persist in such an error after it was pointed out
to them, the way ChatGPT 4 and Gemini do.

This isn't an argument against the possibility of AGI.  It's an argument
that "attention is all you need" might be a bit of an overstatement.

-- Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20240219/5d4a4bcd/attachment.html>