Connectionists: Statistics versus “Understanding” in Generative AI.

[Dave Touretzky]

Even GPT 3.5 can answer many questions about the spelling of words, so I
don't think the problem is that the tokenization has removed this
information.  I think the problem is that a stack of attention heads
isn't good at applying a novel, arbitrary rule consistently to a long
list of items.

Also, while the average human would be forgiven if they missed a few
entries in the list of US states that do not contain "a" in their name,
no normal human would claim that "Alaska" or "Texas" belongs in this
list, nor would they persist in such an error after it was pointed out
to them, the way ChatGPT 4 and Gemini do.

This isn't an argument against the possibility of AGI.  It's an argument
that "attention is all you need" might be a bit of an overstatement.

-- Dave