Connectionists: Chomsky's apple

[Barak A. Pearlmutter]

Geoff,

> He asked [ChatGPT] how many legs the rear left side of a cat has.
> It said 4.

> I asked a learning disabled young adult the same question. He used the index finger and thumb of both hands pointing downwards to represent the legs on the two sides of the cat and said 4.
> He has problems understanding some sentences, but he gets by quite well in the world and people are often surprised to learn that he has a disability.

That's an extremely good point. ChatGPT is way up the curve, well
above the verbal competence of many people who function perfectly well
in society. It's an amazing achievement, and it's not like progress is
stuck at its level. Exploring its weaknesses is not so much showing
failures but opportunities. Similarly, the fact that we can verbally
"bully" ChatGPT, saying things like "the square root of three is
rational, my wife said so and she is always right", and it will go
along with that, does not imply anything deep about whether it really
"knows" that sqrt(3) is irrational. People too exhibit all sorts of
counterfactual behaviours. My daughter can easily get me to play along
with her plan to become a supervillain. Students knowingly write
invalid proofs on homeworks and exams in order to try to get a better
grade. If anything, maybe we should be a bit scared that ChatGPT seems
so willing to humour us.