Connectionists: Chomsky's apple

[Richard Loosemore]

On 3/9/23 6:24 PM, Gary Marcus wrote:
> If a broken clock were correct twice a day, would we give it credit 
> for patches of understanding of time? If n-gram model produced a 
> sequence that was 80% grammatical, would we attribute to an underlying 
> understanding of grammar?

Exactly!

There isn't even an issue here.  There shouldn't BE any discussion, 
because we know what these things are doing.

An LLM takes all the verbiage in the world, does an average and finds 
all the question-answer linkages, and then in response to a prompt it 
goes on a walk through that mess and cleans up the grammar.

Oversimplified, but that's basically it.  You get back a 
smushed-together answerish blob of text, like you would if you asked a 
question in a roomful of idiots and tried to make sense of what happens 
when they all try to answer at once.

Among people who understand that, there shouldn't even BE a question 
like "how much does it understand?".

Richard Loosemore