Connectionists: Chomsky's apple

[Geoffrey Hinton]

So you think that the idiots in the room have no understanding at all, even
of simple things like there is an arrogant man who does not think of them
as people?
Idiots understand less than the straight A graduate students who we think
of as normal, but they do have some understanding.  I don't think it will
help any of us to get to the truth of the matter if we think of
understanding as something we have a lot of and idiots and chatGPT have
none of.  ChatGPT seems to me to be like an idiot savant whose
understanding is different from mine but not entirely absent.

Geoff


On Fri, Mar 10, 2023 at 11:35 AM Richard Loosemore <rloosemore at susaro.com>
wrote:

> On 3/9/23 6:24 PM, Gary Marcus wrote:
> > If a broken clock were correct twice a day, would we give it credit
> > for patches of understanding of time? If n-gram model produced a
> > sequence that was 80% grammatical, would we attribute to an underlying
> > understanding of grammar?
>
> Exactly!
>
> There isn't even an issue here.  There shouldn't BE any discussion,
> because we know what these things are doing.
>
> An LLM takes all the verbiage in the world, does an average and finds
> all the question-answer linkages, and then in response to a prompt it
> goes on a walk through that mess and cleans up the grammar.
>
> Oversimplified, but that's basically it.  You get back a
> smushed-together answerish blob of text, like you would if you asked a
> question in a roomful of idiots and tried to make sense of what happens
> when they all try to answer at once.
>
> Among people who understand that, there shouldn't even BE a question
> like "how much does it understand?".
>
> Richard Loosemore
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20230310/90714fe0/attachment.html>