Connectionists: Chomsky's apple

[Kevin McKee]

Re: the nature of understanding in these models: in *Critique of Pure
Reason, *Kant argued that statistical impressions are only half of the
story. Some basic, axiomatic ontology both enables and invokes the need for
understanding.
In other words, a model could only understand something if it took as input
not just the data, but the operators binding that data together, basic
recognition that the data *exist*, and basic recognition that the operators
binding the data also exist.
Then counterfactuals arise from processing both data and the axioms of its
ontology: what can't exist, doesn't exist, can exist, probably exists. The
absolute versions: what does exist or what cannot exist, can only be
undertaken by reference to the forms in which the data are presented (space
and time), so somehow, the brain observes not just input data but the
*necessary
facts of* input data.

This definition of understanding is different from, and independent of,
intelligence. A weak understanding is still an understanding, and it is
nothing at all if not applying structure to ontological propositions about
what can or cannot be.
Without ontology and whatever necessary forms that ontology takes (e.g.
space and time), the system is always divorced from the information it
processes in the sense of Searle's "chinese room". There is no modeling of
the information's nature *as* real or *as *counterfactual and so there is
neither a criterion nor a need for classifying anything as understood or
understandable.

Of course you can get ChatGPT to imitate all the *behaviors *of
understanding, and for me that has made it at least as useful a research
assistant as most humans. But I cannot see how it could possibly be
subjected, as I am, to the immutable impression that things exist, and
hence my need to organize information according to what exactly it is that
exists, and what exactly does not, cannot, will not, and so on.



On Tue, Mar 14, 2023 at 4:12 AM Miguel I. Solano <miguel at vmindai.com> wrote:

> Iam, Connectionists,
>
> Not an expert by any means but, as an aside, I understand
> Cremonini's 'refusal' seems to have been subtler than typically portrayed
> (see P. Gualdo to Galileo, July 29, 1611, *Opere*, II, 564).
>
> Best,
> --ms
>
> On Mon, Mar 13, 2023 at 5:49 PM Iam Palatnik <iam.palat at gmail.com> wrote:
>
>> Dear Brad, thank you for your insightful answers.
>> The compression analogy is really nice, although the 'Fermi-style'
>> problem of estimating whether all of the possible questions and answers one
>> could ask ChatGPT in all sorts of languages could be encoded within 175
>> billion parameters is definitely above my immediate intuition. It'd be
>> interesting to try to estimate which of these quantities is largest. Maybe
>> that could explain why ~175B seems to be the threshold that made models
>> start sounding so much more natural.
>>
>> In regards to generating nonsense, I'm imagining an uncooperative human
>> (say, a fussy child), that refuses to answer homework questions, or just
>> replies with nonsense on purpose despite understanding the question. Maybe
>> that child could be convinced to reply correctly with different prompting,
>> rewards or etc, which kinda mirrors what it takes to transform a raw LLM
>> like GPT-3 onto something like ChatGPT. It's possible we're still in the
>> early stages of learning how to make LLM 'cooperate' with us. Maybe we're
>> not asking them questions in a favorable way to extract their
>> understanding, or there's still work to be done regarding decoding
>> strategies. Even ChatGPT probably sounds way less impressive if we start
>> tinkering too much with hyperparameters like temperature/top-p/top-k. Does
>> that mean it 'understands' less when we change those parameters? I agree a
>> lot of the problem stems from the word 'understanding' and how we use it in
>> various contexts.
>>
>> A side note, that story about Galileo and the telescope is one of my
>> favorites. The person that refused to look through it was Cremonini
>> <https://en.wikipedia.org/wiki/Cesare_Cremonini_(philosopher)>.
>>
>>
>> Cheers,
>>
>> Iam
>>
>> On Mon, Mar 13, 2023 at 10:54 AM Miguel I. Solano <miguel at vmindai.com>
>> wrote:
>>
>>> Geoff, Gary, Connectionists,
>>>
>>> To me the risk is ChatGPT and the like may be 'overfitting'
>>> understanding, as it were. (Especially at nearly a hundred billion
>>> parameters.)
>>>
>>> --ms
>>>
>>> On Mon, Mar 13, 2023 at 6:56 AM Barak A. Pearlmutter <
>>> barak at pearlmutter.net> wrote:
>>>
>>>> Geoff,
>>>>
>>>> > He asked [ChatGPT] how many legs the rear left side of a cat has.
>>>> > It said 4.
>>>>
>>>> > I asked a learning disabled young adult the same question. He used
>>>> the index finger and thumb of both hands pointing downwards to represent
>>>> the legs on the two sides of the cat and said 4.
>>>> > He has problems understanding some sentences, but he gets by quite
>>>> well in the world and people are often surprised to learn that he has a
>>>> disability.
>>>>
>>>> That's an extremely good point. ChatGPT is way up the curve, well
>>>> above the verbal competence of many people who function perfectly well
>>>> in society. It's an amazing achievement, and it's not like progress is
>>>> stuck at its level. Exploring its weaknesses is not so much showing
>>>> failures but opportunities. Similarly, the fact that we can verbally
>>>> "bully" ChatGPT, saying things like "the square root of three is
>>>> rational, my wife said so and she is always right", and it will go
>>>> along with that, does not imply anything deep about whether it really
>>>> "knows" that sqrt(3) is irrational. People too exhibit all sorts of
>>>> counterfactual behaviours. My daughter can easily get me to play along
>>>> with her plan to become a supervillain. Students knowingly write
>>>> invalid proofs on homeworks and exams in order to try to get a better
>>>> grade. If anything, maybe we should be a bit scared that ChatGPT seems
>>>> so willing to humour us.
>>>>
>>>
>>>
>>> --
>>> Miguel I. Solano
>>> Co-founder & CEO, VMind Technologies, Inc.
>>>
>>> If you are not an intended recipient of this email, do not read, copy,
>>> use, forward or disclose the email or any of its attachments to others. Instead,
>>> please inform the sender and then delete it. Thank you.
>>>
>>
>
> --
> Miguel I. Solano
> Co-founder & CEO, VMind Technologies, Inc.
>
> If you are not an intended recipient of this email, do not read, copy,
> use, forward or disclose the email or any of its attachments to others. Instead,
> please inform the sender and then delete it. Thank you.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20230314/8af52e13/attachment.html>