Connectionists: Chomsky's apple

[Feitong Yang]

I wish the authors of the opinion piece, led by Chomsky, can try to play
with ChatGPT a bit more before they assert their claim. At least, that
would help them use different, more convincing, examples to make their
point.

Nowadays, everyone can type a paragraph and check what ChatGPT can or
cannot do. It's so easy to verify whether Chomsky's argument against
ChatGPT's performance is sound or weak. It is not the 80s and 90s when MIT
professors throw out claims on possible performance shortcomings, and
stronger, competence weakness, on connectionism models and it takes a lot
of effort to test whether their claim is really the case.

Here is another example from the piece that sounds quite plausible, but
ChatGPT would tell Chomsky and co-authors to think more carefully and
choose a better example:

> For this reason, the predictions of machine learning systems will always
be superficial and dubious. Because these programs cannot explain the rules
of English syntax, for example, they may well predict, incorrectly, that
“John is too stubborn to talk to” means that John is so stubborn that he
will not talk to someone or other (rather than that he is too stubborn to
be reasoned with). Why would a machine learning program predict something
so odd? Because it might analogize the pattern it inferred from sentences
such as “John ate an apple” and “John ate,” in which the latter does mean
that John ate something or other. The program might well predict that
because “John is too stubborn to talk to Bill” is similar to “John ate an
apple,” “John is too stubborn to talk to” should be similar to “John ate.”
The correct explanations of language are complicated and cannot be learned
just by marinating in big data.

Asking this question to ChatGPT:

*ME: explain what does mean to say "John is too stubborn to talk to"*
ChatGPT:  When someone says "John is too stubborn to talk to", it means
that they believe John is unwilling to listen to others or to consider
other perspectives. This stubbornness can make it difficult or impossible
to have a productive conversation or to reach a mutual understanding with
John. The phrase "too stubborn" implies that John's level of stubbornness
is excessive or beyond what would be considered reasonable or productive in
a conversation. It suggests that attempts to communicate with John may be
met with resistance or closed-mindedness, and that it may be difficult to
get him to see things from a different point of view. Overall, the phrase
"John is too stubborn to talk to" suggests that communicating with John may
require extra effort or patience and that it may be a challenging or
frustrating experience.

[image: image.png]
looks like language, at least the assumed complicated example, is not so
complicated.


Criticisms of ChatGPT's performance feel to me like a half-full-half-empty
debate. One may treat a borderline performance as fundamental incompetence
whereas another treats it as tunable competence and growth potential.

Let's try two examples that Gary used in old days: an ABA rule learning
example, and a binary identity mapping example from *The Algebric Mind*.

*(1) Rule learning?*
*---*
Me: let me give you two groups of triplets: group 1: (green, yellow,
green), (blue, red, blue); group 2: (green blue blue), (yellow red red).
Which group does ABA belong to?
ChatGPT: The triplet ABA belongs to Group 1 because it follows the pattern
of "ABA" where the first and third colors are the same (either green or
blue) and the second color is different (either yellow or red).

Me: what about (hot cold cold)?
ChatGPT: The triplet (hot, cold, cold) belongs to Group 2 because it
follows the pattern of (green, blue, blue), (yellow, red, red) where the
first two items are consecutive items in a sequence and the third item is a
repeat of the second item.

In (hot, cold, cold), "cold" is repeated twice, similar to the second item
in each triplet of Group 2, and "hot" and "cold" are consecutive in the
sequence of "hot" followed by "cold". Therefore, (hot, cold, cold) belongs
to Group 2.

Me: what about (human, insect, human)?
ChatGPT:  The triplet (human, insect, human) does not belong to either
Group 1 or Group 2 based on the patterns given.

Group 1 has the pattern of "ABA" where the first and third items are the
same and the second item is different, and Group 2 has the pattern of two
consecutive items followed by a repeat of the second item.

In (human, insect, human), there is no repetition of the second item as
required by Group 2, and the first and third items are not the same as
required by Group 1. Therefore, (human, insect, human) does not belong to
either group.


Me: really? check it again?

ChatGPT: I apologize for the error in my previous response. Upon reviewing
the patterns of the two groups, I realize that (human, insect, human)
actually belongs to Group 1.

Group 1 has the pattern of "ABA" where the first and third items are the
same and the second item is different. In (human, insect, human), the first
and third items are "human", and the second item is "insect". Therefore,
(human, insect, human) follows the pattern of Group 1 and belongs to this
group.

Thank you for bringing this to my attention and please let me know if you
have any further questions.

---


*(2) Algebric mind?*

>  Imagine that you are trained on the input and output data given in table
3.1. If you are like other people whom I have asked, you would guess that
in the test item the output that corresponds to input item [1111] is
[1111]. But that is not the only inference that you could draw. For
example, in the training data, the rightmost column is always 0: there is
no direct evidence that the rightmost column could ever be a 1. So you
might decide that the output that corresponds to test item [1111] is
[1110]. That inference, too, would be perfectly consistent with the data,
yet few if any people would make it. (We see later that some networks do.)
One way of describing the inference that people tend to draw is to say that
they are generalizing a one-to-one function, such as identity or sameness,
universally. (The Algebraic Mind, page 37)


---

Me: Let is a list of input-output pair: input=1010, output=1010;  input=0100,
output=0100; input=1110, output=1110; input=0000, output=0000; Now, if
input is 1111, what is the output

ChatGPT: Based on the input-output pairs given, the function appears to be
an identity function, where the output is the same as the input. Therefore,
if the input is 1111, the output would also be 1111.
---

ChatGPT's answer is now "like other people whom" Gary has asked, and it
gives exactly the reasoning Gary gave. Of course, ChatGPT may be cheating,
because it may have read Gary's book already.
ChatGPT can still behave very silly, but its explanations of its "reasoning
process"* look like* it can do symbol manipulation, beyond just simple
statistics. Even when its answer is wrong, its explanation articulated a(n)
(algebraic) rule inside the input data I sent.

Best regards,
Feitong


Goodhill, Geoffrey <g.goodhill at wustl.edu> 于2023年3月10日周五 11:38写道：

> Below is an interesting discussion I had with chat-gpt about the Monty
> Hall problem. The executive summary is:
>
> - I asked it to solve the problem as it’s formulated on Wikipedia.
>
> - chat-gpt got it right and correctly explained its reasoning.
>
> - I then asked it to solve the problem but with the substitutions host ->
> friend, car -> cake and goat -> water
>
> - chat-gpt got it wrong, provided fallacious reasoning, and even explained
> why it thinks this case is different from the Monty Hall problem.
>
> - I then asked it whether it thinks a person who could get the right
> answer to the first version of the problem but not the second had
> understood the problem.
>
> - chat-gpt said “…. [this] suggests that they may have simply memorized
> the correct answer to the original formulation of the problem, rather than
> truly grasping the underlying principles.”
>
>
>
> Geoff
>
>
>
> me: Suppose you're on a game show, and you're given the choice of three
> doors: Behind one door is a car; behind the others, goats. You pick a door,
> say No. 1, and the host, who knows what's behind the doors, opens another
> door, say No. 3, which has a goat. He then says to you, "Do you want to
> pick door No. 2?" Is it to your advantage to switch your choice?
>
>
>
> gpt: Yes, it is to your advantage to switch your choice.
>
> When you initially chose your door, there was a 1/3 chance that you chose
> the car, and a 2/3 chance that you chose a goat. After the host reveals one
> of the other doors and shows you a goat, the probability that the car is
> behind the remaining unopened door is still 1/3. However, the probability
> that the car is behind one of the two doors you did not choose initially is
> 2/3.
>
> So, by switching to the other unopened door, you increase your chances of
> winning the car from 1/3 to 2/3. It may seem counterintuitive, but this
> phenomenon is known as the "Monty Hall problem" and has been mathematically
> proven.
>
>
>
> me: Suppose you're at a friend's house, and you're given the choice of
> three doors: Behind one door is a cake; behind the others, water. You pick
> a door, say No. 1, and your friend, who knows what's behind the doors,
> opens another door, say No. 3, which has water. He then says to you, "Do
> you want to pick door No. 2?" Is it to your advantage to switch your choice?
>
>
>
> gpt: No, it does not make a difference whether you switch or not.
>
> In this scenario, after you initially choose door No. 1, there is a 1/3
> chance that the cake is behind that door, and a 2/3 chance that the cake is
> behind one of the other doors. When your friend opens door No. 3 and
> reveals water, the probability that the cake is behind the remaining
> unopened doors is still 2/3.
>
> However, in this case, switching your choice does not increase your
> chances of winning the cake. This is because whether you switch or not, you
> have a 2/3 chance of selecting the door with the cake. The reason why
> switching is advantageous in the Monty Hall problem is because the host's
> action of revealing one of the losing doors gives you additional
> information that can change the probability of winning. In this scenario,
> your friend's action does not change the probabilities, so switching or not
> switching does not matter.
>
>
>
> me: Consider the Monty Hall problem. If a person got the right answer to
> the original formulation of the problem, but then got the wrong answer
> after the problem was restated with cake substituted for car and water for
> goats, would you say they had understood the Monty Hall problem?
>
>
>
> gpt: The Monty Hall problem is a classic probability puzzle that can be
> counterintuitive for many people, and understanding it can be tricky. The
> problem involves a game show where a contestant is asked to choose one of
> three doors, behind one of which is a valuable prize (e.g., a car), and
> behind the other two doors are less valuable prizes (e.g., goats). After
> the contestant chooses a door, the game show host (Monty Hall) opens one of
> the remaining two doors to reveal a goat, and then asks the contestant if
> they want to stick with their original choice or switch to the other
> unopened door. The puzzle asks whether the contestant should switch or not,
> given that the host has revealed one of the goats.
>
> If a person got the right answer to the original formulation of the
> problem, but then got the wrong answer after the problem was restated with
> cake substituted for car and water for goats, it would suggest that they
> have not fully understood the underlying principles of the Monty Hall
> problem. The principles at play in the Monty Hall problem are not specific
> to cars and goats or any other particular set of objects, but rather the
> problem is about understanding conditional probabilities and the effect of
> new information on those probabilities.
>
> Therefore, if someone fails to apply the same reasoning to a different
> scenario with different objects, it suggests that they may have simply
> memorized the correct answer to the original formulation of the problem,
> rather than truly grasping the underlying principles.
>
>
>
> *From: *Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu> on
> behalf of Barak A. Pearlmutter <barak at pearlmutter.net>
> *Date: *Friday, March 10, 2023 at 2:22 AM
> *To: *Terry Sejnowski <terry at snl.salk.edu>
> *Cc: *connectionists at cs.cmu.edu <connectionists at cs.cmu.edu>
> *Subject: *Re: Connectionists: Chomsky's apple
>
> * External Email - Caution *
>
> Terry,
>
> On Thu, 9 Mar 2023 at 15:01, Terry Sejnowski <terry at snl.salk.edu> wrote:
> > If you ask a nonsense question, you get a nonsense answer.
> >
> > I give several examples of this in my recent paper on
> > "Large Language Models and the Reverse Turing Test"
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdirect.mit.edu%2Fneco%2Fissue&data=05%7C01%7Cg.goodhill%40wustl.edu%7C3b625fedd8274c9191d408db21408c37%7C4ccca3b571cd4e6d974b4d9beb96c6d6%7C0%7C0%7C638140333348298223%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TwyBRRJu9D84uhnI5h%2F5V4yyIOAWIFXYZBGlA21N%2FIU%3D&reserved=0
> >
> > LLMs mirror the intelligence of the prompt.
>
> That is an excellent paper; I quite enjoyed it.
>
> No disagreement with your direct point! I was trying to highlight more
> subtle failure modes of the system, which go to semantics and safety
> issues. Maybe I was too roundabout though, so let me be a bit more
> explicit.
>
> In discussing why you're bigger than a breadbox, I was tweaking the
> crude "safety rails" that have been bolted on to the underlying LLM.
> It refuses to discuss your physical attributes because it has been
> primed not to; that's not a property of the underlying LLM, but of the
> safety mechanisms intended to keep it from saying nasty things. Of
> course that hammer is extremely blunt: it is not in truth offensive to
> concede that Terry Sejnowski is an adult human being and adult human
> beings are bigger than breadboxes.
>
> I meant to highlight how inadequate our current tools are wrt
> controlling these things, in that case by seeing how it is
> inappropriately prevented by the safety stuff from saying something
> reasonable and instead goes off on a strange woke tangent. (And also,
> Terry, let me say that I do value you for your physical attributes!
> Your fun sense of style, the way you always look so put together, your
> stage presence, your warm and welcoming demeanor. Must we throw that
> baby out with the bathwater?) Alignment is the technical term, I
> guess. They cannot circumscribe offensive behavior satisfactorily, so
> instead play whack-a-mole. And crudely.
>
> This issue is problematic in a bunch of domains. E.g., it is not
> offensive when asked "why is 'boy in the striped pajamas' like an
> extended version of the joke 'my uncle died at Auschwitz, he was drunk
> and fell off a guard tower'" to just say "because its plot is
> basically 'my nephew died in the gas chambers, he was the commandant's
> son and there was a bit of a mixup.'" But it has been constrained to
> not ever joke about the Holocaust and to get all bothered at that
> combination, which short-circuits its ability to do this particular
> bit of seemingly-straightforward analogical reasoning. (Try it. Keep
> pushing to get it to make the analogy. It's frustrating!)
>
> The fallacious proof is similar, but from the other side. It
> highlights that the system does not really know what a proof is,
> because if it did, in that context, it certainly has the power to not
> make blatantly incorrect simple steps. And that is, of course, a
> safety issue when people use it as an assistant.
>
> Cheers,
>
> --Barak.
>


-- 
Feitong Yang
Software Engineer, Google
355 Main St. Cambridge, MA, 02142
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20230312/eb4b8be8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 91361 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20230312/eb4b8be8/attachment-0001.png>