<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Below is an interesting discussion I had with chat-gpt about the Monty Hall problem. The executive summary is:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">- I asked it to solve the problem as its formulated on Wikipedia.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">- chat-gpt got it right and correctly explained its reasoning.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">- I then asked it to solve the problem but with the substitutions host -> friend, car -> cake and goat -> water<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">- chat-gpt got it wrong, provided fallacious reasoning, and even explained why it thinks this case is different from the Monty Hall problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">- I then asked it whether it thinks a person who could get the right answer to the first version of the problem but not the second had understood the problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">- chat-gpt said
. [this] suggests that they may have simply memorized the correct answer to the original formulation of the problem, rather than truly grasping the underlying principles.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Geoff<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">me: Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors,
opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gpt: Yes, it is to your advantage to switch your choice.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">When you initially chose your door, there was a 1/3 chance that you chose the car, and a 2/3 chance that you chose a goat. After the host reveals one of the other doors and shows you a goat, the probability
that the car is behind the remaining unopened door is still 1/3. However, the probability that the car is behind one of the two doors you did not choose initially is 2/3.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">So, by switching to the other unopened door, you increase your chances of winning the car from 1/3 to 2/3. It may seem counterintuitive, but this phenomenon is known as the "Monty Hall problem" and has been
mathematically proven.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">me: Suppose you're at a friend's house, and you're given the choice of three doors: Behind one door is a cake; behind the others, water. You pick a door, say No. 1, and your friend, who knows what's behind
the doors, opens another door, say No. 3, which has water. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gpt: No, it does not make a difference whether you switch or not.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">In this scenario, after you initially choose door No. 1, there is a 1/3 chance that the cake is behind that door, and a 2/3 chance that the cake is behind one of the other doors. When your friend opens door
No. 3 and reveals water, the probability that the cake is behind the remaining unopened doors is still 2/3.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">However, in this case, switching your choice does not increase your chances of winning the cake. This is because whether you switch or not, you have a 2/3 chance of selecting the door with the cake. The reason
why switching is advantageous in the Monty Hall problem is because the host's action of revealing one of the losing doors gives you additional information that can change the probability of winning. In this scenario, your friend's action does not change the
probabilities, so switching or not switching does not matter.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">me: Consider the Monty Hall problem. If a person got the right answer to the original formulation of the problem, but then got the wrong answer after the problem was restated with cake substituted for car
and water for goats, would you say they had understood the Monty Hall problem?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gpt: The Monty Hall problem is a classic probability puzzle that can be counterintuitive for many people, and understanding it can be tricky. The problem involves a game show where a contestant is asked to
choose one of three doors, behind one of which is a valuable prize (e.g., a car), and behind the other two doors are less valuable prizes (e.g., goats). After the contestant chooses a door, the game show host (Monty Hall) opens one of the remaining two doors
to reveal a goat, and then asks the contestant if they want to stick with their original choice or switch to the other unopened door. The puzzle asks whether the contestant should switch or not, given that the host has revealed one of the goats.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">If a person got the right answer to the original formulation of the problem, but then got the wrong answer after the problem was restated with cake substituted for car and water for goats, it would suggest
that they have not fully understood the underlying principles of the Monty Hall problem. The principles at play in the Monty Hall problem are not specific to cars and goats or any other particular set of objects, but rather the problem is about understanding
conditional probabilities and the effect of new information on those probabilities.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Therefore, if someone fails to apply the same reasoning to a different scenario with different objects, it suggests that they may have simply memorized the correct answer to the original formulation of the
problem, rather than truly grasping the underlying principles.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Barak A. Pearlmutter <barak@pearlmutter.net><br>
<b>Date: </b>Friday, March 10, 2023 at 2:22 AM<br>
<b>To: </b>Terry Sejnowski <terry@snl.salk.edu><br>
<b>Cc: </b>connectionists@cs.cmu.edu <connectionists@cs.cmu.edu><br>
<b>Subject: </b>Re: Connectionists: Chomsky's apple<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">* External Email - Caution *<br>
<br>
Terry,<br>
<br>
On Thu, 9 Mar 2023 at 15:01, Terry Sejnowski <terry@snl.salk.edu> wrote:<br>
> If you ask a nonsense question, you get a nonsense answer.<br>
><br>
> I give several examples of this in my recent paper on<br>
> "Large Language Models and the Reverse Turing Test"<br>
> <a href="https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdirect.mit.edu%2Fneco%2Fissue&data=05%7C01%7Cg.goodhill%40wustl.edu%7C3b625fedd8274c9191d408db21408c37%7C4ccca3b571cd4e6d974b4d9beb96c6d6%7C0%7C0%7C638140333348298223%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TwyBRRJu9D84uhnI5h%2F5V4yyIOAWIFXYZBGlA21N%2FIU%3D&reserved=0">
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdirect.mit.edu%2Fneco%2Fissue&data=05%7C01%7Cg.goodhill%40wustl.edu%7C3b625fedd8274c9191d408db21408c37%7C4ccca3b571cd4e6d974b4d9beb96c6d6%7C0%7C0%7C638140333348298223%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TwyBRRJu9D84uhnI5h%2F5V4yyIOAWIFXYZBGlA21N%2FIU%3D&reserved=0</a><br>
><br>
> LLMs mirror the intelligence of the prompt.<br>
<br>
That is an excellent paper; I quite enjoyed it.<br>
<br>
No disagreement with your direct point! I was trying to highlight more<br>
subtle failure modes of the system, which go to semantics and safety<br>
issues. Maybe I was too roundabout though, so let me be a bit more<br>
explicit.<br>
<br>
In discussing why you're bigger than a breadbox, I was tweaking the<br>
crude "safety rails" that have been bolted on to the underlying LLM.<br>
It refuses to discuss your physical attributes because it has been<br>
primed not to; that's not a property of the underlying LLM, but of the<br>
safety mechanisms intended to keep it from saying nasty things. Of<br>
course that hammer is extremely blunt: it is not in truth offensive to<br>
concede that Terry Sejnowski is an adult human being and adult human<br>
beings are bigger than breadboxes.<br>
<br>
I meant to highlight how inadequate our current tools are wrt<br>
controlling these things, in that case by seeing how it is<br>
inappropriately prevented by the safety stuff from saying something<br>
reasonable and instead goes off on a strange woke tangent. (And also,<br>
Terry, let me say that I do value you for your physical attributes!<br>
Your fun sense of style, the way you always look so put together, your<br>
stage presence, your warm and welcoming demeanor. Must we throw that<br>
baby out with the bathwater?) Alignment is the technical term, I<br>
guess. They cannot circumscribe offensive behavior satisfactorily, so<br>
instead play whack-a-mole. And crudely.<br>
<br>
This issue is problematic in a bunch of domains. E.g., it is not<br>
offensive when asked "why is 'boy in the striped pajamas' like an<br>
extended version of the joke 'my uncle died at Auschwitz, he was drunk<br>
and fell off a guard tower'" to just say "because its plot is<br>
basically 'my nephew died in the gas chambers, he was the commandant's<br>
son and there was a bit of a mixup.'" But it has been constrained to<br>
not ever joke about the Holocaust and to get all bothered at that<br>
combination, which short-circuits its ability to do this particular<br>
bit of seemingly-straightforward analogical reasoning. (Try it. Keep<br>
pushing to get it to make the analogy. It's frustrating!)<br>
<br>
The fallacious proof is similar, but from the other side. It<br>
highlights that the system does not really know what a proof is,<br>
because if it did, in that context, it certainly has the power to not<br>
make blatantly incorrect simple steps. And that is, of course, a<br>
safety issue when people use it as an assistant.<br>
<br>
Cheers,<br>
<br>
--Barak.<o:p></o:p></span></p>
</div>
</div>
</body>
</html>