<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-AU" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Further to Geoff’s point:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">How much human knowledge is encoded in the know-how of language use (and encoded in LLMs as models of that know-how)?
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Each Indigenous language is a unique survival guide to the traditional land, ecosystem and culture where it evolved. In one language from Arnhem Land (NT Aus), an edible fish has the same root form
as the tree whose berries it feeds on. If you see the tree at the river side, you know where to fish. In another language of Cape York Peninsula (QLD Aus), animals and plants have a grammatical marker that indicates if they are edible. Poisonous snakes are
non-edible, non-poisonous ones are generally edible. If you can identify a snake by name, other meaningful information is built into the grammar. Knowledge of the ecosystem has been bootstrapped into each language over thousands of years, just as it has been
bootstrapped by evolution into the genome of organisms over millennia.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Colonising languages like English have lost such direct connections to the lands where they evolved, but still have meaning, logic and reason encoded in their words, sentence forms, and common usage.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">LLMs can be considered ‘performance’ models of the meaningful^ human language use they were trained on, encoding much more than ‘competence’ models of disembodied grammar. Why would someone think
that stats necessarily strips all meaning from such models?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Languages don’t “think” per se, but they are compressed encodings of the thoughts of millennia. LLMs are also models of their training data.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Janet<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">^more or less meaningful, depending on which part of the internet they were trained on.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Connectionists <<a href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu">connectionists-bounces@mailman.srv.cs.cmu.edu</a>>
<b>On Behalf Of </b>Geoffrey Hinton<br>
<b>Sent:</b> Tuesday, 21 March 2023 3:59 AM<br>
<b>To:</b> Paul Cisek <<a href="mailto:paul.cisek@umontreal.ca">paul.cisek@umontreal.ca</a>><br>
<b>Cc:</b> <a href="mailto:connectionists@mailman.srv.cs.cmu.edu">connectionists@mailman.srv.cs.cmu.edu</a><br>
<b>Subject:</b> Re: Connectionists: Can LLMs think?<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">LLM's do not do pattern matching in the sense that most people understand it. They use the data to create huge numbers of features and interactions between features such that these interactions can predict the next word.<o:p></o:p></p>
<div>
<p class="MsoNormal">The first neural net language model (so far as I know) made bets about the third term of a triple using word embedding vectors with 6 components. Retrospectively, the components of these vectors could be interpreted as sensible features
for capturing the structure of the domain (which was very conventional family relationships). For example, there was a three-valued feature for a person's generation and the interactions between features ensured that the triple Victoria has-father ? took
the generation of Victoria and produced an answer that was of a higher generation because it understood that the relationship has-father requires this. Of course, in complicated domains there will be huge numbers of regularities which will make conflicting
predictions for the next word but the consensus can still be fairly reliable. I believe that factoring the discrete symbolic information into a very large number of features and interactions IS intuitive understanding and that this is true for both brains
and LLMs even though they may use different learning algorithms for arriving at these factorizations. I am dismayed that so many people fall prey to the well-known human disposition to think that there is something special about people.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Geoff<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Mar 20, 2023 at 3:53 AM Paul Cisek <<a href="mailto:paul.cisek@umontreal.ca">paul.cisek@umontreal.ca</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">I must say that I’m somewhat dismayed when I read these kinds of discussions, here or elsewhere. Sure, it’s understandable that many people are fooled into thinking
that LLMs are intelligent, just like many people were fooled by Eliza and Eugene Goostman. Humans are predisposed into ascribing intention and purpose to events in the world, which helped them construct complex societies by (often correctly) interpreting the
actions of other people around them. But this same predisposition also led them to believe that the volcano was angry when it erupted because they did something to offend the gods. Given how susceptible humans are to this false ascription of agency, it is
not surprising that they get fooled when something acts in a complex way.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">But (most of) the people on this list know what’s under the hood! We know that LLMs are very good at pattern matching and completion, we know about the universal
approximation theorem, we know that there is a lot of structure in the pattern of human-written text, and we know that humans are predisposed to ascribe meaning and intention even where there are none. We should therefore not be surprised that LLMs can produce
text patterns that generalize well within-distribution but not so well out-of-distribution, and that when the former happens, people may be fooled into thinking they are speaking with a thinking being. Again, they were fooled by Eliza, and Eugene Goostman,
and the Heider-Simmel illusion (ascribing emotion to animated triangles and circles)… and the rumblings of volcanos. But we know how LLMs and volcanos do what they do, and can explain their behavior without any additional assumptions (of thinking, or sentience,
or whatever). So why add them?<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">In a sense, we are like a bunch of professional magicians, who know where all of the little strings and hidden compartments are, and who know how we just redirected
the audience’s attention to slip the card into our pocket… but then we are standing around backstage wondering: “Maybe there really is magic?”<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">I think it’s not that machines have passed the Turing Test, but rather that we failed it.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Paul Cisek<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-CA"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-CA"> </span><span lang="EN-US"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rothganger, Fredrick <</span><a href="mailto:frothga@sandia.gov" target="_blank"><span lang="EN-US">frothga@sandia.gov</span></a><span lang="EN-US">>
<br>
<b>Sent:</b> Thursday, March 16, 2023 11:39 AM<br>
<b>To:</b> </span><a href="mailto:connectionists@mailman.srv.cs.cmu.edu" target="_blank"><span lang="EN-US">connectionists@mailman.srv.cs.cmu.edu</span></a><span lang="EN-US"><br>
<b>Subject:</b> Connectionists: Can LLMs think?<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span lang="EN-US" style="color:black">Noting the examples that have come up on this list over the last week, it's interesting that it takes some of the most brilliant AI researchers in the world to devise questions that break LLMs. Chatbots have always been
able to fool some people some of the time, ever since ELIZA. But we now have systems that can fool a lot of people a lot of the time, and even the occasional expert who loses their perspective and comes to believe the system is sentient. LLMs have either already
passed the classic Turning test, or are about to in the next generation.</span><span lang="EN-US"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span lang="EN-US" style="color:black"> </span><span lang="EN-US"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span lang="EN-US" style="color:black">What does that mean exactly? Turing's expectation was that "the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted".
The ongoing discussion here is an indication that we are approaching that threshold. For the average person, we've probably already passed it.</span><span lang="EN-US"><o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>