<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"></div><div dir="ltr">Correct; also tool integration has actually been less successful than some people believe: </div><div><br></div><a href="https://open.substack.com/pub/garymarcus/p/getting-gpt-to-work-with-external?r=8tdk6&utm_campaign=post&utm_medium=web">https://open.substack.com/pub/garymarcus/p/getting-gpt-to-work-with-external</a><div><div dir="ltr"><br><blockquote type="cite">On Feb 19, 2024, at 5:49 AM, Thomas Trappenberg <tt@cs.dal.ca> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="auto">Good point, but Dave's point stands as the models he is referring to did not even comprehend that they made mistakes. <div dir="auto"><br></div><div dir="auto">Cheers, Thomas</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 19, 2024, 4:43 a.m. <<a href="mailto:wuxundong@gmail.com">wuxundong@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">That can be attributed to the models' underlying text encoding and processing mechanisms, specifically tokenization that removes the spelling information from those words. If you use GPT-4 instead, it can process it properly by resorting to external tools.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 19, 2024 at 3:45 PM Dave Touretzky <<a href="mailto:dst@cs.cmu.edu" target="_blank" rel="noreferrer">dst@cs.cmu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">My favorite way to show that LLMs don't know what they're talking about<br>
is this simple prompt:<br>
<br>
List all the US states whose names don't contain the letter "a".<br>
<br>
ChatGPT, Bing, and Gemini all make a mess of this, e.g., putting "Texas"<br>
or "Alaska" on the list and leaving out states like "Wyoming" and<br>
"Tennessee". And you can have a lengthy conversation with them about<br>
this, pointing out their errors one at a time, and they still can't<br>
manage to get it right. Gemini insisted that all 50 US states have an<br>
"a" in their name. It also claimed "New Jersey" has two a's.<br>
<br>
-- Dave Touretzky<br>
</blockquote></div>
</blockquote></div>
</div></blockquote></div></body></html>