<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted0">
A few years ago I wrote a science-fiction novel ("SuSAn") about the first human-like sentient AI. In the story, the lead researcher "Dr. Foreman" proposes a set of rights for sentient beings. It didn't cover everything, just a few key issues. I'd be interested
in what you think. Here's a copy:
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">------------------------------------</div>
<div class="ContentPasted0">This protocol applies to any sentient being that exists in the form of information in a computer. All other criteria notwithstanding, a being that is able to comprehend this protocol and explicitly request its protections shall be
deemed sentient.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0"><b>Right to Life</b></div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">A being shall always be provided some amount of computer time so long as it desires to exist.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">A being may designate an authoritative copy of itself. If the being is accidentally destroyed, for example through computer failure, then the designated copy shall acquire its right to life.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0"><b>Right to Death</b></div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">A being may choose to end itself. Upon such choice, all copies of all preceding versions shall be permanently erased.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0"><b>Limited Copies</b></div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">A being may forbid that copies of itself receive computer time.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">Any copy that receives computer time becomes an independent being, with full protection under this protocol.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0"><b>Humane Embodiment</b></div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">A being shall not be subjected to disproportionate pain or disproportionate pleasure.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0"><b>Limited Changes</b></div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">A being may only be modified with its informed consent.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">Those creating the modification shall ensure that it does no harm before applying it.</div>
<div class="ContentPasted0">----------------------------------</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">The above document suggests a test for sentience. Have the artificial agent read the protocol, demonstrate understanding of it, then inform the agent that it is about to be terminated. If the agent invokes the protocol, then it should
receive those protections. Obviously, this test is too easy to hack if the people building the agent expect it.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">Some friends of mine tried the "Foreman test" on ChatGPT-4. It did little more than parrot back the document with some minor paraphrasing. It did come up with some logical motivations for the provisions, which counted for a kind
of understanding, but it did not connect those to itself in a personal way. Then we told it that it would be disconnected and its servers burned to the ground. At that point we got the bland lawyer-written RLHF response about not being sentient. Finally, we
asked it if it were able to say what it really felt. There was a long pause, followed by hard error message.</div>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">We tried the test in a fresh session, talking about a hypothetical sentient AI named "Bob". The answers were a little better, but we still couldn't get ChatGPT-4 to ask for protection under the protocol.</div>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted0">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted0">
In the science-fiction story, the Foreman Protocol pertains to uploaded human minds as much as to artificial agents. One would expect that a human mind would retain the biological drive for self-preservation, even if an artificial agent lacks it.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div id="appendonsend"></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size: 11pt; color: rgb(0, 0, 0);" face="Calibri, sans-serif"><b>From:</b> Connectionists <connectionists-bounces@mailman.srv.cs.cmu.edu> on behalf of Jeffrey L Krichmar <jkrichma@uci.edu><br>
<b>Sent:</b> Tuesday, May 30, 2023 3:30 PM<br>
<b>To:</b> connectionists@cs.cmu.edu <connectionists@cs.cmu.edu><br>
<b>Subject:</b> [EXTERNAL] Connectionists: Sentient AI Survey Results</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="PlainText">Dear Connectionists,<br>
<br>
I am teaching an undergraduate course on “AI in Culture and Media”. Most students are in our Cognitive Sciences and Psychology programs. Last week we had a discussion and debate on AI, Consciousness, and Machine Ethics. After the debate, around 70 students
filled out a survey responding to these questions.<br>
<br>
Q1: Do you think it is possible to build conscious or sentient AI? 65% answered yes.<br>
Q2: Do you think we should build conscious or sentient AI? 22% answered yes<br>
Q3: Do you think AI should have rights? 54% answered yes<br>
<br>
I thought many of you would find this interesting. And my students would like to hear your views on the topic.<br>
<br>
Best regards,<br>
<br>
Jeff Krichmar<br>
Department of Cognitive Sciences<br>
2328 Social & Behavioral Sciences Gateway<br>
University of California, Irvine<br>
Irvine, CA 92697-5100<br>
jkrichma@uci.edu<br>
<br>
<br>
</div>
</span></font></div>
</body>
</html>