<div dir="ltr">Dear Asim,<br><div>(A) You wrote: "<span style="font-family:Calibri,sans-serif;font-size:14.6667px">you mean a certain kind of mathematical formulation can give rise to consciousness?"</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">The maximum likelihood mathematical formulation is no</span><span style="font-family:Calibri,sans-serif;font-size:14.6667px">t a sufficient condition for conscious learning, but a necessary condition. </span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">This local minima issue is a CENTRAL issue for people on this list. <br>The local minima problems, </span><span style="font-family:Calibri,sans-serif;font-size:14.6667px">including the Turing Aware 2018, </span><span style="font-family:Calibri,sans-serif;font-size:14.6667px">have been giving neural networks a lot of </span>doubts<span style="font-family:Calibri,sans-serif;font-size:14.6667px"> .</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">For example, minimizing supervised motor errors in backprop deep learning have two consequences:<br>(1) It has violated the sensorimotor recurrence that is necessary for conscious learning (all big data violated it) and<br>(2) It requires Post-Selections which amounts to a rarely disclosed protocol flaw: Any such products have a big uncertainty: Each customer of CNN, LSTM and ELM systems has to cast a dice.</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">One of (1) and (2) above is sufficient for such neural networks to become impossible to learn consciousness.</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">Of course, as I posted yesterday, there are about 20 million-dollar problems that prevent such neural networks to </span><span style="font-family:Calibri,sans-serif;font-size:14.6667px">earn consciousness. All these </span><span style="font-family:Calibri,sans-serif;font-size:14.6667px">20 million-dollar problems must be solved in order to claim to learn consciousness. </span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">(B) You need to spend more time, as you are a mathematician (you can understand). <br>The ML optimality in DN and minimizing fitting errors in </span><span style="font-family:Calibri,sans-serif;font-size:14.6667px">CNN, LSTM and ELM are greatly different.</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">The former optimality has been mathematically proven (Weng IJIS 2015).</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">The latter optimality is never proven. The formulation is superficial (only about the fitting error, not studying distributions of system weights). I have proved below that error-backprop depends on casting a dice.<br></span><font color="#000000">J. Weng, "On Post Selections Using Test Sets
(PSUTS) in AI", in Proc. International Joint Conference on Neural
Networks, pp. 1-8, Shengzhen, China, July 18-22, 2021.
<a href="http://www.cse.msu.edu/%7eweng/research/PSUTS-IJCNN2021rvsd-cite.pdf">PDF file</a>.</font><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">Best regards,</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">-John<br><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 14, 2022 at 11:16 PM Asim Roy <<a href="mailto:ASIM.ROY@asu.edu">ASIM.ROY@asu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_8535131749916216955WordSection1">
<p class="MsoNormal">Dear John,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<ol style="margin-top:0in" start="1" type="1">
<li class="gmail-m_8535131749916216955MsoListParagraph" style="margin-left:0in">On your statement that “<span style="background:yellow">Maximum likelihood: DN formulation that
</span><span style="background:lime">gives rise to brain-like consciousness</span>” – you mean a certain kind of mathematical formulation can give rise to consciousness? I wish you were right. That solves our consciousness problem and I don’t
know why others are arguing about it in a different Connectionists email chain. <span style="background:red">
You should claim this on that chain and get some feedback</span>.<u></u><u></u></li></ol>
<p class="MsoNormal"><u></u> <u></u></p>
<ol style="margin-top:0in" start="2" type="1">
<li class="gmail-m_8535131749916216955MsoListParagraph" style="margin-left:0in">On your statement that “<span style="background:yellow">Do you mean the difference between maximum likelihood and a specially defined minimization of a cost function
is not a whole lot?</span>” – I have not studied this deeply, but did a quick search. For some distributions, they can be equivalent. Here are a few blogs. Again, I didn’t go through their mathematics.<u></u><u></u></li></ol>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal" style="text-indent:0.5in"><a href="https://abhimanyu08.github.io/blog/deep-learning/mathematics/2021/06/18/final.html" target="_blank">Why Squared Error Minimization = Maximum Likelihood Estimation | Abhimanyu (abhimanyu08.github.io)</a><u></u><u></u></p>
<p class="MsoNormal" style="text-indent:0.5in"><a href="https://towardsdatascience.com/linear-regression-91eeae7d6a2e" target="_blank">Linear Regression. A unification of Maximum Likelihood… | by William Fleshman | Towards Data Science</a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="background:yellow">But you consciousness claim is really an eye-opener. I didn’t know about it. You should claim it on the other Connectionists email chain</span>.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Best,<u></u><u></u></p>
<p class="MsoNormal">Asim<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>> <br>
<b>Sent:</b> Monday, February 14, 2022 7:26 AM<br>
<b>To:</b> Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>><br>
<b>Cc:</b> Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" target="_blank">gary.marcus@nyu.edu</a>>; John K Tsotsos <<a href="mailto:tsotsos@cse.yorku.ca" target="_blank">tsotsos@cse.yorku.ca</a>><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in conversation with Geoff Hinton<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Dear Asim,<u></u><u></u></p>
<div>
<p class="MsoNormal">Do you mean the difference between maximum likelihood and a specially defined minimization of a cost function is not a whole lot?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Maximum likelihood: DN formulation that gives rise to brain-like consciousness.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Deep Learning: minimize an error rate with supervised class labels.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">John<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Sun, Feb 13, 2022 at 6:17 PM Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal">John,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I don’t think this needs a response. There are some difference, but I don’t think a whole lot.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Asim<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>>
<br>
<b>Sent:</b> Sunday, February 13, 2022 1:22 PM<br>
<b>To:</b> Asim Roy <<a href="mailto:ASIM.ROY@asu.edu" target="_blank">ASIM.ROY@asu.edu</a>>; Gary Marcus <<a href="mailto:gary.marcus@nyu.edu" target="_blank">gary.marcus@nyu.edu</a>><br>
<b>Cc:</b> John K Tsotsos <<a href="mailto:tsotsos@cse.yorku.ca" target="_blank">tsotsos@cse.yorku.ca</a>><br>
<b>Subject:</b> Re: Connectionists: Stephen Hanson in conversation with Geoff Hinton<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">Dear Aim:<br>
You wrote "If I understand correctly, all learning systems do something along the lines of maximum likelihood learning or error minimization, like your DN. What?s your point?"<br>
<br>
"Maximum likelihood learning" (ML) is EXTREMELY different from "error minimization" (EM) like what Geoff Hinton's group did.<u></u><u></u></p>
<div>
<p class="MsoNormal">ML incrementally estimates the best solution from the distribution of a huge number of parameters such as weights, agees, connections patterns etc. conditioned on (I) Incremental
Learning, (II) a learning experience, and (III) a limited computations resource, such as the number of neurons.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">EM (like what Geoff Hinton's group did) only finds a luckiest network from multiple trained networks without condition (III) above. All such trained networks do not estimate the
distribution of a huge number of parameters. Thus, they are all local minima, actually very bad local minima because their error-backprop does not have competition as I explained in my YouTube talk:<br>
BMTalk 3D Episode 6: Did Turing Awards Go to Fraud?<br>
<a href="https://urldefense.com/v3/__https:/youtu.be/Rz6CFlKrx2k__;!!IKRxdwAv5BmarQ!I1p-4pKL2zAIFM9Pt4wDY6nQg045EGaYn4qK8faSR_4QYqnxCGifNuDBlUKidiY$" target="_blank">https://youtu.be/Rz6CFlKrx2k</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I am writing a paper in which I have proved that without condition (III) above, a special nearest neighbor classifier I designed can give any non-zero verification error rate and
any non-zero test error rate! <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Best regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-John<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal">--
<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Juyang (John) Weng<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Juyang (John) Weng<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Juyang (John) Weng<br></div></div>