<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"></div><div dir="ltr">when we really full understand everything that individual neurons can do computationally, we will look back at the current era and shake our heads.</div><div dir="ltr"><br><blockquote type="cite">On Feb 7, 2022, at 16:39, Juyang Weng <juyang.weng@gmail.com> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div>Dear Gary,</div><div>Thank you for the link to Geoff's GLOM paper. I quickly browsed it just now. </div><div>Some fundamental comments, not all, to be concise.</div><div>(1) You are right, Geoff's GLOM seems to be a symbolic network, which assigns a certain role to each group of neurons. </div><div>(2) Geoff's part-whole problem is too narrow, a dead end to solving his part-whole problem. I quote:</div>"Perhaps we can learn how to encode the information at each location in such a way that simply averaging<br>the encodings from different locations is the only form of interaction we need." He got stuck with "locations".<br><div>Like Convolution that I used in Cresceptron 1992 and Geoff also used much longer, as soon a feature representation is centered as a location, the system does not abstract as Michaal Jordan complained at an IJCNN conference. Michael Jordan did not sa=y what he meant by "does not abstract well", but his point is valid. </div><div>(3) Two feed-forward networks in Geoff's GLOM, one bottom-up and the other top-down, are efficient pattern recognizers that do not abstract. The brain is not just a pattern recognizer.</div><div>(4) It is very unfortunate that many neural network researchers including Alpha's DeepMinds have not dug deep into what a cell can do and what a cell cannot. Geoff's GLOM is an example. </div><div>I have a paper about a brain model and I sent it to some people to pre-review. But like my Conscious Learning paper that was rejected by ICDL 2021 and AAAI 2022, this brain model would be rejected too. </div><div>Your humbly,</div><div>-John</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 7, 2022 at 10:57 AM Gary Marcus <<a href="mailto:gary.marcus@nyu.edu">gary.marcus@nyu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="ltr"></div><div dir="ltr">Dear John, </div><div dir="ltr"><br></div><div dir="ltr">I agree with you that cluttered scenes are critical, but Geoff’s GLOM paper [<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cs.toronto.edu_-7Ehinton_absps_glomfinal.pdf&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=voC1zjhvlxIAVFIkA_eoVBeB42MRwtRgPU5DRYQDkmJsvZCcgooYIELkOp2I7STt&s=2MAQINrgJYSS6Cq2p5n26kkHCIX-XNyWMUeoCL1nR0c&e=" target="_blank">https://www.cs.toronto.edu/~hinton/absps/glomfinal.pdf</a>] might actually have some relevance. It may well be that we need to do a better job with parts and whole before we can fully address clutter, and Geoff is certainly taking that question seriously.</div><div dir="ltr"><br></div><div dir="ltr">Geoff’s “Stable islands of identical vectors” do sound suspiciously like symbols to me (in a good way!), but regardless, they seem to me to be a plausible candidate as a foundation for coping with clutter. </div><div dir="ltr"><br></div><div dir="ltr">And not just cluttered scenes, but also <b>relations between multiple objects in a scene</b>, which is another example of the broader issue you raise, challenging for pure MLPs but critical for deeper AI.</div><div dir="ltr"><br></div><div dir="ltr">Gary</div><div dir="ltr"><br></div><div dir="ltr"><blockquote type="cite">On Feb 7, 2022, at 00:23, Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Dear Geoff Hinton,</span></div><div>I respect that you have been working on pattern recognition on isolated characters using neural networks.</div><div><br></div><div>However, I am deeply disappointed that after receiving the Turing Award 2018, you are still falling behind your own award work by talking about "<span style="color:rgb(0,0,0);font-family:arial,sans-serif">how you</span></div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">recognize that a handwritten 2 is a 2." You have fallen behind our group's </span><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Creceptron work in 1992, let alone our group's work on 3D-to-2D-to-3D Conscious Learning using DNs. Both deal with cluttered scenes. </span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif"><br></span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Specifically, you will never be able to get a correct </span><span style="color:rgb(0,0,0);font-family:arial,sans-serif">causal explanation by looking at </span><span style="color:rgb(0,0,0);font-family:arial,sans-serif">a single hand-written 2. Your problem is too small to explain a brain network. You must look at cluttered sciences, with many objects. </span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif"><br></span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Yours humbly,</span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">-John</span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> </span><span style="color:rgb(0,0,0);font-family:arial,sans-serif">----</span><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Message: 7</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Date: Fri, 4 Feb 2022 15:24:02 -0500</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">From: Geoffrey Hinton <</span><a href="mailto:geoffrey.hinton@gmail.com" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">geoffrey.hinton@gmail.com</a><span style="color:rgb(0,0,0);font-family:arial,sans-serif">></span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">To: "Dietterich, Thomas" <</span><a href="mailto:tgd@oregonstate.edu" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">tgd@oregonstate.edu</a><span style="color:rgb(0,0,0);font-family:arial,sans-serif">></span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Cc: AIhub <</span><a href="mailto:aihuborg@gmail.com" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">aihuborg@gmail.com</a><span style="color:rgb(0,0,0);font-family:arial,sans-serif">>,</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> "</span><a href="mailto:connectionists@mailman.srv.cs.cmu.edu" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">connectionists@mailman.srv.cs.cmu.edu</a><span style="color:rgb(0,0,0);font-family:arial,sans-serif">"</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> <</span><a href="mailto:connectionists@mailman.srv.cs.cmu.edu" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">connectionists@mailman.srv.cs.cmu.edu</a><span style="color:rgb(0,0,0);font-family:arial,sans-serif">></span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Subject: Re: Connectionists: Stephen Hanson in conversation with Geoff</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> Hinton</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Message-ID:</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> <CAK8NvqpAHbC=T2U3dZb=</span><span style="color:rgb(0,0,0);font-family:arial,sans-serif">QaSMVkY4=</span><a href="mailto:xCqaYmzu%2Bpq5rnoh%2Bp2mQ@mail.gmail.com" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">xCqaYmzu+pq5rnoh+p2mQ@mail.gmail.com</a><span style="color:rgb(0,0,0);font-family:arial,sans-serif">></span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">Content-Type: text/plain; charset="utf-8"</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">I agree that it's nice to have a causal explanations. But I am not</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">convinced there will ever be a simple causal explanation for how you</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif"><span style="color:rgb(0,0,0);font-family:arial,sans-serif">recognize that a handwritten 2 is a 2. </span><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">Juyang (John) Weng<br></div></div></div></div></div>
</div></blockquote></div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Juyang (John) Weng<br></div></div>
</div></blockquote></body></html>