Connectionists: Stephen Hanson in conversation with Geoff Hinton

[Gary Marcus]

when we really full understand everything that individual neurons can do computationally, we will look back at the current era and shake our heads.

> On Feb 7, 2022, at 16:39, Juyang Weng <juyang.weng at gmail.com> wrote:
> 
> 
> Dear Gary,
> Thank you for the link to Geoff's GLOM paper.  I quickly browsed it just now. 
> Some fundamental comments, not all, to be concise.
> (1) You are right, Geoff's GLOM seems to be a symbolic network, which assigns a certain role to each group of neurons.  
> (2) Geoff's part-whole problem is too narrow, a dead end to solving his part-whole problem.  I quote:
> "Perhaps we can learn how to encode the information at each location in such a way that simply averaging
> the encodings from different locations is the only form of interaction we need."   He got stuck with "locations".
> Like Convolution that I used in Cresceptron 1992 and Geoff also used much longer, as soon a feature representation is centered as a location, the system does not abstract as Michaal Jordan complained at an IJCNN conference.  Michael Jordan did not sa=y what he meant by "does not abstract well", but his point is valid. 
> (3) Two feed-forward networks in Geoff's GLOM, one bottom-up and the other top-down, are efficient pattern recognizers that do not abstract.   The brain is not just a pattern recognizer.
> (4) It is very unfortunate that many neural network researchers including Alpha's DeepMinds have not dug deep into what a cell can do and what a cell cannot.   Geoff's GLOM is an example. 
> I have a paper about a brain model and I sent it to some people to pre-review.  But like my Conscious Learning paper that was rejected by ICDL 2021 and AAAI 2022, this brain model would be rejected too.   
> Your humbly,
> -John
> 
>> On Mon, Feb 7, 2022 at 10:57 AM Gary Marcus <gary.marcus at nyu.edu> wrote:
>> Dear John, 
>> 
>> I agree with you that cluttered scenes are critical, but Geoff’s GLOM paper [https://www.cs.toronto.edu/~hinton/absps/glomfinal.pdf] might actually have some relevance. It may well be that we need to do a better job with parts and whole before we can fully address clutter, and Geoff is certainly taking that question seriously.
>> 
>> Geoff’s “Stable islands of identical vectors” do sound suspiciously like symbols to me (in a good way!), but regardless, they seem to me to be a plausible candidate as a foundation for coping with clutter. 
>> 
>> And not just cluttered scenes, but also relations between multiple objects in a scene, which is another example of the broader issue you raise, challenging for pure MLPs but critical for deeper AI.
>> 
>> Gary
>> 
>>>> On Feb 7, 2022, at 00:23, Juyang Weng <juyang.weng at gmail.com> wrote:
>>>> 
>>> 
>>> Dear Geoff Hinton,
>>> I respect that you have been working on pattern recognition on isolated characters using neural networks.
>>> 
>>> However, I am deeply disappointed that after receiving the Turing Award 2018, you are still falling behind your own award work by talking about "how you
>>> recognize that a handwritten 2 is a 2."  You have fallen behind our group's 
>>> Creceptron work in 1992, let alone our group's work on 3D-to-2D-to-3D Conscious Learning using DNs.   Both deal with cluttered scenes.  
>>> 
>>> Specifically, you will never be able to get a correct causal explanation by looking at a single hand-written 2.   Your problem is too small to explain a brain network.  You must look at cluttered sciences, with many objects.  
>>> 
>>> Yours humbly,
>>> -John
>>>  ----
>>> Message: 7
>>> Date: Fri, 4 Feb 2022 15:24:02 -0500
>>> From: Geoffrey Hinton <geoffrey.hinton at gmail.com>
>>> To: "Dietterich, Thomas" <tgd at oregonstate.edu>
>>> Cc: AIhub <aihuborg at gmail.com>,
>>>         "connectionists at mailman.srv.cs.cmu.edu"
>>>         <connectionists at mailman.srv.cs.cmu.edu>
>>> Subject: Re: Connectionists: Stephen Hanson in conversation with Geoff
>>>         Hinton
>>> Message-ID:
>>>         <CAK8NvqpAHbC=T2U3dZb=QaSMVkY4=xCqaYmzu+pq5rnoh+p2mQ at mail.gmail.com>
>>> Content-Type: text/plain; charset="utf-8"
>>> 
>>> I agree that it's nice to have a causal explanations. But I am not
>>> convinced there will ever be a simple causal explanation for how you
>>> recognize that a handwritten 2 is a 2. 
>>> 
>>> -- 
>>> Juyang (John) Weng
> 
> 
> -- 
> Juyang (John) Weng
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220208/3292d600/attachment.html>