Connectionists: Stephen Hanson in conversation with Geoff Hinton

[Juyang Weng]

Dear Gary,
Thank you for the link to Geoff's GLOM paper.  I quickly browsed it just
now.
Some fundamental comments, not all, to be concise.
(1) You are right, Geoff's GLOM seems to be a symbolic network, which
assigns a certain role to each group of neurons.
(2) Geoff's part-whole problem is too narrow, a dead end to solving his
part-whole problem.  I quote:
"Perhaps we can learn how to encode the information at each location in
such a way that simply averaging
the encodings from different locations is the only form of interaction we
need."   He got stuck with "locations".
Like Convolution that I used in Cresceptron 1992 and Geoff also used much
longer, as soon a feature representation is centered as a location, the
system does not abstract as Michaal Jordan complained at an IJCNN
conference.  Michael Jordan did not sa=y what he meant by "does not
abstract well", but his point is valid.
(3) Two feed-forward networks in Geoff's GLOM, one bottom-up and the other
top-down, are efficient pattern recognizers that do not abstract.   The
brain is not just a pattern recognizer.
(4) It is very unfortunate that many neural network researchers including
Alpha's DeepMinds have not dug deep into what a cell can do and what a
cell cannot.   Geoff's GLOM is an example.
I have a paper about a brain model and I sent it to some people to
pre-review.  But like my Conscious Learning paper that was rejected by ICDL
2021 and AAAI 2022, this brain model would be rejected too.
Your humbly,
-John

On Mon, Feb 7, 2022 at 10:57 AM Gary Marcus <gary.marcus at nyu.edu> wrote:

> Dear John,
>
> I agree with you that cluttered scenes are critical, but Geoff’s GLOM
> paper [https://www.cs.toronto.edu/~hinton/absps/glomfinal.pdf] might
> actually have some relevance. It may well be that we need to do a better
> job with parts and whole before we can fully address clutter, and Geoff is
> certainly taking that question seriously.
>
> Geoff’s “Stable islands of identical vectors” do sound suspiciously like
> symbols to me (in a good way!), but regardless, they seem to me to be a
> plausible candidate as a foundation for coping with clutter.
>
> And not just cluttered scenes, but also *relations between multiple
> objects in a scene*, which is another example of the broader issue you
> raise, challenging for pure MLPs but critical for deeper AI.
>
> Gary
>
> On Feb 7, 2022, at 00:23, Juyang Weng <juyang.weng at gmail.com> wrote:
>
> 
> Dear Geoff Hinton,
> I respect that you have been working on pattern recognition on isolated
> characters using neural networks.
>
> However, I am deeply disappointed that after receiving the Turing Award
> 2018, you are still falling behind your own award work by talking about "how
> you
> recognize that a handwritten 2 is a 2."  You have fallen behind our
> group's
> Creceptron work in 1992, let alone our group's work on 3D-to-2D-to-3D
> Conscious Learning using DNs.   Both deal with cluttered scenes.
>
> Specifically, you will never be able to get a correct causal explanation
> by looking at a single hand-written 2.   Your problem is too small to
> explain a brain network.  You must look at cluttered sciences, with many
> objects.
>
> Yours humbly,
> -John
>  ----
> Message: 7
> Date: Fri, 4 Feb 2022 15:24:02 -0500
> From: Geoffrey Hinton <geoffrey.hinton at gmail.com>
> To: "Dietterich, Thomas" <tgd at oregonstate.edu>
> Cc: AIhub <aihuborg at gmail.com>,
>         "connectionists at mailman.srv.cs.cmu.edu"
>         <connectionists at mailman.srv.cs.cmu.edu>
> Subject: Re: Connectionists: Stephen Hanson in conversation with Geoff
>         Hinton
> Message-ID:
>         <CAK8NvqpAHbC=T2U3dZb=QaSMVkY4=
> xCqaYmzu+pq5rnoh+p2mQ at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> I agree that it's nice to have a causal explanations. But I am not
> convinced there will ever be a simple causal explanation for how you
> recognize that a handwritten 2 is a 2.
>
> --
> Juyang (John) Weng
>
>

-- 
Juyang (John) Weng
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220207/8b5fd856/attachment.html>