Connectionists: Stephen Hanson in conversation with Geoff Hinton

[Juyang Weng]

Dear Asim,
The following information might be very useful to Gary Marcus and many
others on this list.
I try to make my complexity analysis a little more complete, as John
Tsotsos seems to have considered (2) below (?), if my memory serves me
correctly.
(1) The exponential complexity of recognizing and segmenting a part: each
pixel has c colors, a part with e pixel-elements has O(c^e) complexity.
(2) Group "parts" into an attended object:
"Suppose that each part is centered at location l, the number of
combinations of p parts of your object (dog) is O(p^l), another exponential
complexity."
This exponential O(p^l) has never been addressed by any neural networks
other than our DN.
(3) Segment an object from a cluttered background.   Suppose a cluttered
scene has m parts, m>> p.  Segmenting an object from a cluttered scene
(many parts!)
has a complexity O(2^m) where 2 is belonging or not-belonging to the object.
The real complexity is at least a product of above three
exponential complexities.  O(c^e p^l 2^m).
In other words, what you wrote "*we can also identify parts of wholes in
these scenes" is an illusion, since you have not discussed how your network
deals with*
*NP hard problems.  Just three examples are an illusion.  It is a toy
illusion. *
*Of course, our DN can do all above and more, with a constant (ML) frame
complexity, but the network size is of a brain-size. *

*I am not saying that we solved the NP completeness problem.  The NO
completeness problem is pure symbolic.  The problem of the brain is not
symbolic (e.g., pixels).We should not expect to do better than humans,
unlike Li Fei-Fei incorrectly claimed.  *
*Best regards,*
*-John*

On Fri, Feb 11, 2022, 9:49 PM Asim Roy <ASIM.ROY at asu.edu> wrote:

> Dear John,
>
>
>
> If I understand correctly, all learning systems do something along the
> lines of maximum likelihood learning or error minimization, like your DN.
> What’s your point?
>
>
>
> JOHN: *“Of course, the brain network does not remember all shapes and all
> configurations of parts.  That is why our DN must do maximum likelihood
> optimality, using a limited number of resources to best estimate such a
> huge space of cluttered scenes.”*
>
>
>
> So, can your DN model identify the parts of objects in the cluttered
> images below? Here was my note:
>
>
>
> ASIM: *“And we can also identify parts of wholes in these scenes. Here
> are some example scenes. In the first two scenes, we can identify the
> huskies along with the ears, eyes, legs, faces and so on. In the satellite
> image below, we can identify parts of the planes like the fuselage, tail,
> wing and so on. That’s the fundamental part of DARPA’s XAI model – to be
> able to **identify the parts to confirm the whole object. And if you can
> identify the parts, a school bus will never become an ostrich with change
> of a few pixels. So you get a lot of things with Explainable models of this
> form – a symbolic XAI model, robustness against adversarial attacks, and a
> model that you can trust. Explainable AI of this form can become the best
> defense against adversarial attacks. You may not need any adversarial
> training of any kind.”*
>
>
>
>
>
> Best,
>
> Asim Roy
>
> Professor, Information Systems
>
> Arizona State University
>
> Lifeboat Foundation Bios: Professor Asim Roy
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lifeboat.com_ex_bios.asim.roy&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=oDRJmXX22O8NcfqyLjyu4Ajmt8pcHWquTxYjeWahfuw&e=>
>
> Asim Roy | iSearch (asu.edu)
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__isearch.asu.edu_profile_9973&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=waSKY67JF57IZXg30ysFB_R7OG9zoQwFwxyps6FbTa1Zh5mttxRot_t4N7mn68Pj&s=jCesWT7oGgX76_y7PFh4cCIQ-Ife-esGblJyrBiDlro&e=>
>
>
>
> [image: A dog and a cat lying on a bed Description automatically generated
> with low confidence]  [image: A wolf walking in the snow Description
> automatically generated with medium confidence]    [image: An aerial view
> of a city Description automatically generated with medium confidence]
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220213/b95b5795/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 27597 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220213/b95b5795/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 91893 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220213/b95b5795/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 399723 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220213/b95b5795/attachment-0002.jpg>