Connectionists: LeCun on Marcus (Gary Marcus)

[Stadelmann Thilo (stdm)]

Interesting article indeed, thank you for a good read.

Regarding symbol manipulation in neural networks (biological and artificial), the recent proposal of von der Malsburg et al. (see preprint: https://arxiv.org/abs/2205.00002) might give a revealing answer as to how this might be implemented (and: learnt) using NN architecture (specifically, an artificial one). The basic ideas is to assume self-organization of stimuli-specific net fragments as the inductive bias that guides such learning (making such net fragments, in terms of the previous discussion, "symbols", or a code, that is compositional and thus generalizes well). 

Quoting the abstract: 

Introduction: In contrast to current AI technology, natural intelligence - the kind of autonomous intelligence that is realized in the brains of animals and humans to attain in their natural environment goals defined by a repertoire of innate behavioral schemata - is far superior in terms of learning speed, generalization capabilities, autonomy and creativity.  How are these strengths, by what means are ideas and imagination produced in natural neural networks?
Methods: Reviewing the literature, we put forward the argument that both our natural environment and the brain are of low complexity, that is, require for their generation very little information and are consequently both highly structured. We further argue that the structures of brain and natural environment are closely related.
Results: We propose that the structural regularity of the brain takes the form of net fragments (self-organized network patterns) and that these serve as the powerful inductive bias that enables the brain to learn quickly, generalize from few examples and bridge the gap between abstractly defined general goals and concrete situations.
Conclusions: Our results have important bearings on open problems in artificial neural network research.

Best, Thilo

-----Ursprüngliche Nachricht-----

Message: 2
Date: Thu, 16 Jun 2022 12:39:01 -0700
From: Gary Marcus <gary.marcus at nyu.edu>
To: Connectionists List <connectionists at cs.cmu.edu>
Subject: Connectionists: LeCun on Marcus
Message-ID: <759D13FB-57A5-4A6D-9A10-052817D7F841 at nyu.edu>
Content-Type: text/plain; charset="utf-8"

I?ll probably write a bit of reply later, but this is an excellent new essay by Yann LeCun, quite relevant to many recent discussions here:

https://www.noemamag.com/what-ai-can-tell-us-about-intelligence
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220616/2236ab75/attachment-0001.html>