Connectionists: Stephen Hanson in conversation with Geoff Hinton

[Gary Marcus]

sure,   but a person can learn the idea for n-bits from a few examples with a small number of bits, generalizing it to large values of n. most current systems learn it for a certain number of bits and don’t generalize beyond that number of bits.

> On Jul 18, 2022, at 7:17 AM, Barak A. Pearlmutter <barak at pearlmutter.net> wrote:
> 
> 
> 
> On Mon, 18 Jul 2022 at 14:43, Danko Nikolic <danko.nikolic at gmail.com <mailto:danko.nikolic at gmail.com>> wrote:
> <image.png>
> 
> It is a hard problem to learn for a connectionist network.
> 
> We don't need to invent new terminology, like "inverters problem" or "generalized xor." This is parity. Four (4) bit parity.
> 
> https://en.wikipedia.org/wiki/Parity_function <https://urldefense.com/v3/__https://en.wikipedia.org/wiki/Parity_function__;!!BhJSzQqDqA!SymFAG_laXvyUDdhFw_r1ISi5nIQSZWSrOosRKtv9NMqcadSQYnFL016TNGGLHTyVZc8YnrrFxRtAyZTQLiE$>
> 
> Parity is *not* a hard function to learn. Even for a connectionist network.
> 
> It is an interesting function for historic reasons (n-bit parity cannot be loaded by a k-th order perceptron, for k<n, although there are loopholes if a random bits are available or if you are allowed to only almost load it) and because it's an interesting function for many mathematical constructions. See the above wikipedia page for some details. But it is not super difficult to learn.
> 
> --Barak Pearlmutter.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220718/d2c126ae/attachment.html>