Connectionists: 2024 Nobel Prize in Physics goes to Hopfield and Hinton

[Brian M.]

Leabra is a type of Boltzmann Machine. And a Boltzmann Machine can be
reduced to a single layer recurrent network, which, via intuitive
application of the superposition theorem (short-circuit), inherits
application of the Universal Approximation Theorem.

(If you didn't notice...just the other day... RNN's were re-christened as
champ).

But then we have it that MNIST can be solved by hill climbing the entire
corpus as the batch size with either uniform random noise added to the
weights, or by using normal noise as the inputs as using SGD.

So it's not clear what the fuss is all about vis backpropagation. It looks
pretty neat in *emergent*, though. Backpropagation+emergent is pretty much
fit-for-service for lighting technicians at parties.

For everything else? You're just moving variance around in your "end-to-end
pipeline". How am I not right? :)

To the victor the spoils.

Brian

On Tue, Oct 8, 2024 at 12:26 PM Stephen José Hanson <jose at rubic.rutgers.edu>
wrote:

> Yes, Jon good point here, and  although there is a through line from
> Hopfield to Hinton and Sejnowski.. Ie boltzmann machines and onto DL and
> LLMs
>
> Dave of course invented BP, Geoff would always say.. his contribution was
> to try and talk Dave out of it as it had so many computational problems and
> could be in no way considered biologically plausible.
>
> Steve
> On 10/8/24 8:47 AM, Jonathan D. Cohen wrote:
>
> I’d like to add, in this context, a note in memoriam of David Rumelhart, who was an integral contributor to the work honored by today’s Nobel Prize.
>
> jdc
>
>
> --
> Stephen José Hanson
> Professor, Psychology Department
> Director, RUBIC (Rutgers University Brain Imaging Center)
> Member, Executive Committee, RUCCS
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20241008/cd93c13c/attachment.html>