Connectionists: 2024 Nobel Prize in Physics goes to Hopfield and Hinton

[Axel Hutt]

Dear all, 

essentially all research is based on the current knowledge and the work of 
previous generations. 

For instance, Albert Einstein knew the Lorentz equations and their meaning 
from the work of Hendrik Lorentz developed at the end of the 19th century. 
Henri Poincaré, Oliver Heaviside and many others have worked on them and 
their meaning assuming a world of aether. Then, 1905 Einstein derived them 
without assuming aether in his special relativity theory (surely motivated and 
heavily influenced by these previous studies) and won the Nobel Prize for it in 
1921. Even Einstein's own wife (also a physicist) contributed much in scientific 
discussions with him. 

Axel 

----- On 9 Oct, 2024, at 01:55, Brad Wyble <bwyble at gmail.com> wrote: 

> We really can trace the current AI boom back to John Carmack who wrote Doom,
> which ushered in the era of GPU-hungry computing. Credit where it's due please.

> On Tue, Oct 8, 2024 at 4:10 PM Stephen José Hanson < [
> mailto:jose at rubic.rutgers.edu | jose at rubic.rutgers.edu ] > wrote:

>> Hi Steve,

>> The problem every writer encounters is what can be concluded as resolved
>> knowledge rather then new/novel knowledge. In the law this is of course “legal
>> precedence”, so does the reference refer to a recent precedent, or does the one
>> for the 17 th century hold precedence? In the present case, I agree that
>> calculating gradients of functions using the chain rule was invented (Legendre
>> -- Least squares) far before Rumelhart and Hinton applied it to error gradients
>> in acyclic/cyclic networks, and of course there were others as you say, in the
>> 20 th century that also applied error gradient to networks (Parker, Le cun et
>> al). Schmidhuber says all that matters is the “math” not the applied context.
>> However, I seriously doubt that Legendre could have imagined using gradients of
>> function error through succesive application in a acylic network would have
>> produced a hierarchical kinship relationship (distinguishing between an italian
>> and english family mother, fathers, sons, aunts, grandparents etc.) in the
>> hidden units of a network, simply by observing individuals with fixed feature
>> relations. I think any reasonable person would maintain that this application
>> is completely novel and could not be predicted in or out of context from the
>> “math” and certainly not from the 18 th century. Hidden units were new in this
>> context and their representational nature was novel, in this context. Scope of
>> reference is also based on logical or causal proximity to the reference. In
>> this case, referencing Darwin or Newton in all biological or physics papers
>> should be based on the outcome of the metaphorical test of whether the recent
>> results tie back to original source in some direct line, for example, was
>> Oswald’s grandfather responsible for the death of President John F. Kennedy?
>> Failing this test, suggests that the older reference may not have scope. But of
>> course this can be subjective.

>> Steve

>> On 10/8/24 2:38 PM, Grossberg, Stephen wrote:

>>> Actually, Paul Werbos developed back propagation into its modern form, and
>>> worked out computational examples, for his 1974 Harvard PhD thesis.

>>> Then David Parker rediscovered it in 1982, etc.

>>> Schmidhuber provides an excellent and wide-ranging history of many contributors
>>> to Deep Learning and its antecedents:

>>> [
>>> https://www.sciencedirect.com/science/article/pii/S0893608014002135?casa_token=k47YCzFwcFEAAAAA:me_ZGF5brDqjRihq5kHyeQBzyUMYBypJ3neSinZ-cPn1pnyi69DGyM9eKSyLsdiRf759I77c7w
>>> |
>>> https://www.sciencedirect.com/science/article/pii/S0893608014002135?casa_token=k47YCzFwcFEAAAAA:me_ZGF5brDqjRihq5kHyeQBzyUMYBypJ3neSinZ-cPn1pnyi69DGyM9eKSyLsdiRf759I77c7w
>>> ]

>>> This article has been cited over 23,000 times.

>>> From: Connectionists [ mailto:connectionists-bounces at mailman.srv.cs.cmu.edu |
>>> <connectionists-bounces at mailman.srv.cs.cmu.edu> ] on behalf of Stephen José
>>> Hanson [ mailto:jose at rubic.rutgers.edu | <jose at rubic.rutgers.edu> ]
>>> Date: Tuesday, October 8, 2024 at 2:25 PM
>>> To: Jonathan D. Cohen [ mailto:jdc at princeton.edu |
>>> <jdc at princeton.edu> ] , Connectionists [ mailto:connectionists at cs.cmu.edu |
>>> <connectionists at cs.cmu.edu> ]
>>> Subject: Re: Connectionists: 2024 Nobel Prize in Physics goes to Hopfield and
>>> Hinton

>>> Yes, Jon good point here, and although there is a through line from Hopfield to
>>> Hinton and Sejnowski.. Ie boltzmann machines and onto DL and LLMs

>>> Dave of course invented BP, Geoff would always say.. his contribution was to try
>>> and talk Dave out of it as it had so many computational problems and could be
>>> in no way considered biologically plausible.

>>> Steve

>>> On 10/8/24 8:47 AM, Jonathan D. Cohen wrote:

>>>> I’d like to add, in this context, a note in memoriam of David Rumelhart, who was
>>>> an integral contributor to the work honored by today’s Nobel Prize.
>>>> jdc

>>> --
>>> Stephen José Hanson
>>> Professor, Psychology Department
>>> Director, RUBIC (Rutgers University Brain Imaging Center)
>>> Member, Executive Committee, RUCCS

>> --
>> Stephen José Hanson
>> Professor, Psychology Department
>> Director, RUBIC (Rutgers University Brain Imaging Center)
>> Member, Executive Committee, RUCCS

> --
> Brad Wyble (he/him)

-- 
Axel Hutt 
Directeur de Recherche 
Equipe MIMESIS - INRIA Nancy Grand Est 
Equipe MLMS - iCube Strasbourg 
Bâtiment NextMed 
2, rue Marie Hamm 
67000 Strasbourg, France 
https://mimesis.inria.fr/members/axel-hutt/ 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20241009/29396bcb/attachment.html>