Connectionists: NEW PAPER ON "DEEP HISTORY OF DEEP LEARNING".

[Stephen José Hanson]

Please find a new paper on Deep Learning, its history and connections with Psychological Learning Theory from 100 years ago.

This is a sketch of how Deep Learning works and why.    This paper reviews relevant literature and provides testable hypothesis concerning the development of feature structures in large networks and how they develop and are curated.

Even now we don't understand how deep learning works or why it applies so generally across many kinds of domain.  It generality is more of a mystery when one considers the feature construction process.   This of course is even worse for LLMs which noone has a clue how they come to talk, or seems to have executive functions.  And yet it is being monetized and exploited on a daily basis (not new in the history of technology).    Yoshua Bengio,  last year at the introduction of an LLM summer school, said something quite prescient:  "I believe we are all sleep walking through this".

This paper is an attempt to look under the hood and make sense of the generality of deep learning and the AI it enabled.


L.L. Thurstone, The law of effect and the dynamics of deep learning “a deep history of deep learning”

SJ Hanson and C Hanson

ABSTRACT

Deep learning (DL), a variant of the neural network algorithms originally proposed in the early twentieth century, has resulted in a renaissance of artificial intelligence. Despite the growing dominance of DL networks, little is understood about the learning mechanism that makes these networks so effective across such a wide range of applications. Drawing on a century of psychological learning theory (e.g., LL Thurstone), an account is offered of the learning mechanism that may enable DL networks to perform so successfully across so many different tasks. Specifically, evidence is provided that learning in DL networks is fit best by a hyperbolic function. This function is independent of hyper/meta parameters–not a scaling function but a learning curve to a specific equilibrium, a function that also entails an autocatalytic mechanism through which complex structures, abstracted from sensory features, can in principle create and support cognitive function in both biological and artificial systems. Keywords: Learning theory, Deep Learning, hyperbolic, autocatalytic, AI, Thurstone

https://link.springer.com/article/10.1007/s10462-026-11560-3


--
[cid:part1.pF7L9rnF.P83i0J01 at rubic.rutgers.edu]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20260709/5c8292e1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SIGRUBIC.png
Type: image/png
Size: 92763 bytes
Desc: SIGRUBIC.png
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20260709/5c8292e1/attachment.png>