Connectionists: Standing with Geoff Hinton, against Elon Musk

[Tsvi Achler]

Academic oligarchs receive much of the federal funding and can control
narratives in many ways [1,2] including through funding, peer-review,
impact, pay-to-play, and gatekeeping.

For example, the connectionist field continues to assume that core
inference is a feedforward process while not requiring the necessary
rehearsal mechanisms for its learning to be implemented with neurons.
There is research showing that inference is possible in a
feedforward-feedback process which simplifies learning (no need for network
normalization parameters, balanced data and onerous iid rehearsal). It is
also a step forward for continuous learning. Some of this work on
regulatory feedback is now over 20 years old.

The predominant theories of biological learning and pattern recognition use
feedforward weights in alignment with current AI: inputs multiplied by
weights to obtain outputs (inputs-to-output).
Confusingly named recurrent neural networks, or neural networks with loops
allow information from later processing stages to feed back to earlier
stages for sequence processing. However, at every stage of inference a
feedforward multiplication remains the core, essential for backpropagation
or backpropagation through time. Thus neural networks cannot contain
feedback like negative feedback
<https://en.wikipedia.org/wiki/Negative_feedback> where the outputs feed
back to the very same inputs and modify them, because this forms an
infinite loop which is not possible to rewind in time to generate an error
signal through backpropagation.
Yet these connections are in the brain and used during recognition and
learning, heck they are the basis of a thermostat.

For example, homeostatic plasticity is found throughout brain recognition
processing areas. It requires pre-synaptic stabilizing feedback to the same
neurons that activate the post-synaptic neuron (negative feedback or
inhibitory feedback connections). In homeostatic plasticity feedback
between pre- and post-synaptic neurons are shown to be meticulously
maintained by homeostatic plasticity through multiple, often redundant
mechanisms [3,4]. Thus it appears their existence and maintenance are very
important to the brain.
The Regulatory Feedback model provides a real time recognition-inference
hypothesis and role for the pre-synaptic inhibition, inhibitory feedback to
the very same inputs, connections which are meticulously maintained.

I am attaching a recent paper [5] and a link to a detailed video
<https://youtu.be/9gTJorBeLi8> [6] about the counterintuitive inference
mechanism .

After over 20 years it is clear that with an academic oligarchy controlling
high impact pay-to-play conferences and platforms there is little
opportunity to speak with enough detail about a completely
unintuitive mechanism given today's norms or publish an article with enough
Impact.

It is this oligarchy with its less-than-honest depiction about open
policies while accepting funding that deserves Elon Musk and company.

Sincerely,
-Tsvi Achler

1) Azoulay P, Fons-Rosen C, Zivin JSG. Does Science Advance One Funeral at
a Time? Am Econ Rev. 2019 Aug;109(8):2889-2920. doi: 10.1257/aer.20161574.
PMID: 31656315; PMCID: PMC6814193.
2) Casadevall A, Fang FC. Impacted science: impact is not importance. mBio.
2015 Oct 13;6(5):e01593-15. doi: 10.1128/mBio.01593-15. PMID: 26463169;
PMCID: PMC4620476.
3) Marder E, Goaillard JM. Variability, compensation and homeostasis in
neuron and network function. Nat Rev Neurosci. 2006;7:563–74.
https://doi.org/10.1038/nrn1949.
4) Turrigiano G. Homeostatic synaptic plasticity: local and
global mechanisms for stabilizing neuronal function. Cold Spring
Harb Perspect Biol. 2012.
5) Achler, T. What AI, Neuroscience, and Cognitive Science Can Learn from
Each Other: An Embedded Perspective. Cogn Comput 16, 2428–2436 (2024).
https://doi.org/10.1007/s12559-023-10194-9
6)  Achler T. Neural phenomena focus. 2016. https://youtu.be/9gTJorBeLi8

On Mon, Mar 3, 2025 at 11:21 AM Gary Marcus <gary.marcus at nyu.edu> wrote:

> [image: 0c4823b8-7a3b-4c5c-8862-51cf7d4178a5_1423x1925.jpeg]
>
> Hinton vs Musk
> <https://open.substack.com/pub/garymarcus/p/hinton-vs-musk?r=8tdk6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false>
> open.substack.com
> <https://open.substack.com/pub/garymarcus/p/hinton-vs-musk?r=8tdk6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false>
>
> <https://open.substack.com/pub/garymarcus/p/hinton-vs-musk?r=8tdk6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20250304/4ae60d3f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0c4823b8-7a3b-4c5c-8862-51cf7d4178a5_1423x1925.jpeg
Type: image/jpeg
Size: 84864 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20250304/4ae60d3f/attachment-0001.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: What AI and Neuroscience Can Learn From Each Other - Cognitive Computation.pdf
Type: application/pdf
Size: 775209 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20250304/4ae60d3f/attachment-0001.pdf>