Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.

Mon Nov 1 05:23:25 EDT 2021

Daniel,

Does your book include a discussion of Regulatory or Inhibitory Feedback
published in several low impact journals between 2008 and 2014 (and in
videos subsequently)?
These are networks where the primary computation is inhibition back to the
inputs that activated them and may be very counterintuitive given today's
trends.  You can almost think of them as the opposite of Hopfield networks.

I would love to check inside the book but I dont have an academic budget
that allows me access to it and that is a huge part of the problem with how
information is shared and funding is allocated. I could not get access to
any of the text or citations especially Chapter 4: "Competition, Lateral
Inhibition, and Short-Term Memory", to weigh in.

I wish the best circulation for your book, but even if the Regulatory
Feedback Model is in the book, that does not change the fundamental problem
if the book is not readily available.

The same goes with Steve Grossberg's book, I cannot easily look inside.
With regards to Adaptive Resonance I dont subscribe to lateral inhibition
as a predominant mechanism, but I do believe a function such as vigilance
is very important during recognition and Adaptive Resonance is one of
a very few models that have it.  The Regulatory Feedback model I have
developed (and Michael Spratling studies a similar model as well) is built
primarily using the vigilance type of connections and allows multiple
neurons to be evaluated at the same time and continuously during
recognition in order to determine which (single or multiple neurons
together) match the inputs the best without lateral inhibition.

Unfortunately within conferences and talks predominated by the Adaptive
Resonance crowd I have experienced the familiar dismissiveness and did not
have an opportunity to give a proper talk. This goes back to the larger
issue of academic politics based on small self-selected committees, the
same issues that exist with the feedforward crowd, and pretty much all of
academia.

Today's information age algorithms such as Google's can determine relevance
of information and ways to display them, but hegemony of the journal
systems and the small committee system of academia developed in the middle
ages (and their mutual synergies) block the use of more modern methods in
research.  Thus we are stuck with this problem, which especially affects
those that are trying to introduce something new and counterintuitive, and
hence the results described in the two National Bureau of Economic Research
articles I cited in my previous message.

Thomas, I am happy to have more discussions and/or start a different thread.

Sincerely,
Tsvi Achler MD/PhD

On Sun, Oct 31, 2021 at 12:49 PM Levine, Daniel S <levine at uta.edu> wrote:

> Tsvi,
>
> While deep learning and feedforward networks have an outsize popularity,
> there are plenty of published sources that cover a much wider variety of
> networks, many of them more biologically based than deep learning.  A
> treatment of a range of neural network approaches, going from simpler to
> more complex cognitive functions, is found in my textbook * Introduction
> to Neural and Cognitive Modeling* (3rd edition, Routledge, 2019).  Also
> Steve Grossberg's book *Conscious Mind, Resonant Brain* (Oxford, 2021)
> emphasizes a variety of architectures with a strong biological basis.
>
>
> Best,
>
>
> Dan Levine
> ------------------------------
> *From:* Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu> on
> behalf of Tsvi Achler <achler at gmail.com>
> *Sent:* Saturday, October 30, 2021 3:13 AM
> *To:* Schmidhuber Juergen <juergen at idsia.ch>
> *Cc:* connectionists at cs.cmu.edu <connectionists at cs.cmu.edu>
> *Subject:* Re: Connectionists: Scientific Integrity, the 2021 Turing
> Lecture, etc.
>
> Since the title of the thread is Scientific Integrity, I want to point out
> some issues about trends in academia and then especially focusing on the
> connectionist community.
>
> In general analyzing impact factors etc the most important progress gets
> silenced until the mainstream picks it up Impact Factiors in novel
> research www.nber.org/.../working_papers/w22180/w22180.pdf
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nber.org%2Fsystem%2Ffiles%2Fworking_papers%2Fw22180%2Fw22180.pdf%3Ffbclid%3DIwAR1zHhU4wmkrHASTaE-6zwIs6gI9-FxZcCED3BETxUJlMsbN_2hNbmJAmOA&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300122043%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9o%2FzcYY8gZVZiAwyEL5SVI9TEzBWfKf7nfhdWWg8LHU%3D&reserved=0>  and
> often this may take a generation
> https://www.nber.org/.../does-science-advance-one-funeral...
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nber.org%2Fdigest%2Fmar16%2Fdoes-science-advance-one-funeral-time%3Ffbclid%3DIwAR1Lodsf1bzje-yQU9DvoZE2__S6R7UPEgY1_LxZCSLdoAYnj-uco0JuyVk&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300132034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=DgxnJTT7MsN5KCzZlA7VAHKrHXVsRsYhopJv0FCwbtw%3D&reserved=0>
>   .
>
> The connectionist field is stuck on feedforward networks and variants such
> as with inhibition of competitors (e.g. lateral inhibition), or other
> variants that are sometimes labeled as recurrent networks for learning time
> where the feedforward networks can be rewound in time.
>
> This stasis is specifically occuring with the popularity of deep
> learning.  This is often portrayed as neurally plausible connectionism but
> requires an implausible amount of rehearsal and is not connectionist if
> this rehearsal is not implemented with neurons (see video link for further
> clarification).
>
> Models which have true feedback (e.g. back to their own inputs) cannot
> learn by backpropagation but there is plenty of evidence these types of
> connections exist in the brain and are used during recognition. Thus they
> get ignored: no talks in universities, no featuring in "premier" journals
> and no funding.
>
> But they are important and may negate the need for rehearsal as needed in
> feedforward methods.  Thus may be essential for moving connectionism
> forward.
>
> If the community is truly dedicated to brain motivated algorithms, I
> recommend giving more time to networks other than feedforward networks.
>
> Video:
> https://www.youtube.com/watch?v=m2qee6j5eew&list=PL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3&index=2
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dm2qee6j5eew%26list%3DPL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3%26index%3D2&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300132034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EaEp5zLZ7HkDhsBHmP3x3ObPl8j14B8%2BFcOkkNEWZ9w%3D&reserved=0>
>
> Sincerely,
> Tsvi Achler
>
>
>
> On Wed, Oct 27, 2021 at 2:24 AM Schmidhuber Juergen <juergen at idsia.ch>
> wrote:
>
> Hi, fellow artificial neural network enthusiasts!
>
> The connectionists mailing list is perhaps the oldest mailing list on
> ANNs, and many neural net pioneers are still subscribed to it. I am hoping
> that some of them - as well as their contemporaries - might be able to
> provide additional valuable insights into the history of the field.
>
> Following the great success of massive open online peer review (MOOR) for
> my 2015 survey of deep learning (now the most cited article ever published
> in the journal Neural Networks), I've decided to put forward another piece
> for MOOR. I want to thank the many experts who have already provided me
> with comments on it. Please send additional relevant references and
> suggestions for improvements for the following draft directly to me at
> juergen at idsia.ch:
>
>
> https://people.idsia.ch/~juergen/scientific-integrity-turing-award-deep-learning.html
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.idsia.ch%2F~juergen%2Fscientific-integrity-turing-award-deep-learning.html&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300142030%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mW3lH7SqKg4EuJfDwKcC2VhwEloC3ndh6kI5gfQ2Ofw%3D&reserved=0>
>
> The above is a point-for-point critique of factual errors in ACM's
> justification of the ACM A. M. Turing Award for deep learning and a
> critique of the Turing Lecture published by ACM in July 2021. This work can
> also be seen as a short history of deep learning, at least as far as ACM's
> errors and the Turing Lecture are concerned.
>
> I know that some view this as a controversial topic. However, it is the
> very nature of science to resolve controversies through facts. Credit
> assignment is as core to scientific history as it is to machine learning.
> My aim is to ensure that the true history of our field is preserved for
> posterity.
>
> Thank you all in advance for your help!
>
> Jürgen Schmidhuber
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20211101/e793c87b/attachment.html>