<div dir="ltr">Thank you Richard and Danko for helping answer Adam's questions/comments.<div><br></div><div>Adam's questions make sense when given the outward narrative of academic departments, grant agencies and even companies. These narratives are meant to bring more funds to the institutions promulgating them, but ultimately misinform the public and naïve students. The reality is much different. Think of statements coming from academia being as reliable as statements coming from politicians who are trying to get themselves promoted and elected.</div><div><br></div><div>To address the statement "<font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">I think you will find that the community is incredibly open" </span></font> from an academic - computational neuroscience perspective <font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">I have found it is absolutely the opposite. Each field is siloed and unnecessarily competitive.</span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px"><br></span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">Notice for example when I talked about my approach I got 3 responses suggesting I am mistaken and what I am doing is X and I should read Y. Each X and Y varied greatly (there is no way all of them could be true at once and most likely the commenters have not even looked at the mechanism) yet any such review coming back from an article or grant application would reject the article or grant.</span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px"><br></span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">I have written countless grant and articles where I purposefully write several times in the abstract, and within the article that: the process of feedback in regulatory feedback occurs during recognition, not to adjust weights but to determine neuron activations. The review comes back with: clearly the author does not understand how learning in feedforward works, they are wrong and feedback occurs during learning to adjust weights.</span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px"><br></span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">One such review will reject the work </span><span style="font-size:14.6667px">and in a group of 3 to 4 reviewers there is very little chance that all of the reviewers that did not completely understand the novel mechanism not will say "</span></font><span style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14.6667px">wrong this is X read Y </span><span style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14.6667px">"</span><span style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14.6667px"> </span><span style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14.6667px">instead</span><span style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14.6667px"> of all saying hey this is a mechanism I am </span><span style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14.6667px">unfamiliar</span><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px"> with, but seems interesting so publish it/award funding anyways. This is why novel research cannot be published or supported by academia. This is in huge contrast to grant agencies and departments who make decisions by committees going beyond themselves to explain how they are looking for novel and multidisciplinary research. In fact I talked to an NSF director who said that they are conservative because of committee decisions. When I asked him then why do you project the opposite message about novelty, he replied "this is operational". Thus nonchalantly acknowledging that projecting deceiving messages is part of their core.</span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px"><br></span></font></div><div><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">Now the </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">business</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> world is more open minded and honest (your "google" statements are actually referring to the </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">business</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> world). They do not care what type of mechanism as long as it "works". But what they do care about is solving a </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">business</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> case; that is solving a </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">business</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> problem and demonstrating that it makes or saves money. Often the value of solution is nothing about the technology but knowing how to </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">address</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> a certain business problem. O</span><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">ne thing that they expressly do not want to invest in is a "science project". Thus as a person who focused on brain science neuroscience, cognitive psychology, etc. is not </span></font><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">particularly</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> prepared to </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">address</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> </span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif">business</span><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"> problems. Although not ideal for a scientist, I find it is at least a more open minded, honest and forthcoming system and indeed I am working on the business problem these days.</span><br></div><div><span style="font-size:14.6667px;color:rgb(0,0,0);font-family:Calibri,sans-serif"><br></span></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">Adam, I hope that answers some of your (and others') questions. I have also created a video channel to help those from the outside understand how the problems of academia occur and possibly how they may be solved. See the "Updating Research" channel and playlist </span></font><a href="https://www.youtube.com/playlist?list=PLM3bZImI0fj3rM3ZrzSYbfozkf8m4102j">https://www.youtube.com/playlist?list=PLM3bZImI0fj3rM3ZrzSYbfozkf8m4102j</a> </div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px"><br></span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">Sincerely,</span></font></div><div><font color="#000000" face="Calibri, sans-serif"><span style="font-size:14.6667px">-Tsvi</span></font></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Nov 5, 2021 at 2:13 PM Adam Krawitz <<a href="mailto:akrawitz@uvic.ca" target="_blank">akrawitz@uvic.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-CA">
<div>
<p class="MsoNormal"><span lang="EN-US">Tsvi,<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>I’m just a lurker on this list, with no skin in the game, but perhaps that gives me a more neutral perspective. In the spirit of progress:<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<ol style="margin-top:0cm" start="1" type="1">
<li style="margin-left:0cm"><span>If you have a neural network approach that you feel provides a new and important perspective on cognitive processes, then write up a paper
making that argument clearly, and I think you will find that the community is incredibly open to that. Yes, if they see holes in the approach they will be pointed out, but that is all part of the scientific exchange. Examples of this approach include: Elman
(1990) Finding Structure in Time, Kohonen (1990) The Self-Organizing Map, Tenenbaum et al. (2011) How to Grow a Mind: Statistics, Structure, and Abstraction (not neural nets, but a “new” approach to modelling cognition). I’m sure others can provide more examples.<u></u><u></u></span></li><li style="margin-left:0cm"><span>I’m much less familiar with how things work on the applied side, but I have trouble believing that Google or anyone else will be dismissive
of a computational approach that actually works. Why would they? They just want to solve problems efficiently. Demonstrate that your approach can solve a problem more effectively (or at least as effectively) as the existing approaches, and they will come running.
Examples of this include: Tesauro’s TD-Gammon, which was influential in demonstrating the power of RL, and LeCun et al.’s convolutional NN for the MNIST digits.<u></u><u></u></span></li></ol>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>Clearly communicate the novel contribution of your approach and I think you will find a receptive audience.<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>Thanks,<u></u><u></u></span></p>
<p class="MsoNormal"><span>Adam<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Connectionists <<a href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu" target="_blank">connectionists-bounces@mailman.srv.cs.cmu.edu</a>>
<b>On Behalf Of </b>Tsvi Achler<br>
<b>Sent:</b> November 4, 2021 9:46 AM<br>
<b>To:</b> <a href="mailto:gary@ucsd.edu" target="_blank">gary@ucsd.edu</a><br>
<b>Cc:</b> <a href="mailto:connectionists@cs.cmu.edu" target="_blank">connectionists@cs.cmu.edu</a><br>
<b>Subject:</b> Re: Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Lastly Feedforward methods are predominant in a large part because they have financial backing from large companies with advertising and clout like Google and the self-driving craze that never fully materialized. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Feedforward methods are not fully connectionist unless rehearsal for learning is implemented with neurons. That means storing all patterns, mixing them randomly and then presenting to a network to learn. As far as I know, no one is doing
this in the community, so feedforward methods are only partially connectionist. By allowing popularity to predominate and choking off funds and presentation of alternatives we are cheating ourselves from pursuing other more rigorous brain-like methods.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-Tsvi<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Nov 2, 2021 at 7:08 PM Tsvi Achler <<a href="mailto:achler@gmail.com" target="_blank">achler@gmail.com</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal">Gary- Thanks for the accessible online link to the book. <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I looked especially at the inhibitory feedback section of the book which describes an Air Conditioner AC type feedback. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">It then describes a general field-like inhibition based on all activations in the layer. It also describes the role of inhibition in sparsity and feedforward inhibition,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The feedback described in Regulatory Feedback is similar to the AC feedback but occurs for each neuron individually, vis-a-vis its inputs.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thus for context, regulatory feedback is not a field-like inhibition, it is very directed based on the neurons that are activated and their inputs. This sort of regulation is also the foundation of Homeostatic Plasticity findings (albeit
with changes in Homeostatic regulation in experiments occurring in a slower time scale). The regulatory feedback model describes the effect and role in recognition of those regulated connections in real time during recognition.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I would be happy to discuss further and collaborate on writing about the differences between the approaches for the next book or review.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">And I want to point out to folks, that the system is based on politics and that is why certain work is not cited like it should, but even worse these politics are here in the group today and they continue to very strongly influence decisions
in the connectionist community and holds us back.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-Tsvi<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Nov 1, 2021 at 10:59 AM <a href="mailto:gary@ucsd.edu" target="_blank">
gary@ucsd.edu</a> <<a href="mailto:gary@eng.ucsd.edu" target="_blank">gary@eng.ucsd.edu</a>> wrote:<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:18pt;font-family:"Times New Roman",serif">Tsvi - While I think
<a href="https://www.amazon.com/dp/0262650541/" target="_blank">Randy and Yuko's book
</a>is actually somewhat better than the online version (and buying choices on amazon start at $9.99), there
<b>is</b> <a href="https://compcogneuro.org/" target="_blank">an online version.</a> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:18pt;font-family:"Times New Roman",serif">Randy & Yuko's models take into account feedback and inhibition. <u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Nov 1, 2021 at 10:05 AM Tsvi Achler <<a href="mailto:achler@gmail.com" target="_blank">achler@gmail.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal">Daniel,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Does your book include a discussion of Regulatory or Inhibitory Feedback published in several low impact journals between 2008 and 2014 (and in videos subsequently)?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">These are networks where the primary computation is inhibition back to the inputs that activated them and may be very counterintuitive given today's trends. You can almost think of them as the opposite of Hopfield networks.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I would love to check inside the book but I dont have an academic budget that allows me access to it and that is a huge part of the problem with how information is shared and funding is allocated. I could not get access to any of the text
or citations especially Chapter 4: "Competition, Lateral Inhibition, and Short-Term Memory", to weigh in.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I wish the best circulation for your book, but even if the Regulatory Feedback Model is in the book, that does not change the fundamental problem if the book is not readily available. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The same goes with Steve Grossberg's book, I cannot easily look inside. With regards to Adaptive Resonance I dont subscribe to lateral inhibition as a predominant mechanism, but I do believe a function such as vigilance is very important
during recognition and Adaptive Resonance is one of a very few models that have it. The Regulatory Feedback model I have developed (and Michael Spratling studies a similar model as well) is built primarily using the vigilance type of connections and allows
multiple neurons to be evaluated at the same time and continuously during recognition in order to determine which (single or multiple neurons together) match the inputs the best without lateral inhibition.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Unfortunately within conferences and talks predominated by the Adaptive Resonance crowd I have experienced the familiar dismissiveness and did not have an opportunity to give a proper talk. This goes back to the larger issue of academic
politics based on small self-selected committees, the same issues that exist with the feedforward crowd, and pretty much all of academia.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Today's information age algorithms such as Google's can determine relevance of information and ways to display them, but hegemony of the journal systems and the small committee system of academia developed in the middle ages (and their
mutual synergies) block the use of more modern methods in research. Thus we are stuck with this problem, which especially affects those that are trying to introduce something new and counterintuitive, and hence the results described in the two National Bureau
of Economic Research articles I cited in my previous message.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Thomas, I am happy to have more discussions and/or start a different thread.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Tsvi Achler MD/PhD<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Sun, Oct 31, 2021 at 12:49 PM Levine, Daniel S <<a href="mailto:levine@uta.edu" target="_blank">levine@uta.edu</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black">Tsvi,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black">While deep learning and feedforward networks have an outsize popularity, there are plenty of published sources that cover a much wider variety of networks, many of them more biologically based
than deep learning. A treatment of a range of neural network approaches, going from simpler to more complex cognitive functions, is found in my textbook
<i>Introduction to Neural and Cognitive Modeling</i> (3rd edition, Routledge, 2019). Also Steve Grossberg's book
<i>Conscious Mind, Resonant Brain</i> (Oxford, 2021) emphasizes a variety of architectures with a strong biological basis.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black">Best,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;color:black">Dan Levine<u></u><u></u></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="gmail-m_-7927256727939359610gmail-m_7233223245906544196gmail-m_-2278469877946975064gmail-m_-3449454411413463923gmail-m_-99491911606122447gmail-m_-1743973086040584978m_-7977962918574028451m_3158366256145482305m_-7622355768744816385m_-898643676068276388gmail-m_-7809501330019106925gmail-m_8996447038276730094gmail-m_-2305817410909496922gmail-m_7665975300539281535divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Connectionists <<a href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu" target="_blank">connectionists-bounces@mailman.srv.cs.cmu.edu</a>> on behalf of Tsvi
Achler <<a href="mailto:achler@gmail.com" target="_blank">achler@gmail.com</a>><br>
<b>Sent:</b> Saturday, October 30, 2021 3:13 AM<br>
<b>To:</b> Schmidhuber Juergen <<a href="mailto:juergen@idsia.ch" target="_blank">juergen@idsia.ch</a>><br>
<b>Cc:</b> <a href="mailto:connectionists@cs.cmu.edu" target="_blank">connectionists@cs.cmu.edu</a> <<a href="mailto:connectionists@cs.cmu.edu" target="_blank">connectionists@cs.cmu.edu</a>><br>
<b>Subject:</b> Re: Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.</span>
<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Since the title of the thread is Scientific Integrity, I want to point out some issues about trends in academia and then especially focusing on the connectionist community.
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">In general analyzing impact factors etc the most important progress gets silenced until the mainstream picks it up <a href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nber.org%2Fsystem%2Ffiles%2Fworking_papers%2Fw22180%2Fw22180.pdf%3Ffbclid%3DIwAR1zHhU4wmkrHASTaE-6zwIs6gI9-FxZcCED3BETxUJlMsbN_2hNbmJAmOA&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300122043%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9o%2FzcYY8gZVZiAwyEL5SVI9TEzBWfKf7nfhdWWg8LHU%3D&reserved=0" target="_blank">Impact
Factiors in novel research www.nber.org/.../working_papers/w22180/w22180.pdf</a> and often this may take a generation <a href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nber.org%2Fdigest%2Fmar16%2Fdoes-science-advance-one-funeral-time%3Ffbclid%3DIwAR1Lodsf1bzje-yQU9DvoZE2__S6R7UPEgY1_LxZCSLdoAYnj-uco0JuyVk&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300132034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=DgxnJTT7MsN5KCzZlA7VAHKrHXVsRsYhopJv0FCwbtw%3D&reserved=0" target="_blank">https://www.nber.org/.../does-science-advance-one-funeral...</a> .<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The connectionist field is stuck on feedforward networks and variants such as with inhibition of competitors (e.g. lateral inhibition), or other variants that are sometimes labeled as recurrent networks for learning time where the feedforward
networks can be rewound in time.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This stasis is specifically occuring with the popularity of deep learning. This is often portrayed as neurally plausible connectionism but requires an implausible amount of rehearsal and is not connectionist if this rehearsal is not implemented
with neurons (see video link for further clarification).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Models which have true feedback (e.g. back to their own inputs) cannot learn by backpropagation but there is plenty of evidence these types of connections exist in the brain and are used during recognition. Thus they get ignored: no talks
in universities, no featuring in "premier" journals and no funding. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">But they are important and may negate the need for rehearsal as needed in feedforward methods. Thus may be essential for moving connectionism forward.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">If the community is truly dedicated to brain motivated algorithms, I recommend giving more time to networks other than feedforward networks.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Video: <a href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dm2qee6j5eew%26list%3DPL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3%26index%3D2&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300132034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EaEp5zLZ7HkDhsBHmP3x3ObPl8j14B8%2BFcOkkNEWZ9w%3D&reserved=0" target="_blank">https://www.youtube.com/watch?v=m2qee6j5eew&list=PL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3&index=2</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Tsvi Achler<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Wed, Oct 27, 2021 at 2:24 AM Schmidhuber Juergen <<a href="mailto:juergen@idsia.ch" target="_blank">juergen@idsia.ch</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-bottom:12pt">Hi, fellow artificial neural network enthusiasts!<br>
<br>
The connectionists mailing list is perhaps the oldest mailing list on ANNs, and many neural net pioneers are still subscribed to it. I am hoping that some of them - as well as their contemporaries - might be able to provide additional valuable insights into
the history of the field.<br>
<br>
Following the great success of massive open online peer review (MOOR) for my 2015 survey of deep learning (now the most cited article ever published in the journal Neural Networks), I've decided to put forward another piece for MOOR. I want to thank the many
experts who have already provided me with comments on it. Please send additional relevant references and suggestions for improvements for the following draft directly to me at
<a href="mailto:juergen@idsia.ch" target="_blank">juergen@idsia.ch</a>:<br>
<br>
<a href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.idsia.ch%2F~juergen%2Fscientific-integrity-turing-award-deep-learning.html&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300142030%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mW3lH7SqKg4EuJfDwKcC2VhwEloC3ndh6kI5gfQ2Ofw%3D&reserved=0" target="_blank">https://people.idsia.ch/~juergen/scientific-integrity-turing-award-deep-learning.html</a><br>
<br>
The above is a point-for-point critique of factual errors in ACM's justification of the ACM A. M. Turing Award for deep learning and a critique of the Turing Lecture published by ACM in July 2021. This work can also be seen as a short history of deep learning,
at least as far as ACM's errors and the Turing Lecture are concerned.<br>
<br>
I know that some view this as a controversial topic. However, it is the very nature of science to resolve controversies through facts. Credit assignment is as core to scientific history as it is to machine learning. My aim is to ensure that the true history
of our field is preserved for posterity.<br>
<br>
Thank you all in advance for your help! <br>
<br>
Jürgen Schmidhuber<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<p class="MsoNormal">Gary Cottrell 858-534-6640 FAX: 858-534-7029<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">Computer Science and Engineering 0404<br>
IF USING FEDEX INCLUDE THE FOLLOWING LINE: <br>
CSE Building, Room 4130<br>
University of California San Diego -<br>
9500 Gilman Drive # 0404<br>
La Jolla, Ca. 92093-0404<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Email: <a href="mailto:gary@ucsd.edu" target="_blank">gary@ucsd.edu</a><br>
Home page: <a href="http://www-cse.ucsd.edu/~gary/" target="_blank">http://www-cse.ucsd.edu/~gary/</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt">Schedule: <a href="http://tinyurl.com/b7gxpwo" target="_blank">http://tinyurl.com/b7gxpwo</a></span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt"><u></u> <u></u></span></p>
</div>
<p><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">Listen carefully,</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">Neither the Vedas</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">Nor the Qur'an</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">Will teach you this:</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">Put the bit in its mouth,</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">The saddle on its back,</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">Your foot in the stirrup,</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">And ride your wild runaway mind</span></i><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black"><br>
</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black">All the way to heaven.<u></u><u></u></span></i></p>
<p><i><span style="font-size:14pt;font-family:"Book Antiqua",serif;color:black">-- Kabir</span></i><i><span style="font-size:11.5pt;font-family:"Book Antiqua",serif;color:black"><u></u><u></u></span></i></p>
</div>
</div>
</blockquote></div>