<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Adam,</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">1) Tsvi Achler has already done the
things you ask, many times over, so it behooves you to check for
that before you tell him to do it. Instructing someone to "<span
style="mso-fareast-language:EN-US">clearly communicate the novel
contribution of your approach" when they have already done is is
an insult.<br>
</span></div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">2) The whole point of this discussion
is that when someone "makes an argument clearly" the community is
NOT "<span style="mso-fareast-language:EN-US">incredibly open to
that." Quite the opposite: the community's attention is fickle,
tribal, fad-driven, and fundamentally broken.</span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><br>
</span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US">3) When you say that you "</span><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US">have trouble believing that
Google or anyone else will be dismissive of a computational
approach that actually works," that truly boggles the mind.</span></span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US"><br>
</span></span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US"> a) There is no precise
definition for "actually works" -- there is no global measure
of goodness in the space of approaches.</span></span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US"><br>
</span></span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US"> b) Getting the
attention of someone at e.g. Google is a non-trivial feat in
itself: just ignoring outsiders is, for Google, a perfectly
acceptable option.</span></span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US"><br>
</span></span></div>
<div class="moz-cite-prefix"><span
style="mso-fareast-language:EN-US"><span
style="mso-fareast-language:EN-US"> c) What do you suppose
would be the reaction of an engineer at Google who gets handed
a paper by their boss, and is asked "What do you think of
this?" Suppose the paper describes an approach that is
inimicable to what that engineer has been doing their whole
career. So much so, that if Google goes all-in on this new
thing, the engineer's skillset will be devalued to junk
status. What would the engineer do? They would say "I read
it. It's just garbage."<br>
</span></span></div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Best</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Richard Loosemore</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 11/5/21 1:01 PM, Adam Krawitz wrote:<br>
</div>
<blockquote type="cite"
cite="mid:d6a74d88665d42318db545a1a08b25a4@tailor.uvic.ca">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:"Book Antiqua";
panose-1:2 4 6 2 5 3 5 3 3 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}ol
{margin-bottom:0cm;}ul
{margin-bottom:0cm;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Tsvi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I’m
just a lurker on this list, with no skin in the game, but
perhaps that gives me a more neutral perspective. In the
spirit of progress:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<ol style="margin-top:0cm" type="1" start="1">
<li class="MsoListParagraph"
style="margin-left:0cm;mso-list:l0 level1 lfo1"><span
style="mso-fareast-language:EN-US">If you have a neural
network approach that you feel provides a new and
important perspective on cognitive processes, then write
up a paper making that argument clearly, and I think you
will find that the community is incredibly open to that.
Yes, if they see holes in the approach they will be
pointed out, but that is all part of the scientific
exchange. Examples of this approach include: Elman (1990)
Finding Structure in Time, Kohonen (1990) The
Self-Organizing Map, Tenenbaum et al. (2011) How to Grow a
Mind: Statistics, Structure, and Abstraction (not neural
nets, but a “new” approach to modelling cognition). I’m
sure others can provide more examples.<o:p></o:p></span></li>
<li class="MsoListParagraph"
style="margin-left:0cm;mso-list:l0 level1 lfo1"><span
style="mso-fareast-language:EN-US">I’m much less familiar
with how things work on the applied side, but I have
trouble believing that Google or anyone else will be
dismissive of a computational approach that actually
works. Why would they? They just want to solve problems
efficiently. Demonstrate that your approach can solve a
problem more effectively (or at least as effectively) as
the existing approaches, and they will come running.
Examples of this include: Tesauro’s TD-Gammon, which was
influential in demonstrating the power of RL, and LeCun et
al.’s convolutional NN for the MNIST digits.<o:p></o:p></span></li>
</ol>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Clearly
communicate the novel contribution of your approach and I
think you will find a receptive audience.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Adam<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> Connectionists
<a class="moz-txt-link-rfc2396E" href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu"><connectionists-bounces@mailman.srv.cs.cmu.edu></a>
<b>On Behalf Of </b>Tsvi Achler<br>
<b>Sent:</b> November 4, 2021 9:46 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:gary@ucsd.edu">gary@ucsd.edu</a><br>
<b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:connectionists@cs.cmu.edu">connectionists@cs.cmu.edu</a><br>
<b>Subject:</b> Re: Connectionists: Scientific Integrity,
the 2021 Turing Lecture, etc.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Lastly Feedforward methods are
predominant in a large part because they have financial
backing from large companies with advertising and clout
like Google and the self-driving craze that never fully
materialized. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Feedforward methods are not fully
connectionist unless rehearsal for learning is implemented
with neurons. That means storing all patterns, mixing
them randomly and then presenting to a network to learn.
As far as I know, no one is doing this in the community,
so feedforward methods are only partially connectionist.
By allowing popularity to predominate and choking off
funds and presentation of alternatives we are cheating
ourselves from pursuing other more rigorous brain-like
methods.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-Tsvi<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Tue, Nov 2, 2021 at 7:08 PM Tsvi
Achler <<a href="mailto:achler@gmail.com" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">achler@gmail.com</a>>
wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal">Gary- Thanks for the accessible online link
to the book. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I looked especially at the inhibitory
feedback section of the book which describes an Air
Conditioner AC type feedback. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">It then describes a general field-like
inhibition based on all activations in the layer. It also
describes the role of inhibition in sparsity and feedforward
inhibition,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The feedback described in Regulatory
Feedback is similar to the AC feedback but occurs for each
neuron individually, vis-a-vis its inputs.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Thus for context, regulatory feedback is
not a field-like inhibition, it is very directed based on
the neurons that are activated and their inputs. This sort
of regulation is also the foundation of Homeostatic
Plasticity findings (albeit with changes in Homeostatic
regulation in experiments occurring in a slower time
scale). The regulatory feedback model describes the effect
and role in recognition of those regulated connections in
real time during recognition.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I would be happy to discuss further and
collaborate on writing about the differences between the
approaches for the next book or review.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">And I want to point out to folks, that
the system is based on politics and that is why certain work
is not cited like it should, but even worse these politics
are here in the group today and they continue to very
strongly influence decisions in the connectionist community
and holds us back.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-Tsvi<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Mon, Nov 1, 2021 at 10:59 AM <a
href="mailto:gary@ucsd.edu" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">
gary@ucsd.edu</a> <<a href="mailto:gary@eng.ucsd.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">gary@eng.ucsd.edu</a>>
wrote:<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:18.0pt;font-family:"Times New
Roman",serif">Tsvi - While I think
<a href="https://www.amazon.com/dp/0262650541/"
target="_blank" moz-do-not-send="true">Randy and
Yuko's book
</a>is actually somewhat better than the online version
(and buying choices on amazon start at $9.99), there
<b>is</b> <a href="https://compcogneuro.org/"
target="_blank" moz-do-not-send="true">an online
version.</a> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:18.0pt;font-family:"Times New
Roman",serif">Randy & Yuko's models take into
account feedback and inhibition. <o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Nov 1, 2021 at 10:05 AM Tsvi
Achler <<a href="mailto:achler@gmail.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">achler@gmail.com</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0cm 0cm 0cm
6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal">Daniel,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Does your book include a
discussion of Regulatory or Inhibitory Feedback
published in several low impact journals between
2008 and 2014 (and in videos subsequently)?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">These are networks where the
primary computation is inhibition back to the inputs
that activated them and may be very counterintuitive
given today's trends. You can almost think of them
as the opposite of Hopfield networks.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I would love to check inside the
book but I dont have an academic budget that allows
me access to it and that is a huge part of the
problem with how information is shared and
funding is allocated. I could not get access to any
of the text or citations especially Chapter 4:
"Competition, Lateral Inhibition, and Short-Term
Memory", to weigh in.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I wish the best circulation for
your book, but even if the Regulatory Feedback Model
is in the book, that does not change the fundamental
problem if the book is not readily available. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The same goes with Steve
Grossberg's book, I cannot easily look inside. With
regards to Adaptive Resonance I dont subscribe to
lateral inhibition as a predominant mechanism, but I
do believe a function such as vigilance is very
important during recognition and Adaptive Resonance
is one of a very few models that have it. The
Regulatory Feedback model I have developed (and
Michael Spratling studies a similar model as well)
is built primarily using the vigilance type of
connections and allows multiple neurons to be
evaluated at the same time and continuously during
recognition in order to determine which (single or
multiple neurons together) match the inputs the best
without lateral inhibition.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Unfortunately within conferences
and talks predominated by the Adaptive Resonance
crowd I have experienced the familiar dismissiveness
and did not have an opportunity to give a proper
talk. This goes back to the larger issue of academic
politics based on small self-selected committees,
the same issues that exist with the feedforward
crowd, and pretty much all of academia.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Today's information age
algorithms such as Google's can determine relevance
of information and ways to display them, but
hegemony of the journal systems and the small
committee system of academia developed in the middle
ages (and their mutual synergies) block the use of
more modern methods in research. Thus we are stuck
with this problem, which especially affects those
that are trying to introduce something new and
counterintuitive, and hence the results described in
the two National Bureau of Economic Research
articles I cited in my previous message.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Thomas,
I am happy to have more discussions and/or start a
different thread.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Tsvi Achler MD/PhD<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Sun, Oct 31, 2021 at 12:49 PM
Levine, Daniel S <<a href="mailto:levine@uta.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">levine@uta.edu</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0cm 0cm 0cm
6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">Tsvi,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">While
deep learning and feedforward networks have an
outsize popularity, there are plenty of
published sources that cover a much wider
variety of networks, many of them more
biologically based than deep learning. A
treatment of a range of neural network
approaches, going from simpler to more complex
cognitive functions, is found in my textbook
<i>Introduction to Neural and Cognitive
Modeling</i> (3rd edition, Routledge,
2019). Also Steve Grossberg's book
<i>Conscious Mind, Resonant Brain</i> (Oxford,
2021) emphasizes a variety of architectures
with a strong biological basis.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">Best,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">Dan
Levine<o:p></o:p></span></p>
</div>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr width="98%" size="2" align="center">
</div>
<div
id="gmail-m_-3449454411413463923gmail-m_-99491911606122447gmail-m_-1743973086040584978m_-7977962918574028451m_3158366256145482305m_-7622355768744816385m_-898643676068276388gmail-m_-7809501330019106925gmail-m_8996447038276730094gmail-m_-2305817410909496922gmail-m_7665975300539281535divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span
style="color:black"> Connectionists <<a
href="mailto:connectionists-bounces@mailman.srv.cs.cmu.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">connectionists-bounces@mailman.srv.cs.cmu.edu</a>>
on behalf of Tsvi Achler <<a
href="mailto:achler@gmail.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">achler@gmail.com</a>><br>
<b>Sent:</b> Saturday, October 30, 2021 3:13
AM<br>
<b>To:</b> Schmidhuber Juergen <<a
href="mailto:juergen@idsia.ch"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">juergen@idsia.ch</a>><br>
<b>Cc:</b> <a
href="mailto:connectionists@cs.cmu.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">connectionists@cs.cmu.edu</a>
<<a href="mailto:connectionists@cs.cmu.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">connectionists@cs.cmu.edu</a>><br>
<b>Subject:</b> Re: Connectionists: Scientific
Integrity, the 2021 Turing Lecture, etc.</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Since the title of the
thread is Scientific Integrity, I want to
point out some issues about trends in academia
and then especially focusing on the
connectionist community.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">In general analyzing
impact factors etc the most important
progress gets silenced until the
mainstream picks it up <a
href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nber.org%2Fsystem%2Ffiles%2Fworking_papers%2Fw22180%2Fw22180.pdf%3Ffbclid%3DIwAR1zHhU4wmkrHASTaE-6zwIs6gI9-FxZcCED3BETxUJlMsbN_2hNbmJAmOA&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300122043%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9o%2FzcYY8gZVZiAwyEL5SVI9TEzBWfKf7nfhdWWg8LHU%3D&reserved=0"
target="_blank" moz-do-not-send="true">Impact
Factiors in novel research
www.nber.org/.../working_papers/w22180/w22180.pdf</a> and
often this may take a generation <a
href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nber.org%2Fdigest%2Fmar16%2Fdoes-science-advance-one-funeral-time%3Ffbclid%3DIwAR1Lodsf1bzje-yQU9DvoZE2__S6R7UPEgY1_LxZCSLdoAYnj-uco0JuyVk&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300132034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=DgxnJTT7MsN5KCzZlA7VAHKrHXVsRsYhopJv0FCwbtw%3D&reserved=0"
target="_blank" moz-do-not-send="true">https://www.nber.org/.../does-science-advance-one-funeral...</a> .<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The connectionist field
is stuck on feedforward networks and
variants such as with inhibition of
competitors (e.g. lateral inhibition), or
other variants that are sometimes labeled
as recurrent networks for learning time
where the feedforward networks can be
rewound in time.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This stasis is
specifically occuring with the popularity
of deep learning. This is often portrayed
as neurally plausible connectionism but
requires an implausible amount of
rehearsal and is not connectionist if this
rehearsal is not implemented with neurons
(see video link for further
clarification).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Models which have true
feedback (e.g. back to their own inputs)
cannot learn by backpropagation but there
is plenty of evidence these types of
connections exist in the brain and are
used during recognition. Thus they get
ignored: no talks in universities, no
featuring in "premier" journals and no
funding. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">But they are important
and may negate the need for rehearsal as
needed in feedforward methods. Thus may
be essential for moving connectionism
forward.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If the community is
truly dedicated to brain motivated
algorithms, I recommend giving more time
to networks other than feedforward
networks.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Video: <a
href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dm2qee6j5eew%26list%3DPL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3%26index%3D2&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300132034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EaEp5zLZ7HkDhsBHmP3x3ObPl8j14B8%2BFcOkkNEWZ9w%3D&reserved=0"
target="_blank" moz-do-not-send="true">https://www.youtube.com/watch?v=m2qee6j5eew&list=PL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3&index=2</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Sincerely,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Tsvi Achler<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, Oct 27, 2021 at
2:24 AM Schmidhuber Juergen <<a
href="mailto:juergen@idsia.ch"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">juergen@idsia.ch</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid
#CCCCCC 1.0pt;padding:0cm 0cm 0cm
6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"
style="margin-bottom:12.0pt">Hi, fellow
artificial neural network enthusiasts!<br>
<br>
The connectionists mailing list is perhaps
the oldest mailing list on ANNs, and many
neural net pioneers are still subscribed to
it. I am hoping that some of them - as well
as their contemporaries - might be able to
provide additional valuable insights into
the history of the field.<br>
<br>
Following the great success of massive open
online peer review (MOOR) for my 2015 survey
of deep learning (now the most cited article
ever published in the journal Neural
Networks), I've decided to put forward
another piece for MOOR. I want to thank the
many experts who have already provided me
with comments on it. Please send additional
relevant references and suggestions for
improvements for the following draft
directly to me at
<a href="mailto:juergen@idsia.ch"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">juergen@idsia.ch</a>:<br>
<br>
<a
href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.idsia.ch%2F~juergen%2Fscientific-integrity-turing-award-deep-learning.html&data=04%7C01%7Clevine%40uta.edu%7Cb1a267e3b6a64ada666208d99ca37f6d%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C1%7C0%7C637713048300142030%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mW3lH7SqKg4EuJfDwKcC2VhwEloC3ndh6kI5gfQ2Ofw%3D&reserved=0"
target="_blank" moz-do-not-send="true">https://people.idsia.ch/~juergen/scientific-integrity-turing-award-deep-learning.html</a><br>
<br>
The above is a point-for-point critique of
factual errors in ACM's justification of the
ACM A. M. Turing Award for deep learning and
a critique of the Turing Lecture published
by ACM in July 2021. This work can also be
seen as a short history of deep learning, at
least as far as ACM's errors and the Turing
Lecture are concerned.<br>
<br>
I know that some view this as a
controversial topic. However, it is the very
nature of science to resolve controversies
through facts. Credit assignment is as core
to scientific history as it is to machine
learning. My aim is to ensure that the true
history of our field is preserved for
posterity.<br>
<br>
Thank you all in advance for your help! <br>
<br>
Jürgen Schmidhuber<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal">Gary Cottrell 858-534-6640 FAX:
858-534-7029<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Computer
Science and Engineering 0404<br>
IF USING FEDEX INCLUDE THE FOLLOWING LINE: <br>
CSE Building, Room 4130<br>
University of California San Diego
-<br>
9500 Gilman Drive # 0404<br>
La Jolla, Ca. 92093-0404<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Email: <a href="mailto:gary@ucsd.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">gary@ucsd.edu</a><br>
Home page: <a href="http://www-cse.ucsd.edu/~gary/"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://www-cse.ucsd.edu/~gary/</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt">Schedule: <a
href="http://tinyurl.com/b7gxpwo" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">http://tinyurl.com/b7gxpwo</a></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt"><o:p> </o:p></span></p>
</div>
<p><i><span style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">Listen carefully,</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">Neither the Vedas</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">Nor the Qur'an</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">Will teach you this:</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">Put the bit in its mouth,</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">The saddle on its back,</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">Your foot in the stirrup,</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">And ride your wild
runaway mind</span></i><i><span
style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black"><br>
</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black">All the way to heaven.<o:p></o:p></span></i></p>
<p><i><span style="font-size:14.0pt;font-family:"Book
Antiqua",serif;color:black">-- Kabir</span></i><i><span
style="font-size:11.5pt;font-family:"Book
Antiqua",serif;color:black"><o:p></o:p></span></i></p>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>