<div dir="ltr">Hi Danko,<div>Indeed there are so many top down effects based on what has been learned, which is further evidence that salience is inseparable from recognition.<br><div>However as a note about education, I dont buy into defining especially STEM fields that need to be studied. If what is needed to be known is known, it wouldn't be science. </div><div>My undergraduate degree is in (ahem) Electrical Engineering Computer Science.</div><div>The electrical engineering provided useful tools through control theory to understand and manage the dynamics of top-down feedback. This includes the mathematical tools to determine stability and steady state endpoints of dynamical systems.</div><div><br></div><div>In the recognition model I developed, regulatory feedback networks, each input is regulated by the outputs it activates which subsequently creates a dynamical salience which makes sure inputs are not over or under represented within the network and lets the system evaluate all the inputs together.</div><div><br></div><div>This model inherently displays the signal-to-noise salience phenomena that includes difficulty with similarity and asymmetry seen in humans even if the inputs are not spatial.</div><div>It shows Excitation-Inhibition balance findings in neuroscience (Xue et al 2014) or what I call network-wide-bursting.</div><div>It also does not require the iid rehearsal to learn like feedforward models because it uses the salience-feedback dynamics to determine the best relevance of each input given what has been learned and what is present at recognition time. This dynamic salience is inseparable from the mechanism of recognition. </div><div><br></div><div>Unfortunately I dont seem to be able to convince the academic cognitive psychology and neuroscience communities that while they can model any of their phenomena with enough parameters, models with huge parameter spaces are less desirable. Scalability and minimizing degrees of freedom are important in models: models that show the most amount of phenomena (across multiple disciplines) with the least amount of free parameters are better. </div><div>Neither do I seem able to convince the academic connectionist community that the iid rehearsal requirements are killing the ability to use models in a natural way and can be avoided using feedback back to the inputs during recognition. Nor does there seem to be cross interest within the academic computational communities where this model can show top-down attention effects and controls similar to Bayesian models (with dynamic priors, priming, and weights similar to likelihoods helping with explainability) but is completely connectionist.</div><div>Sincerely,</div><div>-Tsvi</div><div><div>Xue, M, Atallah, BV, Scanziani, M. (2014). Equalizing excitation-inhibition ratios across visual cortical neurons. Nature 511, 596–600<br></div><div><a href="https://www.youtube.com/watch?v=F-GBIZoZ1mI&list=PL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3&index=1" target="_blank">https://www.youtube.com/watch?v=F-GBIZoZ1mI&list=PL4nMP8F3B7bg3cNWWwLG8BX-wER2PeB-3&index=1</a></div></div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 17, 2022 at 12:15 AM Danko Nikolic <<a href="mailto:danko.nikolic@gmail.com" target="_blank">danko.nikolic@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Dear Juyang,</div><div><br></div><div>You wrote "Senior people do not want to get a PhD in all 6 disciplines in the attached figure: biology, neuroscience, psychology, computer science, electrical engineering, mathematics."</div><div><br></div><div>I would cross electrical engineering from that list. It seems to me that the contribution of electrical engineering is minor. But then I would add philosophy of mind and cybernetics. These two seem a lot more important to acquire a PhD-level knowledge in. </div><div><br></div><div>Best,</div><div><br></div><div>Danko</div><br clear="all"><div><div dir="ltr"><div dir="ltr">Dr. Danko Nikolić<br><a href="http://www.danko-nikolic.com" target="_blank">www.danko-nikolic.com</a><br><a href="https://www.linkedin.com/in/danko-nikolic/" target="_blank">https://www.linkedin.com/in/danko-nikolic/</a><div>--- A progress usually starts with an insight ---</div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 17, 2022 at 8:22 AM Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Tsvi:<br><br>You wrote: "I believe scientists not seeing eye-to-eye with each other and other members of the community is in no small part due to these terms."<div><br></div><div>I agree. This is a HUGE problem, as the attached figure "Blind Men and an Elephant" indicates. What should this multidisciplinary community do? Senior people do not want to get a PhD in all 6 disciplines in the attached figure: biology, neuroscience, psychology, computer science, electrical engineering, mathematics.</div><div><br></div><div>Best regards,</div><div>-John</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 15, 2022 at 10:00 PM Tsvi Achler <<a href="mailto:achler@gmail.com" target="_blank">achler@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">After studying the brain from a multidisciplinary perspective I am well aware of the difficulties speaking and understanding each other across disciplines. There are many terms that are defined differently in different fields... and unfortunately things are not as simple as looking them up in a dictionary. <div><br></div><div>For example the term recurrent connections have different meanings in the computational neuroscience, neural networks, and cognitive psychology communities.</div><div>In neural networks recurrent means an output used back as an input within a paradigm of delayed inputs. It is a method of representing time or sequences. Often recurrent connections in neural networks are confused with feedback back to the same inputs which are actually never used in neural networks because it forms an infinite loop and is not possible to rewind in order to generate an error signal.</div><div>In computational neuroscience recurrent connections are used to describe lateral connections.</div><div>In cognitive psychology the term re-entrant connections are used to describe feedback back to the same inputs.</div><div> </div><div>I believe in order to truly appreciate "brain-like" ideas, members of this group need to familiarize themselves with these brain-focused fields. For example in cognitive psychology there is a rich literature on salience (which again is a bit different from salience in the neural network community). Salience is a dynamic process which determines how well a certain input or input feature is processed. Salience changes in the brain depending on what other inputs or features are concurrently present or what the person is instructed to focus on. There is very little appreciation, integration or implementation of these findings in feedforward networks, yet salience plays a factor in every recognition decision and modality including smell and touch.</div><div><br></div><div>Consciousness is a particularly problematic minefield which also adds in philosophy, metaphysics and subjectivity into the mix.</div><div><br></div><div>Juyang, I think we both agree about the basics: the need for more realistic real world recognition and to move beyond the rehearsal limitations of neural networks. I believe scientists not seeing eye-to-eye with each other and other members of the community is in no small part due to these terms.</div><div><br></div><div>Sincerely,</div><div>-Tsvi</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 15, 2022 at 9:54 AM Juyang Weng <<a href="mailto:juyang.weng@gmail.com" target="_blank">juyang.weng@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Dear Tsvi,<br>You wrote "A huge part of the problem in any discussion about consciousness is there isn't even a clear definition of consciousness". <br>Look at the 5 level definition of consciousness:<br><a href="https://www.merriam-webster.com/dictionary/consciousness" target="_blank">https://www.merriam-webster.com/dictionary/consciousness</a><br><br></div><div>You wrote: "So consciousness is not necessary or sufficient for complex thoughts or behavior."<br>I was thinking that way too, until recently. <br>I found consciousness IS REQUIRED for even learning basic intelligence. </div><div>To put it in a short way so that people on this list can benefit:<br>The motors (as context/actions) in the brain require consciousness in order to learn correctly in the physical world. Please read the first model about conscious learning:<br><font color="#000000">J. Weng, "3D-to-2D-to-3D Conscious Learning", in
Proc. IEEE 40th International Conference on Consumer Electronics, pp.
1-6, Las Vegas NV, USA, Jan.7-9, 2022.
<a href="http://www.cse.msu.edu/%7eweng/research/ConsciousLearning-ICCE-2022-rvsd-cite.pdf" target="_blank">PDF file</a>.</font><br> </div><div>Best regards,</div><div>-John</div><div>----<div>From: Tsvi Achler <<a href="mailto:achler@gmail.com" target="_blank">achler@gmail.com</a>><br>To: Iam Palatnik <<a href="mailto:iam.palat@gmail.com" target="_blank">iam.palat@gmail.com</a>><br>Cc: Connectionists <<a href="mailto:connectionists@cs.cmu.edu" target="_blank">connectionists@cs.cmu.edu</a>><br>Subject: Re: Connectionists: Weird beliefs about consciousness<br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">Juyang (John) Weng<br></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">Juyang (John) Weng<br></div></div>
</blockquote></div>
</blockquote></div>