 wrote:
> 
> Dear All,
> 
> I know the discussions are getting sometimes heated, but I want to thank everyone for it. I meant to contribute earlier pointing to an early paper by Amari-sensei where he used backprop without even detailed explanations. I always thought that for him it was trivial as it is just the chain rule. While Amari-sensei is so inspiring and has given us so many more insights through information geometry, there is also a huge role for people who popularize some ideas and bring the rest of us commoners along.
> 
> I specifically enjoyed comments on deep learning versus neurosymbolic causal learning. I am so excited to see more awareness of possible relations that might bring these fields closer together in the future. What is your favorite venue for such discussions?
> 
> Respectfully, Thomas Trappenberg 
> 
> On Sun, Jan 29, 2023, 8:49 a.m. Richard Loosemore  wrote:
> 
> Dear Imad,
> 
> Fair comment, although I heard Jeurgen say much the same thing 14 years ago, at the AGI conference in 2009, so perhaps you can forgive me for being a little weary of this tune...?
> 
> More *substantively* let me say that this field is such that many ideas/algorithms/theories can be SEEN as variations on other ideas/algorithms/theories, if you look at them from just the right angle.
> 
> If I may add a tongue-in-cheek comment.  I got into this field in 1981 (my first supervisor was John G. Taylor).  By the time the big explosion happened in 1985-7, I was already thinking far beyond that paradigm.  When thinking about what thesis to do, to satisfy my Warwick Psych Dept overseers in 1989, I invented, on paper, many of the ideas that later became Deep Learning.  But those struck me as tedious and ultimately irrelevant, because I wanted to understand the whole system, not make pattern association machines.  This is NOT a claim that I invented anything first, but it IS meant to convey the idea that to people like me who come up with novel ideas all the time, but try to stay focussed on what they consider the genuine prize, all this fighting for a place in the history books seems pathetic.
> 
> There, that's my arrogant thought-for-the day.  You can now safely ignore me again.
> 
> Richard Loosemore
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On 1/27/23 3:29 AM, Imad Khan wrote:
>> Dear Richard,
>> I find your comment a bit unwarranted. You could, however, follow Gary Marcus' way to put forward critical thoughts. I do not necessarily agree with Gary, but I agree with his style. I am reproducing Gary's text below for your convenience. Juergan is an elder of AI and deserves respect (like all of us do). I did go to your website and you're correct to say that AI systems are complex systems and an integrated approach is needed to save another 20 years!
>> 
>> Gary's excerpt:
>> 
>>  
>> Regards,
>> Dr. M. Imad Khan
>> 
>> 
>> On Thu, 26 Jan 2023 at 04:41, Richard Loosemore  wrote:
>> 
>> Please, somebody reassure me that this isn't just another attempt to 
>> rewrite history so that Schmidhuber's lab invented almost everything.
>> 
>> Because at first glance, that's what it looks like.
>> 
>> Richard
>> 
> 
> 

From timofte.radu at gmail.com  Wed Feb  1 09:01:27 2023
From: timofte.radu at gmail.com (Radu Timofte)
Date: Wed, 1 Feb 2023 15:01:27 +0100
Subject: Connectionists: [CFP] CVPR 2023 New Trends in Image Restoration and
 Enhancement (NTIRE) workshop and challenges
Message-ID: 

 Apologies for multiple postings
***********************************

CALL FOR PAPERS  & CALL FOR PARTICIPANTS IN 13 CHALLENGES

NTIRE: 8th New Trends in Image Restoration and Enhancement workshop and
challenges.
In conjunction with CVPR 2023, June 18, Vancouver, Canada.

Website: https://cvlai.net/ntire/2023/
Contact: radu.timofte at uni-wuerzburg.de

TOPICS

? Image/video inpainting
? Image/video deblurring
? Image/video denoising
? Image/video upsampling and super-resolution
? Image/video filtering
? Image/video de-hazing, de-raining, de-snowing, etc.
? Demosaicing
? Image/video compression
? Removal of artifacts, shadows, glare and reflections, etc.
? Image/video enhancement: brightening, color adjustment, sharpening, etc.
? Style transfer
? Hyperspectral image restoration, enhancement, manipulation
? Underwater image restoration, enhancement, manipulation
? Light field image restoration, enhancement, manipulation
? Methods robust to changing weather conditions / adverse outdoor
conditions
? Image/video restoration, enhancement, manipulation on constrained
settings/mobile devices
? Visual domain translation
? Multimodal translation
? Perceptual enhancement
? Perceptual manipulation
? Depth estimation
? Image/video generation and hallucination
? Image/video quality assessment
? Image/video semantic segmentation
? Saliency and gaze estimation
? Studies and applications of the above.

SUBMISSION

A paper submission has to be in English, in pdf format, and at most 8 pages
(excluding references) in CVPR style.
https://cvpr2023.thecvf.com/Conferences/2023/AuthorGuidelines

The review process is double blind.
Accepted and presented papers will be published after the conference in the
2023 CVPR Workshops Proceedings.

Author Kit:
https://media.icml.cc/Conferences/CVPR2023/cvpr2023-author_kit-v1_1-1.zip
Submission site: https://cmt3.research.microsoft.com/NTIRE2023

WORKSHOP DATES

? *Regular Papers submission deadline: February 28, 2022*
? Challenge Papers submission deadline: March 28, 2022
? *Papers reviewed elsewhere submission deadline: March 28, 2022*

CHALLENGES

   1. *HR Non-Homogeneous Dehazing*
   2.
*Night Photography Rendering *
   3. *Real-Time Super-Resolution*
   4.
*Bokeh Effect Transformation *
   5. *360? Omnidirectional Super-Resolution *
   6. *Super-Resolution*
   7. *Light Field Super-Resolution*
   8. *Stereo Super-Resolution*
   9. *Quality Assessment for Video Enhancement*
   10. *Shadow Removal*
   11. *Denoising*
   12. *HR Depth Estimation*
   13.
*Colorization *

To learn more about the challenges, to participate in the challenges, and
to access the data everybody is invited to check the NTIRE 2023 web page:
https://cvlai.net/ntire/2023/

For those interested in constrained and efficient solutions validated on
mobile devices we refer to the CVPR23* Mobile AI Workshop and Challenges:*
https://ai-benchmark.com/workshops/mai/2023/

CHALLENGES DATES

? *Release of train data: January 30, 2023*
? Competitions end: March 14, 2022

SPEAKERS (TBA)

SPONSORS (TBA)

Website: https://cvlai.net/ntire/2023/
Contact: radu.timofte at uni-wuerzburg.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From stmanion at gmail.com  Wed Feb  1 11:08:36 2023
From: stmanion at gmail.com (Sean Manion)
Date: Wed, 1 Feb 2023 11:08:36 -0500
Subject: Connectionists: Annotated History of Modern AI and Deep Learning
In-Reply-To: 
References: <8246578A-D869-49FB-8AA6-4014D9EB0239@supsi.ch>
 <138FE707-87DB-4DE0-8C2B-5D41800A55D4@supsi.ch>
 <12f78a08-cf83-e437-60ba-a5f8e377256f@susaro.com>

 <5fad3a61-7ea0-344a-8f7c-57034b23575c@susaro.com>

Message-ID: 

Thank you all for this wonderful discussion and (annotated!) history
lesson. While my current project is focused primarily on the pre-AI history
timeframe of 1936(?)-56, I anticipate the next phase will benefit from
Juergen's work and many of the additional items mentioned here.

I do have an ask of the group:

I am currently going through Alcibiades Malapi-Nelson's insightful *The
Nature of the Machine and the Collapse of Cybernetics *(2017). He
references a Macy meeting on May 13-14, 1942 focused on Hypnosis and
Conditioned Reflexes where Arturo Rosenbleuth was able to present
declassified aspects of Norbert Wiener and Julian Bigelow's anti-aircraft
work on their behalf (which pre-cursored the trio's 1943 paper).

The book's author notes that no written record exists of this 1942 Macy
meeting, which preceded Macy meetings on Cybernetics by a few years. He
references Steve Heims 1991 work on the topic and highlights some details,
e.g. Warren McCulloch was at this earlier meeting, but no original material
about it seems to exist otherwise.

I am planning to go through Heims 1991 book (his 1980 book on Wiener and
Von Neumann is phenomenal). My question is whether anyone here has any
knowledge of or suggestions towards where to find info on the May 1942 Macy
meeting on Hypnosis and Conditioned Reflex beyond that source.

If so, please let me know on the thread or reaching out separately.

Thank you!

Sean

On Wed, Feb 1, 2023, 10:43 AM Schmidhuber Juergen  wrote:

> Thanks, Thomas, but one correction:
>
> In 1967-68, Amari trained deep MLP?s by stochastic gradient descent (SGD)
> [GD1], the method proposed in 1951 by Robbins & Monro [STO51-52], which
> also works for neurons with non-differentiable activation functions (
> https://people.idsia.ch/~juergen/deep-learning-history.html#2nddl)
>
> This was not yet backpropagation, i.e., "reverse mode of automatic
> differentiation,? which was published a few years later in 1970 by Seppo
> Linnainmaa [BP1,4,5]. In 1982, Paul Werbos proposed to use the method to
> train NNs [BP2], extending ideas in his 1974 thesis. In 1960, Henry J.
> Kelley already had a precursor of backpropagation in the field of control
> theory [BPA]. (
> https://people.idsia.ch/~juergen/deep-learning-history.html#backprop)
>
> Juergen
>
> PS: Richard, I cannot reasonably reply to your earlier comments (citations
> needed). Have you found anything in the survey (
> https://arxiv.org/abs/2212.11279) that is factually incorrect? As always,
> I'll be happy to correct it.
>
>
> > On 29. Jan 2023, at 16:26, Thomas Trappenberg  wrote:
> >
> > Dear All,
> >
> > I know the discussions are getting sometimes heated, but I want to thank
> everyone for it. I meant to contribute earlier pointing to an early paper
> by Amari-sensei where he used backprop without even detailed explanations.
> I always thought that for him it was trivial as it is just the chain rule.
> While Amari-sensei is so inspiring and has given us so many more insights
> through information geometry, there is also a huge role for people who
> popularize some ideas and bring the rest of us commoners along.
> >
> > I specifically enjoyed comments on deep learning versus neurosymbolic
> causal learning. I am so excited to see more awareness of possible
> relations that might bring these fields closer together in the future. What
> is your favorite venue for such discussions?
> >
> > Respectfully, Thomas Trappenberg
> >
> > On Sun, Jan 29, 2023, 8:49 a.m. Richard Loosemore 
> wrote:
> >
> > Dear Imad,
> >
> > Fair comment, although I heard Jeurgen say much the same thing 14 years
> ago, at the AGI conference in 2009, so perhaps you can forgive me for being
> a little weary of this tune...?
> >
> > More *substantively* let me say that this field is such that many
> ideas/algorithms/theories can be SEEN as variations on other
> ideas/algorithms/theories, if you look at them from just the right angle.
> >
> > If I may add a tongue-in-cheek comment.  I got into this field in 1981
> (my first supervisor was John G. Taylor).  By the time the big explosion
> happened in 1985-7, I was already thinking far beyond that paradigm.  When
> thinking about what thesis to do, to satisfy my Warwick Psych Dept
> overseers in 1989, I invented, on paper, many of the ideas that later
> became Deep Learning.  But those struck me as tedious and ultimately
> irrelevant, because I wanted to understand the whole system, not make
> pattern association machines.  This is NOT a claim that I invented anything
> first, but it IS meant to convey the idea that to people like me who come
> up with novel ideas all the time, but try to stay focussed on what they
> consider the genuine prize, all this fighting for a place in the history
> books seems pathetic.
> >
> > There, that's my arrogant thought-for-the day.  You can now safely
> ignore me again.
> >
> > Richard Loosemore
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On 1/27/23 3:29 AM, Imad Khan wrote:
> >> Dear Richard,
> >> I find your comment a bit unwarranted. You could, however, follow Gary
> Marcus' way to put forward critical thoughts. I do not necessarily agree
> with Gary, but I agree with his style. I am reproducing Gary's text below
> for your convenience. Juergan is an elder of AI and deserves respect (like
> all of us do). I did go to your website and you're correct to say that AI
> systems are complex systems and an integrated approach is needed to save
> another 20 years!
> >>
> >> Gary's excerpt:
> >>
> >>
> >> Regards,
> >> Dr. M. Imad Khan
> >>
> >>
> >> On Thu, 26 Jan 2023 at 04:41, Richard Loosemore 
> wrote:
> >>
> >> Please, somebody reassure me that this isn't just another attempt to
> >> rewrite history so that Schmidhuber's lab invented almost everything.
> >>
> >> Because at first glance, that's what it looks like.
> >>
> >> Richard
> >>
> >
> > 
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From danko.nikolic at gmail.com  Wed Feb  1 12:24:15 2023
From: danko.nikolic at gmail.com (Danko Nikolic)
Date: Wed, 1 Feb 2023 18:24:15 +0100
Subject: Connectionists: Transient subnetwork selection: a new paradigm
 to replace connectionism? A new paradigm is not needed
In-Reply-To: 
References: 

Message-ID: 

Hi Steve,

You wrote: "Our work modeling mGluR does include G proteins.". Correct. But
this is not what I asked you about. What is relevant in my work is G
protein gated ion channels. I cannot find them in your paper.

This image is from your paper. There are no G protein-gated ion channels in
this drawing. Right?

[image: image.png]
Or did I miss something?

Danko

Dr. Danko Nikoli?
www.danko-nikolic.com
https://www.linkedin.com/in/danko-nikolic/
-- I wonder, how is the brain able to generate insight? --

On Wed, Feb 1, 2023 at 6:10 PM Grossberg, Stephen  wrote:

> Dear Danko,
>
> Thanks for your speedy reply.
>
> *Our work modeling mGluR does include G proteins*.
>
> In particular, my first article with John Fiala provides a rather complete
> description of mGluR biochemistry.
>
> See Figure 6, for example, in:
>
> Fiala, J.C., Grossberg, S., and Bullock, D. (1996). Metabotropic glutamate
> receptor activation in cerebellar Purkinje cells
> as substrate for adaptive timing of the classically conditioned eye blink
> response. *Journal of Neuroscience*, *16*, 3760-3774.
> https://sites.bu.edu/steveg/files/2016/06/FiaGroBul1996JouNeuroscience.pdf
>
> All of our later articles build on this foundation.
>
> I also want to emphasize that we developed *neural system models* that
> include such receptors.
>
> We simulated these models to show how their emergent properties
> quantitatively fit
> the different kinds of behaviors that these systems regulate.
>
> This *link between brain and behavior *provides an important test of the
> sufficiency of model hypotheses.
>
> I did not see such a link in your article. If it was there, please let me
> know where I should look.
>
> Thanks, and
>
> Best,
>
> Steve
>
>
> ------------------------------
> *From:* Danko Nikolic 
> *Sent:* Wednesday, February 1, 2023 11:47 AM
> *To:* Grossberg, Stephen 
> *Cc:* Post Connectionists 
> *Subject:* Re: Connectionists: Transient subnetwork selection: a new
> paradigm to replace connectionism?
>
> Dear Steve,
>
> Thank you for your email. I have a hard time seeing the relevance of those
> papers for my present work. There are thousands of papers on metabotropic
> receptors. However, here I put emphasis on their work in combination with G
> protein-gated ion channels. Do you have some papers about this combination:
> MRs and GPGICs working together to ensure transient subnetwork selection?
>
> If you do and if they fit into the paradigm, I will of course gladly cite
> them in my next paper.
>
> Thanks,
>
> Danko
>
> Dr. Danko Nikoli?
> www.danko-nikolic.com
> https://www.linkedin.com/in/danko-nikolic/
> -- I wonder, how is the brain able to generate insight? --
>
>
> On Wed, Feb 1, 2023 at 5:24 PM Grossberg, Stephen  wrote:
>
> Dear Danko,
>
> Thanks very much for sharing your latest article with the connectionists
> list.
>
> In it, you kindly mention my own work in several places. Thanks very much!
>
> However, the statements that you make about it are not correct.
>
> My Magnum Opus
>
> *Conscious Mind, Resonant Brain: How Each Brain Makes a Mind*
>
> https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552
>
> provides a self-contained and non-technical overview of many aspects of my
> work in which these problems
> do not occur. I can provide examples if you would like, but will here
> focus on just one topic:
>
> Your email mentions the importance of metabotropic receptors and suggest
> that they represent "a new paradigm as an alternative to connectionism...we
> think with those proteins".
>
> *Metabotropic glutamate receptors (mGluR)* have played an important role
> in our work about how our brains learn since our
> *Journal of Neuroscience* article in 1996 on this topic:
>
> Fiala, J.C., Grossberg, S., and Bullock, D. (1996). Metabotropic glutamate
> receptor activation in cerebellar Purkinje cells
> as substrate for adaptive timing of the classically conditioned eye blink
> response. *Journal of Neuroscience*, *16*, 3760-3774.
> https://sites.bu.edu/steveg/files/2016/06/FiaGroBul1996JouNeuroscience.pdf
>
> Later work modeled how they play a role in many other brain regions to
> regulate multiple types of behaviors. For example:
>
> Grossberg, S. and Merrill, J.W.L. (1992). A neural network model of
> adaptively timed reinforcement learning and hippocampal dynamics.
> *Cognitive Brain Research*, *1*, 3-38.
> https://sites.bu.edu/steveg/files/2016/06/GroMer1992CogRes.pdf
>
> Grossberg, S. and Merrill, J.W.L. (l996). The hippocampus and cerebellum
> in adaptively timed learning, recognition, and movement.
> *Journal of Cognitive Neuroscience,* *8*, 257-277.
> https://sites.bu.edu/steveg/files/2016/06/GroMer96.pdf
>
> Brown, J., Bullock, D., and Grossberg, S. (1999). How the basal ganglia
> use parallel excitatory and inhibitory learning pathways
> to selectively respond to unexpected rewarding cues. *Journal of
> Neuroscience,* *19*, 10502-10511.
> https://sites.bu.edu/steveg/files/2016/06/BroBulGro99.pdf
>
> Grossberg, S., and Vladusich, T. (2010). How do children learn to follow
> gaze, share joint attention, imitate their teachers, and use tools
> during social interactions? *Neural Networks*, *23*, 940-965.
> https://sites.bu.edu/steveg/files/2016/06/GrossbergVladusichNN2010.pdf
>
> Franklin, D. J., and Grossberg, S. (2017). A neural model of normal and
> abnormal learning and memory consolidation:
> Adaptively timed conditioning, hippocampus, amnesia, neurotrophins, and
> consciousness.
> *Cognitive, Affective, and Behavioral Neuroscience*, 17, 24-76.
> https://link.springer.com/article/10.3758/s13415-016-0463-y
>
> I hope that you find these references useful in your own work, and welcome
> any comments or questions that you may have about
> these results.
>
> Best wishes,
>
> Steve
>
>
> Stephen Grossberg
> sites.bu.edu/steveg
>
>
>
> ------------------------------
> *From:* Connectionists  on
> behalf of Danko Nikolic 
> *Sent:* Tuesday, January 31, 2023 12:43 PM
> *To:* Post Connectionists 
> *Subject:* Connectionists: Transient subnetwork selection: a new paradigm
> to replace connectionism?
>
> Dear all,
>
> I am happy to announce that my paper, the draft of which has been
> discussed on this list, has yesterday finally been published after a
> peer review.
>
> This is probably the most important paper I have done in my career so far.
>
> To remind you, the paper proposes a new paradigm as an alternative to
> connectionism. To understand the mind, synapses are not so important any
> more. Instead, critical are some other types of proteins on the neural
> membrane. These proteins have the capability to transiently select
> subnetworks that will be functional in the next few seconds or minutes. The
> paradigm proposes that cognition emerges from those transient subnetwork
> selections (and not from network computations of the classical, the
> so-called connectionist paradigm). The proteins in question are
> metabotropic receptors and G protein-gated ion channels. Simply put, we
> think with those proteins. A result of a thought is a new state of network
> pathways, not the activity of neurons.
>
> I would like to thank the list for many of the comments that I received
> and that helped me improve the manuscript. For example, very useful was the
> information on the learning algorithms able to learn the n-bit parity
> problem (aka, generalized XOR), which I used to illustrate the scaling
> problems of deep learning. This made my supplementary materials much
> better.
>
> The paper can be downloaded without a paywall for 50 days, here:
> https://authors.elsevier.com/a/1gVvg5Fq7aXeir
>
> The new version of the paper is much better than the original draft. It
> has more information, clearer explanations and improved structure.
>
> I hope the paper inspires people to investigate possibilities beyond
> connectionism both for understanding the brain and for building AI.
>
> For myself, I would love to build an AI based on these principles.
>
> Thanks a lot.
>
> Danko
>
>
> Dr. Danko Nikoli?
> www.danko-nikolic.com
> https://www.linkedin.com/in/danko-nikolic/
> -- I wonder, how is the brain able to generate insight? --
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 852547 bytes
Desc: not available
URL: 

From steve at bu.edu  Wed Feb  1 11:24:43 2023
From: steve at bu.edu (Grossberg, Stephen)
Date: Wed, 1 Feb 2023 16:24:43 +0000
Subject: Connectionists: Transient subnetwork selection: a new paradigm
 to replace connectionism?
In-Reply-To: 
References: 
Message-ID: 

Dear Danko,

Thanks very much for sharing your latest article with the connectionists list.

In it, you kindly mention my own work in several places. Thanks very much!

However, the statements that you make about it are not correct.

My Magnum Opus

Conscious Mind, Resonant Brain: How Each Brain Makes a Mind

https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552

provides a self-contained and non-technical overview of many aspects of my work in which these problems
do not occur. I can provide examples if you would like, but will here focus on just one topic:

Your email mentions the importance of metabotropic receptors and suggest that they represent "a new paradigm as an alternative to connectionism...we think with those proteins".

Metabotropic glutamate receptors (mGluR) have played an important role in our work about how our brains learn since our
Journal of Neuroscience article in 1996 on this topic:

Fiala, J.C., Grossberg, S., and Bullock, D. (1996). Metabotropic glutamate receptor activation in cerebellar Purkinje cells
as substrate for adaptive timing of the classically conditioned eye blink response. Journal of Neuroscience, 16, 3760-3774.
https://sites.bu.edu/steveg/files/2016/06/FiaGroBul1996JouNeuroscience.pdf

Later work modeled how they play a role in many other brain regions to regulate multiple types of behaviors. For example:

Grossberg, S. and Merrill, J.W.L. (1992). A neural network model of adaptively timed reinforcement learning and hippocampal dynamics.
Cognitive Brain Research, 1, 3-38.
https://sites.bu.edu/steveg/files/2016/06/GroMer1992CogRes.pdf

Grossberg, S. and Merrill, J.W.L. (l996). The hippocampus and cerebellum in adaptively timed learning, recognition, and movement.
Journal of Cognitive Neuroscience, 8, 257-277.
https://sites.bu.edu/steveg/files/2016/06/GroMer96.pdf

Brown, J., Bullock, D., and Grossberg, S. (1999). How the basal ganglia use parallel excitatory and inhibitory learning pathways
to selectively respond to unexpected rewarding cues. Journal of Neuroscience, 19, 10502-10511.
https://sites.bu.edu/steveg/files/2016/06/BroBulGro99.pdf

Grossberg, S., and Vladusich, T. (2010). How do children learn to follow gaze, share joint attention, imitate their teachers, and use tools
during social interactions? Neural Networks, 23, 940-965.
https://sites.bu.edu/steveg/files/2016/06/GrossbergVladusichNN2010.pdf

Franklin, D. J., and Grossberg, S. (2017). A neural model of normal and abnormal learning and memory consolidation:
Adaptively timed conditioning, hippocampus, amnesia, neurotrophins, and consciousness.
Cognitive, Affective, and Behavioral Neuroscience, 17, 24-76.
https://link.springer.com/article/10.3758/s13415-016-0463-y

I hope that you find these references useful in your own work, and welcome any comments or questions that you may have about
these results.

Best wishes,

Steve

Stephen Grossberg
sites.bu.edu/steveg

________________________________
From: Connectionists  on behalf of Danko Nikolic 
Sent: Tuesday, January 31, 2023 12:43 PM
To: Post Connectionists 
Subject: Connectionists: Transient subnetwork selection: a new paradigm to replace connectionism?

Dear all,

I am happy to announce that my paper, the draft of which has been discussed on this list, has yesterday finally been published after a peer review.

This is probably the most important paper I have done in my career so far.

To remind you, the paper proposes a new paradigm as an alternative to connectionism. To understand the mind, synapses are not so important any more. Instead, critical are some other types of proteins on the neural membrane. These proteins have the capability to transiently select subnetworks that will be functional in the next few seconds or minutes. The paradigm proposes that cognition emerges from those transient subnetwork selections (and not from network computations of the classical, the so-called connectionist paradigm). The proteins in question are metabotropic receptors and G protein-gated ion channels. Simply put, we think with those proteins. A result of a thought is a new state of network pathways, not the activity of neurons.

I would like to thank the list for many of the comments that I received and that helped me improve the manuscript. For example, very useful was the information on the learning algorithms able to learn the n-bit parity problem (aka, generalized XOR), which I used to illustrate the scaling problems of deep learning. This made my supplementary materials much better.

The paper can be downloaded without a paywall for 50 days, here:
https://authors.elsevier.com/a/1gVvg5Fq7aXeir

The new version of the paper is much better than the original draft. It has more information, clearer explanations and improved structure.

I hope the paper inspires people to investigate possibilities beyond connectionism both for understanding the brain and for building AI.

For myself, I would love to build an AI based on these principles.

Thanks a lot.

Danko

Dr. Danko Nikoli?
www.danko-nikolic.com
https://www.linkedin.com/in/danko-nikolic/
-- I wonder, how is the brain able to generate insight? --
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From danko.nikolic at gmail.com  Wed Feb  1 11:47:17 2023
From: danko.nikolic at gmail.com (Danko Nikolic)
Date: Wed, 1 Feb 2023 17:47:17 +0100
Subject: Connectionists: Transient subnetwork selection: a new paradigm
 to replace connectionism?
In-Reply-To: 
References: 

Message-ID: 

Dear Steve,

Thank you for your email. I have a hard time seeing the relevance of those
papers for my present work. There are thousands of papers on metabotropic
receptors. However, here I put emphasis on their work in combination with G
protein-gated ion channels. Do you have some papers about this combination:
MRs and GPGICs working together to ensure transient subnetwork selection?

If you do and if they fit into the paradigm, I will of course gladly cite
them in my next paper.

Thanks,

Danko

Dr. Danko Nikoli?
www.danko-nikolic.com
https://www.linkedin.com/in/danko-nikolic/
-- I wonder, how is the brain able to generate insight? --

On Wed, Feb 1, 2023 at 5:24 PM Grossberg, Stephen  wrote:

> Dear Danko,
>
> Thanks very much for sharing your latest article with the connectionists
> list.
>
> In it, you kindly mention my own work in several places. Thanks very much!
>
> However, the statements that you make about it are not correct.
>
> My Magnum Opus
>
> *Conscious Mind, Resonant Brain: How Each Brain Makes a Mind*
>
> https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552
>
> provides a self-contained and non-technical overview of many aspects of my
> work in which these problems
> do not occur. I can provide examples if you would like, but will here
> focus on just one topic:
>
> Your email mentions the importance of metabotropic receptors and suggest
> that they represent "a new paradigm as an alternative to connectionism...we
> think with those proteins".
>
> *Metabotropic glutamate receptors (mGluR)* have played an important role
> in our work about how our brains learn since our
> *Journal of Neuroscience* article in 1996 on this topic:
>
> Fiala, J.C., Grossberg, S., and Bullock, D. (1996). Metabotropic glutamate
> receptor activation in cerebellar Purkinje cells
> as substrate for adaptive timing of the classically conditioned eye blink
> response. *Journal of Neuroscience*, *16*, 3760-3774.
> https://sites.bu.edu/steveg/files/2016/06/FiaGroBul1996JouNeuroscience.pdf
>
> Later work modeled how they play a role in many other brain regions to
> regulate multiple types of behaviors. For example:
>
> Grossberg, S. and Merrill, J.W.L. (1992). A neural network model of
> adaptively timed reinforcement learning and hippocampal dynamics.
> *Cognitive Brain Research*, *1*, 3-38.
> https://sites.bu.edu/steveg/files/2016/06/GroMer1992CogRes.pdf
>
> Grossberg, S. and Merrill, J.W.L. (l996). The hippocampus and cerebellum
> in adaptively timed learning, recognition, and movement.
> *Journal of Cognitive Neuroscience,* *8*, 257-277.
> https://sites.bu.edu/steveg/files/2016/06/GroMer96.pdf
>
> Brown, J., Bullock, D., and Grossberg, S. (1999). How the basal ganglia
> use parallel excitatory and inhibitory learning pathways
> to selectively respond to unexpected rewarding cues. *Journal of
> Neuroscience,* *19*, 10502-10511.
> https://sites.bu.edu/steveg/files/2016/06/BroBulGro99.pdf
>
> Grossberg, S., and Vladusich, T. (2010). How do children learn to follow
> gaze, share joint attention, imitate their teachers, and use tools
> during social interactions? *Neural Networks*, *23*, 940-965.
> https://sites.bu.edu/steveg/files/2016/06/GrossbergVladusichNN2010.pdf
>
> Franklin, D. J., and Grossberg, S. (2017). A neural model of normal and
> abnormal learning and memory consolidation:
> Adaptively timed conditioning, hippocampus, amnesia, neurotrophins, and
> consciousness.
> *Cognitive, Affective, and Behavioral Neuroscience*, 17, 24-76.
> https://link.springer.com/article/10.3758/s13415-016-0463-y
>
> I hope that you find these references useful in your own work, and welcome
> any comments or questions that you may have about
> these results.
>
> Best wishes,
>
> Steve
>
>
> Stephen Grossberg
> sites.bu.edu/steveg
>
>
>
> ------------------------------
> *From:* Connectionists  on
> behalf of Danko Nikolic 
> *Sent:* Tuesday, January 31, 2023 12:43 PM
> *To:* Post Connectionists 
> *Subject:* Connectionists: Transient subnetwork selection: a new paradigm
> to replace connectionism?
>
> Dear all,
>
> I am happy to announce that my paper, the draft of which has been
> discussed on this list, has yesterday finally been published after a
> peer review.
>
> This is probably the most important paper I have done in my career so far.
>
> To remind you, the paper proposes a new paradigm as an alternative to
> connectionism. To understand the mind, synapses are not so important any
> more. Instead, critical are some other types of proteins on the neural
> membrane. These proteins have the capability to transiently select
> subnetworks that will be functional in the next few seconds or minutes. The
> paradigm proposes that cognition emerges from those transient subnetwork
> selections (and not from network computations of the classical, the
> so-called connectionist paradigm). The proteins in question are
> metabotropic receptors and G protein-gated ion channels. Simply put, we
> think with those proteins. A result of a thought is a new state of network
> pathways, not the activity of neurons.
>
> I would like to thank the list for many of the comments that I received
> and that helped me improve the manuscript. For example, very useful was the
> information on the learning algorithms able to learn the n-bit parity
> problem (aka, generalized XOR), which I used to illustrate the scaling
> problems of deep learning. This made my supplementary materials much
> better.
>
> The paper can be downloaded without a paywall for 50 days, here:
> https://authors.elsevier.com/a/1gVvg5Fq7aXeir
>
> The new version of the paper is much better than the original draft. It
> has more information, clearer explanations and improved structure.
>
> I hope the paper inspires people to investigate possibilities beyond
> connectionism both for understanding the brain and for building AI.
>
> For myself, I would love to build an AI based on these principles.
>
> Thanks a lot.
>
> Danko
>
>
> Dr. Danko Nikoli?
> www.danko-nikolic.com
> https://www.linkedin.com/in/danko-nikolic/
> -- I wonder, how is the brain able to generate insight? --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From avellido at cs.upc.edu  Wed Feb  1 11:50:55 2023
From: avellido at cs.upc.edu (Alfredo Vellido)
Date: Wed, 1 Feb 2023 17:50:55 +0100
Subject: Connectionists: DEADLINE EXTENSION [CFP] Special Session IJCNN 2023
 - The Coming of Age of Explainable AI (XAI) and ML
Message-ID: <42f09c83-dffc-1326-12cd-9549d2b49d7b@cs.upc.edu>

Apologies for cross-posting
===================
LAST CALL FOR PAPERS: DEADLINE EXTENDED:*7th of February*
===================

*IEEE IJCNN 2023 Special Session on**The Coming of Age of Explainable AI 
(XAI) and Machine Learning**
*
June 18-23, 2023, Queensland, Australia
www.cs.upc.edu/~avellido/research/conferences/IJCNN2023-XAIcomingofage.html
https://2023.ijcnn.org/authors/paper-submission

Aims & Scope
------------
Much of current research on Machine Learning (ML) is dominated by 
methods of the Deep Learning family. The more complex their 
architectures, the more difficult the interpretation or explanation of 
how and why a particular network prediction is obtained, or the 
elucidation of which components of the complex system contributed 
essentially to the obtained decision. This brings about the concern 
about interpretability and non-transparency of complex models, 
especially in high-stakes applications areas such as healthcare, 
national security, industry or public governance, to name a few, in 
which decision making processes may affect citizens. This is, for 
instance, made especially relevant by rapid developments in the field of 
autonomous systems ? from cars that drive themselves to partner robots 
and robotic drones. DARPA (Defense Advanced Research Projects Agency), a 
research agency of US Department of Defense, was the first to start a 
research program on Explainable AI 
(https://www.darpa.mil/program/explainable-artificial-intelligence) with 
the goal ?to create a suite of machine learning techniques that (1) 
Produce more explainable models, while maintaining a high level of 
learning performance (prediction accuracy); and (2) Enable human users 
to understand, appropriately trust, and effectively manage the emerging 
generation of artificially intelligent partners.? Research on 
Explainable AI (XAI) is now supported worldwide by a variety of public 
institutions and legal regulations, such as European Union?s General 
Data Protection Regulation (GDPR) and the forthcoming Artificial 
Intelligence Act. Similar concerns about transparency and 
interpretability are being raised by governments and organizations 
worldwide. The lack of transparency (interpretability and 
explainability) of many ML approaches in the light of regulations may 
end up limiting ML to niche applications and poses a significant risk of 
costly mistakes without the mitigation of a sound understanding about 
the flow of information in the model.
For this special session, we invite papers that address many of the 
challenges of XAI in the context of ML models and algorithms. We are 
interested in papers on efficient and innovative algorithmic approaches 
to XAI and their actual applications all areas. This
session also aims to explore the performance-versus-explanation 
trade-off space for high-stakes applications of ML in light of all types 
of AI regulation. ?Comprehensive survey papers on existing technologies 
for XAI are also welcome.
We aim to bring together researchers from different fields to discuss 
key issues related to the research and applications of XAI methods and 
to share their experiences of solving problems in high-stakes 
applications in all domains.
Topics that are of interest to this session include but are not limited to:

 ?New neural network architectures and algorithms for XAI
 ?Interpretability by design
 ?Rule extraction algorithms for deep neural networks
 ?Augmentations of AI methods to increase interpretability and transparency
 ?Innovative applications of XAI
 ?Verification of AI performance
 ?Regulation-compliant XAI methods
 ?Explanation-generation methods for high-stakes applications
 ?Stakeholder-specific XAI methods for high-stakes applications
 ?XAI methods auditing in specific domains
 ?Human-in-the-loop ML: bridging the gap between data scientists and 
end-users
 ?XAI through Data Visualization
 ?Interpretable ML pipelines
 ?Query Interfaces for DL
 ?Active and Transfer learning with transparency
 ?Relevance and Metric Learning
 ?Deep Neural Reasoning
 ?Interfaces with Rule-Based Reasoning, Fuzzy Logic and Natural Language 
Processing

Important Dates
--------------------------
Paper submission:*February 7, 2023 *
Paper decision notification: March 31, 2023

Session Chairs
------------------------
Qi Chen, Victoria University of Wellington, New Zealand
Jos? M Ju?rez, Universidad de Murcia, Spain
Paulo Lisboa, Liverpool John Moores University, U.K.
Asim Roy, Arizona State University, U.S.A.
Alfredo Vellido, Universitat Polit?cnica de Catalunya, Spain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From steve at bu.edu  Wed Feb  1 12:10:08 2023
From: steve at bu.edu (Grossberg, Stephen)
Date: Wed, 1 Feb 2023 17:10:08 +0000
Subject: Connectionists: Transient subnetwork selection: a new paradigm
 to replace connectionism? A new paradigm is not needed
In-Reply-To: 
References: 

Message-ID: 

Dear Danko,

Thanks for your speedy reply.

Our work modeling mGluR does include G proteins.

In particular, my first article with John Fiala provides a rather complete description of mGluR biochemistry.

See Figure 6, for example, in:

Fiala, J.C., Grossberg, S., and Bullock, D. (1996). Metabotropic glutamate receptor activation in cerebellar Purkinje cells
as substrate for adaptive timing of the classically conditioned eye blink response. Journal of Neuroscience, 16, 3760-3774.
https://sites.bu.edu/steveg/files/2016/06/FiaGroBul1996JouNeuroscience.pdf

All of our later articles build on this foundation.

I also want to emphasize that we developed neural system models that include such receptors.

We simulated these models to show how their emergent properties quantitatively fit
the different kinds of behaviors that these systems regulate.

This link between brain and behavior provides an important test of the sufficiency of model hypotheses.

I did not see such a link in your article. If it was there, please let me know where I should look.

Thanks, and

Best,

Steve

________________________________
From: Danko Nikolic 
Sent: Wednesday, February 1, 2023 11:47 AM
To: Grossberg, Stephen 
Cc: Post Connectionists 
Subject: Re: Connectionists: Transient subnetwork selection: a new paradigm to replace connectionism?

Dear Steve,

Thank you for your email. I have a hard time seeing the relevance of those papers for my present work. There are thousands of papers on metabotropic receptors. However, here I put emphasis on their work in combination with G protein-gated ion channels. Do you have some papers about this combination: MRs and GPGICs working together to ensure transient subnetwork selection?

If you do and if they fit into the paradigm, I will of course gladly cite them in my next paper.

Thanks,

Danko

Dr. Danko Nikoli?
www.danko-nikolic.com
https://www.linkedin.com/in/danko-nikolic/
-- I wonder, how is the brain able to generate insight? --

On Wed, Feb 1, 2023 at 5:24 PM Grossberg, Stephen > wrote:
Dear Danko,

Thanks very much for sharing your latest article with the connectionists list.

In it, you kindly mention my own work in several places. Thanks very much!

However, the statements that you make about it are not correct.

My Magnum Opus

Conscious Mind, Resonant Brain: How Each Brain Makes a Mind

https://www.amazon.com/Conscious-Mind-Resonant-Brain-Makes/dp/0190070552