Connectionists: AI at 50 videos...

Thu Aug 27 16:47:04 EDT 2026

Stephen,

You raise important issues. There are many ways of dealing with them. Here’s one of the ways. The image below shows what DARPA wanted for Explainable AI. Essentially, find the parts of an object for its prediction. So, for a cat, verify some of its unique parts like the fur, whiskers, and claws. Thus, in our explainable models, we “teach” an object detection model about these parts. What we get as outputs, for a cat, are predictions for the cat and its parts. There’s a layer of logic that uses these outputs and verifies the existence of the parts before “finally” predicting that there’s a cat in the image. In essence, it’s neuro-symbolic. One can do the symbolic part with a graphical model too. We can also predict that there’s a cat even if we don’t “see” all of the parts, such as when we just see its face and not the rest of the body. And, in a similar way, your “morning coffee” concept can be built from its parts that you mention.

There’s a long tradition in computer vision of finding parts of objects inside the model. Our initial conception was that the higher-level filters in a CNN correspond to certain abstractions such as a nose, eyes, ears for a human. That was the distributed representation but based on lower-level abstractions (parts of objects). But years of research showed that one could not find these kinds of lower-level abstractions in standard CNN models. So, what do you do? Well, one way is to force some of the filters to correspond to these parts. There’s a long history to this body of work in computer vision, including that of Hinton, to create abstractions within a model. Both ways, we are “teaching” the models about “low-level abstraction.” In a way, in both ways, it’s a hierarchical system. In both ways, we create a neuro-symbolic system.

By the way, abstractions are generalizations, they don’t code specific episodic memories. Your “table,” “coffee,” “cup” and all that are abstract notions.

Asim

Asim Roy

Professor, Information Systems

Arizona State University

Asim Roy | iSearch (asu.edu)<https://isearch.asu.edu/profile/9973>

[A screenshot of a computer  AI-generated content may be incorrect.]

From: Stephen José Hanson <jose at rubic.rutgers.edu>
Sent: Thursday, August 27, 2026 4:26 AM
To: Asim Roy <ASIM.ROY at asu.edu>; Rothganger, Fred <frothga at sandia.gov>; Stevan Harnad <harnad at ecs.soton.ac.uk>
Cc: connectionists at mailman.srv.cs.cmu.edu
Subject: Re: Connectionists: AI at 50 videos...

Asim,

I sit here with my coffee neuron and my cup neuron creating a cup of coffee, and my table neuron holding the cup and coffee neurons representing a cup of coffee.  On the table, on the floor, noting of course the cup of coffee is not on the floor but on the table.   So this is a network of coffee, cups, tables, maybe coded by another neuron in a hierarchical network called morning coffee?    Well this is the typical problem with localist model of the brain.  How would it work exactly?   So far just representing a single episodic event requires hierarchical network of neurons that generalize to what exactly?    Do we need many such networks to represent all episodic memories of having a cup of coffee?

This is similar to the problem with the socalled "fusiform face area" which with some proper controls doesn't really exist (cf Hanson 2022).    What would a small blueberry are in fusiform gyrus be doing with all those faces we recognize?   A brain face cloud server?  Other accounts show that FG, LO, IN and PFC are at least involved in a network of areas (not neurons) in recognizing a face.   But computationally what's the actual underlying functions?

The shift to "neurons represent symbols' is not helpful to explain meaning.

 Remember, we have billions of neurons and 100 trillions connections between them.

I'm not saying symbols aren't important and Harnad's Grounding arguments are becoming more critical (and need to be made more explicit)  as LLMs start to be embedded in robots.     Therefore an arguably simpler task, is to identify "symbols" in LLMs since we have no idea how they work either. Are symbols  in their "neurons"?

Cheers

Stephen
On 8/24/26 22:56, Asim Roy wrote:

  1.  Plate (2002): “Another equivalent property is that in a distributed representation one cannot interpret the meaning of activity on a single neuron in isolation: the meaning of activity on any particular neuron is dependent on the activity in other neurons (Thorpe, 1995).”
  2.  Thorpe (1995, p. 550): “With a local representation, activity in individual units can be interpreted directly … with distributed coding individual units cannot be interpreted without knowing the state of other units in the network.”
  3.  Elman (1995, p. 210): “These representations are distributed, which typically has the consequence that interpretable information cannot be obtained by examining activity of single hidden units.”

1. Elman J. (1995). Language as a dynamical system, in Mind as Motion: Explorations in the Dynamics of Cognition, eds Port R., van Gelder T. (Cambridge, MA: MIT Press), 195–223.
2. PlateT. (2002). Distributed representations, in: Encyclopedia of Cognitive Science, ed Nadel L. (London: Macmillan), 2.
3. ThorpeS. (1995). Localized versus distributed representations, in The Handbook of Brain Theory and Neural Networks, ed Arbib M. (Cambridge, MA: MIT Press), 550.

In that discussion with Horace Barlow and others, interpretation and meaning of the activations that responded only to certain specific stimuli was the main source of discomfort to connectionists including Walter Freeman. Hence Walter asking Rodrigo directly whether those activations had meaning and interpretation.

Asim

Asim Roy

Professor, Information Systems

Arizona State University

Lifeboat Foundation Bios: Professor Asim Roy<https://urldefense.com/v3/__https:/lifeboat.com/ex/bios.asim.roy__;!!IKRxdwAv5BmarQ!NHg1P4byXZZmMFLMgr8A4dQNxSlqyvsqWef4wrPXtCXR0f0bo0eMm-rIF8G6ZV0$>

Asim Roy | iSearch (asu.edu)<https://isearch.asu.edu/profile/9973>

From: Rothganger, Fred <frothga at sandia.gov><mailto:frothga at sandia.gov>
Sent: Monday, August 24, 2026 8:46 AM
To: Asim Roy <ASIM.ROY at asu.edu><mailto:ASIM.ROY at asu.edu>
Cc: connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>
Subject: AI at 50 videos...

Asim,

It makes sense for you to put "meaning" in scare-quotes, since it is another ill-defined term. IMHO, the important thing is adaptive behavior. That a particular neuron fires when the organism is exposed to a stimulus is significant because that spike does work on other parts of the system, ultimately shaping behavior. The notion of "meaning" may just be a convenience for us as observers discussing the system.

-- Fred

________________________________
From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu<mailto:connectionists-bounces at mailman.srv.cs.cmu.edu>> on behalf of Asim Roy <ASIM.ROY at asu.edu<mailto:ASIM.ROY at asu.edu>>
Sent: Friday, August 21, 2026 10:47 PM
To: KENTRIDGE, ROBERT W. <robert.kentridge at durham.ac.uk<mailto:robert.kentridge at durham.ac.uk>>; Gary Marcus <gary.marcus at nyu.edu<mailto:gary.marcus at nyu.edu>>; Stephen José Hanson <jose at rubic.rutgers.edu<mailto:jose at rubic.rutgers.edu>>; Hava Siegelmann (hava.siegelmann at gmail.com<mailto:hava.siegelmann at gmail.com>) <hava.siegelmann at gmail.com<mailto:hava.siegelmann at gmail.com>>; Juergen Schmidhuber <juergen.schmidhuber at kaust.edu.sa<mailto:juergen.schmidhuber at kaust.edu.sa>>; Ali Minai <minaiaa at gmail.com<mailto:minaiaa at gmail.com>>
Cc: director at inc.ucsd.edu<mailto:director at inc.ucsd.edu> <director at inc.ucsd.edu<mailto:director at inc.ucsd.edu>>; connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu> <connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>>
Subject: [EXTERNAL] Re: Connectionists: FW: AI at 50 videos...

By the way, most of these microelectrode-based studies, starting with Horace Barlow, are single cell studies. And the single cell studies are the ones that have won Noble prizes because of the insights they produced. The overall evidence from these studies is not just about grandmother cells, but about single cells encoding abstractions and “meaning.” Reddy and Thorpe (2014) conclude:

“In conclusion, evidence is accumulating that “concept cells” carry high-level, abstract stimulus information.”

Single cells having “meaning” was (and still is) at the heart of discussion at that time because it contradicts the fundamental population coding idea where single neurons by themselves have no meaning, they only have meaning collectively. And Walter Freeman at that time could not accept the idea that single cells have meaning. We were supposed to have a public debate about this at IJCNN 2011 in San Jose. Walter posed the question directly to Rodrigo Quian Quiroga, who conducted many of these experiments under the supervision of Itzhak Fried and Christof Koch. Walter also knew Rodrigo, having been co-authors or co-editors of a book. Rodrigo came through at the last minute before the public debate, acknowledging that the single cells in his studies indeed had “meaning.”

“Meaning” and abstraction is at the heart of these debates, not grandmother cells. And finding “meaning” in activations of single cells directly contradicts the population coding hypothesis and supports instead the symbol system hypothesis.

Asim Roy

Professor, Information Systems

Arizona State University

Lifeboat Foundation Bios: Professor Asim Roy<https://urldefense.com/v3/__https:/lifeboat.com/ex/bios.asim.roy__;!!IKRxdwAv5BmarQ!NHg1P4byXZZmMFLMgr8A4dQNxSlqyvsqWef4wrPXtCXR0f0bo0eMm-rIF8G6ZV0$>

Asim Roy | iSearch (asu.edu)<https://isearch.asu.edu/profile/9973>

--
[cid:image003.jpg at 01DD3625.F5884020]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20260827/11bec007/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 57383 bytes
Desc: image003.jpg
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20260827/11bec007/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 45946 bytes
Desc: image004.jpg
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20260827/11bec007/attachment-0001.jpg>