Connectionists: Brain-like computing fanfare and big data fanfare

[james bower]

Tsvi,

Nice list and I think a productive approach - the nature of the questions is obviously important.

Its another long story, but might I ask that you, and the community consider how the answer to each of these questions might change, if for example, the nervous system already ‘knows’ in advance what it is looking for and ‘learning’ in the usual sense, is not involved.

In our own work,  to my great surprise, evidence has emerged from both detailed physiological models of the olfactory cortex and also from studies of the classification systems that humans use to describe odors, that the olfactory system might already know a great deal in advance about the metabolic structure of the organic world it is trying to detect and that that prior knowledge plays a key role in recognition.  These results, first obtained 15 years ago, were never published except in thesis form in large part because they were so antithetical to current thinking about how the olfactory system worked in particular and learning worked in general that we decided it wasn’t worth the effort.  A previous paper showing that olfactory receptive fields changed in the olfactory bulb, using one of the first ever awake behaving multi-single unit recording procedures, took 5 years to get published.  

However, the metabolic hypothesis (as we have called it) was recently a subject in a meeting in Germany for which I append at the end of this comment a brief description of the idea.

You might find it interesting that these two lines of evidence pointing in the same direction were obtained completely independently, and that the modeling result in particular was completely unexpected.  What we set out to do in what was and I think still is the most detailed biological model of certainly olfactory cortex ever made, was to duplicate the pattern of current source densities found during (natural) cortical oscillations. (BTW: the first version of this model 25 years ago showed that these oscillations are not ‘driven’ from anywhere, but in fact are an intrinsic property of the network itself).   It turned out that the only way the experimental data could be reconstructed was if there were independent subnetworks in the cortex.  For 25 previous years, I had assumed that the olfactory cortex was some kind of associative learning network (influence of the NN community actually), based on its apparently highly defuse and topographically unorganized set of intrinsic excitatory connections.  Turns out, the model predicted that this apparent diffuseness may be concealing what is actually a highly organized network structure - but not of the usual topographic type.  I suspect, although we don’t know that these subnets reflect the structure of the metabolic world.

So, after 25 years, i was forced by the modeling work to completely change how I was thinking about how the system worked.

This is the value and power of this type of modeling, to fundamentally change what you think about how something works.

Perhaps it doesn’t need to be pointed out, that this is also obviously the kind of result that could support and drive the kind of experimental effort that the Brain project is intent on undertaking.  Except that instead of blind data collection - the data collection is organized in the context of a particular hypothesis.  My guess is (and we could probably use the model to test this) that finding these subnetworks with blind data collection would be much more difficult or perhaps even impossible.

Anyway, a good list of questions, but as with any list of questions, they make assumptions about how the system works.

A question like: "what do we have to assume about the intrinsic connectivity of olfactory cortex to  duplicate the pattern of current source density distributions following electrical shock of the lateral olfactory track in a detailed biological model of the olfactory cortex”  makes many fewer functional assumptions.  But, in this case (and in most cases of this type of modeling we have done), what falls out is something we didn’t know was there, with, it would seem, significant potential functional significance

to return again to Newton - while he clearly was interested in why the moon remained in a circular orbit around the earth, he had no idea that the apparent force between them had a regular relationship to the distance, until he first invented (or stole depending) the calculous and actually saw the relationship.

Had nothing to do with the inspiration of an apple falling in his sister’s orchard.  That was a story that he apparently made up subsequently to impress others with his insight and genius.  

:-)

Jim

 

Metabolic – hypothesis  Summary meeting report

The Structure of Olfactory Space

Hannover Germany Sept, 2013.
 

Question:  Is the olfactory system a chemical classifier, or a detector of natural biological chemical processes?

In the first century BC, the Roman poet and philosopher Lucretius speculated about olfaction:  “Thus simple 'tis to see that whatsoever can touch the senses pleasingly are made of smooth and rounded elements, whilst those which seem the bitter and the sharp, are held Entwined by elements more crook'd”.   This intuition that the olfactory system generates olfactory percepts by interpreting the general chemical structure of odorant molecules continues to underlie much olfactory research.  Practically, it is manifest in the continued reliance on monomolecular odorant stimuli most often presented as chemical families (alchohols and aldehydes) varying along a single chemical metric (e.g. carbon length chain).   The results, at multiple levels of scale from single receptor neurons to networks, typically show individual elements responding to a large and complex range of compounds, leading in turn to the suggestion that the olfactory system uses a distributed combinatorial code to learn to recognize objects.  Perceptually however, compounds with highly different chemical structures can elicit similar odors, while small changes in chemical structure can render a highly odorous compound completely odorless.  For these and other reasons, traditional approaches to classifying the perception of odorant molecules based on their physical structure continue to have minimal predictive value.

We believe, as an alternative, it is worth considering whether the olfactory system may not be a chemical classifier in the traditional sense, but instead has evolved to detect known chemical patterns reflecting biologically important signals in nature.  In this view,  “odor perceptual space” is predicted to be organized around the chemical structure of the organic world including, for example, the chemical signature of specific metabolic pathways (from traditional food sources), chemical patterns generated by one species to specifically attract other species (allomones, kairomones, or even compounds given off by fruit to signal ripeness), or stimuli signaling the interactions of “consortia of organisms” (microbial digestion of plant or animal tissue).

What we are proposing as the “Metabolomics Hypothesis” makes several specific predictions:  The core prediction is that the olfactory system will be organized around biologically significant mixtures of molecules, in effect, seeking evidence for the presence of particular chemical interactions within the environment; This structure may be apparent as early as single olfactory receptor proteins which could, for example, bind odorants that are metabolically related, even if structurally dissimilar;  As a special case, molecules employed as signals between species (allomones, kairomones , or molecules signally the ripeness of fruit for example) might induce responses in a broad number of receptors; Receptor neuron projections to the olfactory bulb as well as bulbar projections to the olfactory cortex may be more ordered than previously assumed reflecting this structure; This hypothesis further predicts that metabolic relatedness is more likely to predict perception and perceptual interactions (cross adaptation for example) than would either simple structural similarity, or chemical class.  Finally, and perhaps most importantly, we would predict that this structural knowledge of the chemical world may be ‘built into’ the olfactory system at the outset, providing a non-learned basis for olfactory perception.  Such an existing structure would relegate ‘learning’ to changes in aversive/preference (hedonic) scale based on individual experience. While preliminary evidence exists for each of these predictions (c.f. Chee, 2003; Vanier, 2001), further experimental work is necessary to test this new hypothesis.  That work will depend, however, on the use of panels or mixtures of odorants with known behavioral significance.

 

 

 

 

 




On Jan 27, 2014, at 12:31 PM, Tsvi Achler <achler at gmail.com> wrote:

> Jim has referred twice now to a list of problems and brain-like phenomena that models should strive to emulate. In my mind this gets to the heart of the matter. However, there was a discussion of one or two points and then it fizzled. The brain shows many electrophysiological but also behavioral phenomena.
> 
> I would like to revive that discussion (and include not just neuroscience phenomena) in a list to show: how significant these issues are, the size of the gap in our knowledge, and focus more specifically on what is brain-like.
> 
> Let me motivate this even further. The biggest bottleneck to understanding the brain is understanding how the brain/neurons perform recognition.  Recognition is an essential foundation upon which cognition and intelligence is based.  Without recognition the brain cannot interact with the world.  Thus a better knowledge of recognition will open up the brain for better understanding.
> 
> Here is my humble list, and I would like to open it to discussions, opinions, suggestions, and additions.
> 
> 1) Dynamics. Lets be very specific.  Oscillations are observed during recognition (as Jim and others mentioned) and they are not satisfactorily accounted.  Since single oscillation generators have not been found, I interpret this means the oscillations are likely due to some type of feedforward-feedback connections functioning during recognition.
> 2) Difficulty with Similarity. Discriminating between similar patterns recognition takes longer and is more prone to error.  This is not primarily a spatial search phenomena because it occurs in all modalities including olfaction which has very poor spatial resolution.  Thus appears to be a fundamental part of the neural mechanisms of recognition.
> 
> 3) Asymmetry.  This is related to signal-to-noise like phenomena to which difficulty with similarity belong.  Asymmetry is a special case of difficulty with similarity, where a similar pattern with more information will predominate the one with less.
> 
> 4) Biased competition (priming). Prior expectation affects recognition time and accuracy.
> 
> 5) Recall-ability. The same neural recognition network that can perform recognition likely performs recall.  This is suggested by studies where sensory region activation can be observed when recognition patterns are imagined, and by the existence of mirror neurons.
> 
> 6) Update-ability.  The brain can learn new information (online outside the IID assumption) and immediately use it.  It does not have to retrain on all old information (IID requirement for feed-forward neural networks).
> 
> If we do not seriously consider networks that inherently display these properties, I believe neural the network community will continue rehashing ideas and see limited progress.
> 
> My strong yet humble opinions,
> 
> -Tsvi
> 
> 
> 

 

 

Dr. James M. Bower Ph.D.

Professor of Computational Neurobiology

Barshop Institute for Longevity and Aging Studies.

15355 Lambda Drive

University of Texas Health Science Center 

San Antonio, Texas  78245

 

Phone:  210 382 0553

Email: bower at uthscsa.edu

Web: http://www.bower-lab.org

twitter: superid101

linkedin: Jim Bower

 

CONFIDENTIAL NOTICE:

The contents of this email and any attachments to it may be privileged or contain privileged and confidential information. This information is only for the viewing or use of the intended recipient. If you have received this e-mail in error or are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of, or the taking of any action in reliance upon, any of the information contained in this e-mail, or

any of the attachments to this e-mail, is strictly prohibited and that this e-mail and all of the attachments to this e-mail, if any, must be

immediately returned to the sender or destroyed and, in either case, this e-mail and all attachments to this e-mail must be immediately deleted from your computer without making any copies hereof and any and all hard copies made must be destroyed. If you have received this e-mail in error, please notify the sender by e-mail immediately.

 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140127/a0783f3e/attachment.html>