Connectionists: Brain-like computing fanfare and big data fanfare

[Tsvi Achler]

Jim has referred twice now to a list of problems and brain-like phenomena
that models should strive to emulate. In my mind this gets to the heart of
the matter. However, there was a discussion of one or two points and then
it fizzled. The brain shows many electrophysiological but also behavioral
phenomena.

I would like to revive that discussion (and include not just neuroscience
phenomena) in a list to show: how significant these issues are, the size of
the gap in our knowledge, and focus more specifically on what is brain-like.

Let me motivate this even further. The biggest bottleneck to understanding
the brain is understanding how the brain/neurons perform recognition.
 Recognition is an essential foundation upon which cognition and
intelligence is based.  Without recognition the brain cannot interact with
the world.  Thus a better knowledge of recognition will open up the brain
for better understanding.

Here is my humble list, and I would like to open it to discussions,
opinions, suggestions, and additions.

1) Dynamics. Lets be very specific.  Oscillations are observed during
recognition (as Jim and others mentioned) and they are not satisfactorily
accounted.  Since single oscillation generators have not been found, I
interpret this means the oscillations are likely due to some type of
feedforward-feedback connections functioning during recognition.

2) Difficulty with Similarity. Discriminating between similar patterns
recognition takes longer and is more prone to error.  This is not primarily
a spatial search phenomena because it occurs in all modalities including
olfaction which has very poor spatial resolution.  Thus appears to be a
fundamental part of the neural mechanisms of recognition.

3) Asymmetry.  This is related to signal-to-noise like phenomena to which
difficulty with similarity belong.  Asymmetry is a special case of
difficulty with similarity, where a similar pattern with more information
will predominate the one with less.

4) Biased competition (priming). Prior expectation affects recognition time
and accuracy.

5) Recall-ability. The same neural recognition network that can perform
recognition likely performs recall.  This is suggested by studies where
sensory region activation can be observed when recognition patterns are
imagined, and by the existence of mirror neurons.

6) Update-ability.  The brain can learn new information (online outside the
IID assumption) and immediately use it.  It does not have to retrain on all
old information (IID requirement for feed-forward neural networks).

If we do not seriously consider networks that inherently display these
properties, I believe neural the network community will continue rehashing
ideas and see limited progress.

My strong yet humble opinions,

-Tsvi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140127/b31c3faa/attachment.html>