Connectionist Learning - Some New Ideas

[Sydney M Lamb]

On Fri, 17 May 1996 Jonathan_Stein at comverse.com wrote:

> 
> One needn't draw upon injuries to prove the point. One loses about 100,000
> cortical neurons a day (about a percent of the original number every three
> years) under normal conditions. This loss is apparently not significant
> for brain function. This has been often called the strongest argument for
> distributed processing in the brain. Compare this ability with the fact that
> single conductor disconnection cause total system failure with high
> probability in conventional computers.
> 
> Although certainly acknowledged by the pioneers of artificial neural
> network techniques, very few networks designed and trained by present
> techniques are anywhere near that robust. Studies carried out on the
> Hopfield model of associative memory DO show graceful degradation of
> memory capacity with synapse dilution under certain conditions (see eg.
> DJ Amit's book "Attractor Neural Networks"). Synapse pruning has been 
> applied to trained feedforward networks (eg. LeCun's "Optimal Brain Damage")
> but requires retraining of the network.
> 
> JS
> 

There seems to be some differing information coming from different 
sources.  The way I heard it, the typical person has lost only about 3% 
of the original total of cortical neurons after about 70 or 80 years.

As for the argument about distributed processing, two comments: (1) there 
are different kinds of distributive processing; one of them also uses 
strict localization of points of convergence for distributed subnetworks 
of information (cf. A. Damasio 1989 --- several papers that year).  (2) 
If the brain is like other biological systems, the neurons being lost are 
probably most the ones not being used --- ones that have been remaining 
latent and available to assume some function, but never called upon.  
Hence what you get with old age is not so much loss of information as 
loss of ability to learn new things --- varying in amount, of course, 
from one individual to the next.

Syd Lamb
Linguistics and Cognitive Science
Rice University