Neural computing ideas and biological terms

[Hideyuki Cateau]

In reply to the following discussion:

>Rogene Eichler writes
>  However, I grow tired of defending the validity of models to biologists
>  who do not seem satisfied with any model that does not capture every last
>  nuiance of complexity or that does not explain every last experimental
>  finding. Modeling the brain will provide valuable insights into how we
>  process information and how we can exploit those rules for artificial 
>  systems. But they do not need to duplicate every last brain dynamic to
>  be useful or valid. 
>
>This is especially true if it is NOT the case that the details of brain
>function are the roots of brain behavior.  These minute details may be
>washed out by dynamical principles which have their own behavioral
>"chemistry".  For a mathematical example, look at the universality of
>symbol dynamics in unimodal iterated mapping (that's just a single bump,
>like the logistic function).  As long as the mappings meet some fairly
>general qualifications, the iterated systems based on those mappings share
>the qualitative behavior, namely the bifurcation structure, regardless of
>the quantitative differences between the individual mappings.
>
>John Kolen

   I agree to Dr.Kolen.  I would like the connectionists to pay attention to 
the possibility that neural network models can explain not only the 
qualitative aspects of our brain but also the "quantitative" one of it.  

   In fact, I and my collaborators have found that the learning pace of the 
back propagation modeland the human brain are subject to a same power law 
with  nearly equal values of the exponent.   This is reported in 
"Power law in the human memory and in the neural network model,
 H.Cateau, T.Nakajima, H.Nunokawa and N.Fuchikami", which is 
placed in the neuroprose as a file cateau.power.tar.Z.

   Let us denote the time which is spent when one memorize M items by t(M).  
As M increases the learning pace slows down as you usually experience.   A 
psychologist Foucault (M.Foucaut, Annee Psychol.19 (1913)218) found 
experimentally that this slowing down behavior is described by a following 
power law:

      t(M)= const*M^D

where D is a constant.  He expecially claimed that D=2.

  We have performed the same experiment by ourself and found that the power law
is true with high statistical confidence and that D is between 1 and 2.

  Then we examined whether or not the back propagation(BP)network has the same 
property when it memorize some items.   The answer was yes. The BP network is 
subject to the power law with a fairly nice precision.  Furthermore the 
observed value of the exponent was two up to errors.  All connectionists can 
easily check this interesting property by themselves and convince themselves 
that the fitting of the data to the above law is very good.

  When we make the BP memorize several items, the memories embedded in the 
connection weights interfere each other.  Thus the slowing down of the learning
is expected to occur also fo the BP.  This is a qualitative expectation.  But 
above result shows that the similarity is not only qualitative but also 
quantitative.  I think this shows that the BP model, although it is too simple,
surely simulate some essential feature of the real brain and that the studies 
of the neural network model cast a light on a secret of our brain. 

  When we discuss whether or not the neural network model can explain some 
experimental results of the brain,  we usually have, in our mind, 
physiological experiments.   However, there are also many psychological 
experiments for the human brain.  Many of such results are scientifically 
reliable because the statistical significance of the results were strictly 
checked.
  
  I belive that it is really meaningful as a study of the brain that we 
examine whether the existing neural network models can explain the many 
other psychological experiments.



Hideyuki Cateau
Particle theory group, Department of Physics,University of Tokyo,7-3-1,
Hongo,Bunkyoku,113 Japan
e-mail:cateau at star.phys.metro-u.ac.jp