Neural computing ideas ...

[Hideyuki Cateau]

Bard Ermentrout writes:

>Bacterial cultures in a nutrient depleted medium with inhibited
>motility produce fractal colonies that have the same fractal
>dimension as diffusion-limited aggregation processes.  However the
>mechanism for one is totally different from the other.  All Hopf
>bifurcations from rest scale in the smae fashion.  All of these
>examples are consequences of the fact that many mathematical 
>phenomena obey fixed scaling laws.   But to say that because backprop
>and humnan memory learn with the same scaling laws implies that backprop
>has something to do with real learning is at best specious.  Just becausee
>two phenomena scale the same way doenst mean that the mechanisms are
>identical.


	It is easy to find good examples just like it is easy to find 
bad ones.  Ising model of the magnetism is really simpified model.  No one 
might be able to predict this model is a very good abstraction of the real 
complex material.   However, the scaling behavior in the vicinity of the 
critical point of metal is correctly understood by Ising model.  The model 
of the superconductivity proposed by B.C.S. was also a good model.  It deepen
the understanding of the superconductivity of the real material.  There are 
countless number of good examples as well as bad examples.   So it does not 
seem to make sense to judge that our proposal is a bad one being based on  the
fact that there are many bad examples in the world.

	As he said it is true that there are various kinds of scaling behavior 
in nature.   There is a potential danger that we are incorrectly convinced  
that two of them are derived from the same mechanism just because the values 
of the exponent coincide each other.  But we are not only based on the 
accordance of the exponents but also based on other circumstances in which 
we are working.  

       As I have noted in the original mail,  the reason for the 
slowing  down of the learning pace is considered to be an interfarence between 
different items which are memorized in the brain.   Back prop slows down 
by the same reason.  The configuration of the connection weights Wij which  
is good for one vector to memorize is not good for another vector to memorize 
in general.  Back prop must find out common grounds to memorize serveral items.
It costs time and slows down the memory. At least for us, under these 
circumstances,  it seems natural to expect an analogy between the two slowing 
downs.  Then we performed a simulation to obtain a plausible result.

       Watching one aspect of the matter is always dangerous.  Taking a very 
simple example, we use a trigonometric function when we measure the hieght of 
a tall tree.  On the other hand, we also have a trigonometric function when we 
solve the wave eqution.   Of course it is unnatural to expect some deep 
connection between the two situations.   But for our case it is natural at 
least to us.


      We agree that the evidences might still be not enough.   Our work is 
on the process of exploring more detailed ananlogy between the two systems,
the brain and the back prop.   If the result of our simulation were negative, 
the exploring program had quickly reached the end.  Our conclusion in that 
case would have been that there were at least one evidence which indicated 
back prop was not a good model for our brain. But, at this level of 
exploration, we think we have got a plausible evidence and it is worth 
reporting.   In the future, we might find an aspect which is not common 
between the back prop and brain.  To reach such conclusion is also meaningful
because  our purpose is not to prove that the back prop is the best model of 
the brain but to know to what extent the back prop is a good model of our 
brain.  We think that it is the  starting point of the understanding of 
the brain based on the neural network models.

H.Cateau