What have neural networks achieved?

[David Horn]

Neuronal Regulation: A Complementary Mechanism to Hebbian Learning
------------------------------------------------------------------

We would like to point out the use of neural networks not for modelling
a particular nucleus or cortical area, but for introducing and testing
a general principle of information processing in the brain.
Hebb can be viewed as the founder of this approach,
suggesting a principle of how memories can be encoded in neuronal circuits.
His ideas are still being tested today, and with the advent of knowledge
regarding short term and long term synaptic plasticity, an understanding
of learning and memory seems to be imminent.

Yet there are still many open questions. One of the most interesting ones is
the maintenance of memories over long times in the face of continuous synaptic
metabolic turnover.
We have recently studied this question theoretically (1) and concluded that
to achieve long-term memory there has to exist a Neuronal Regulation
mechanism with the following properties:
  1. Multiplicative modification of all excitatory synapses projecting
  on a pyramidal neuron by a common, joint factor.
  2. The magnitude of this regulatory neuronal factor changes inversely
  with respect to the neuron's post-synaptic potential, or the
  neuron's firing activity.
In contrast to Hebbian changes, the synaptic modifications do not occur
on the individual synaptic level as a function of the correlation between
the firing of its pre and post-synaptic neurons, but take place in unison
over all the synapses projecting on a neuron, as function of its membrane
potential.

In a series of very elegant slice experiments in rat,
Turrigiano et al (2) have recently observed such phenomena.
They find activity dependent changes in AMPA mediated mini EPSCs of
pyramidal neurons. The regulatory process that they have observed has the
features listed above.

We believe that this newly observed mechanism serves as a complement
to Hebbian synaptic learning. In our studies we found that it
regulates basins of attraction of memories, thus preventing formation
of pathologic attractors. Neuronal regulation may hence play a uniquely
important role in preventing clinical and cognitive abnormalities like
schizophrenic positive symptoms, that may result from the formation of
such pathologic attractors (3,4).

Activity-dependent neural regulatory processes have been previously observed
experimentally (5) and studied theoretically (6,7).
We were led to the problem of memory maintenance after first studying
a neural model of Alzheimer's disease, where the late stages of regulatory
processes, that are hypothesized to maintain cognitive function during
normal aging, seem to fail (8).

References:

1. D. Horn, N. Levy and E. Ruppin: Memory maintenance via
neuronal regulation. Neural Computation, 10, 1-18 (1998).

2. G.G. Turrigiano, K. R. Leslie, N. S. Desai, L. C. Rutherford
and S. B. Nelson: Activity-dependent scaling of quantal amplitude in
neocortical neurons. Nature, 391, 892-895 (1998).

3. D. Horn and E. Ruppin: Compensatory mechanisms in an attractor neural network
model of Schizophrenia.  Neural Computation 7, 1494-1517 (1994).

4. E. Ruppin, J. Reggia and D. Horn:  A neural model of positive schizophrenic
symptoms.  Schizophrenia Bulletin 22, 105-123 (1996).

5. G. LeMasson, E. Marder and L. F. Abbott: Activity-dependent regulation of
conductances in model neurons. Science, 259, 1915-1917 (1993).

6. L. F. Abbott and G. LeMasson: Analysis of neuron models with dynamically
regulated conductances. Neural Computation, 5, 823-842 (1993).

7. A. van Ooyen: Activity-dependent neural network development.
Network, 5, 401-423 (1994).

8. D. Horn, N. Levy and E. Ruppin: Neuronal-based synaptic compensation: A
computational study in Alzheimer's disease.  
Neural Computation 8, 1227-1243 (1996).


              David Horn                      Eytan Ruppin
         horn at neuron.tau.ac.il            ruppin at math.tau.ac.il


----------------------------------------------------------------------------
Prof. David Horn                     horn at neuron.tau.ac.il
School of Physics and Astronomy      http://neuron.tau.ac.il/~horn
Tel Aviv University                  Tel: ++972-3-642-9305, 640-7377
Tel Aviv 69978, Israel.              Fax: ++972-3-640-7932