Usefulness of chaotic dynamics

[David Horn]

       Pankaj Mehra has raised the issue of usefulness of chaotic
dynamics in neural network models following a statement by Hopfield
who made a distinction between useful dynamics, which have convergent
behavior, and "unlikely to be useful" dynamics which are divergent.
I would like to point out the importance of dynamics which are
convergent on a short time scale and divergent on a long time scale.
We have worked on neural networks which display such behavior.
In particular we can model a system which converges to a set of fixed
points on a short time scale (thus performing some "useful"
computation), and is "free" to move between them on a longer time scale.
This kind of freedom stems from the unpredictability of chaotic systems.

         Such networks will undoubtedly be very useful for modelling
unguided thinking processes and decision making. We may still be
far from meaningful models of higher cognitive phenomena, but
many will agree that these are valuable long-term goals.
In the meantime, these networks are of interest because they form
good examples of oscillating systems.

         Our networks are feedback systems of formal neurons to which
we attribute dynamical thresholds. Letting a threshold rise when the
neuron with which it is associated keeps firing, we simulate fatigue.
Some short time after the network moves into an attractor, the
fatigue effect destabilizes the attractor and throws the system into a
different basin of attraction. This can go on indefinitely.
Naturally it leads to an oscillatory behavior. The recent
observations of cortical oscillations led to increased interest in
oscillating neural networks, of which ours are particular examples.

         We have developped several models, all of which display
spontaneous and induced transitions between memory patterns.
Such models are simple testing grounds for hypotheses about
the relevance of oscillations to questions of segmentation and binding.
All this is possible because we work with systems which allow for
periodic and chaotic motion between centers of attraction.

References:

D. Horn and M. Usher:
---------------------
Neural Networks with Dynamical Thresholds,
Phys. Rev. A40 (1989) 1036-1044.

Motion in the Space of Memory Patterns,
Proceedings of the 1989 Int. Joint Conf.
on Neural Networks, I-61-69.

Excitatory-Inhibitory Networks with Dynamical Thresholds,
Int. Journal of Neural Systems 1 (1990) 249-257.

Parallel Activation of Memories in an Oscillatory Neural Network
Neural Computation 3 (1991) 31 - 43.

Segmentation and Binding in an Oscillatory Neural Network
submitted to IJCNN-91-Seattle.

O. Hendin, D. Horn and M. Usher:
--------------------------------
Chaotic behavior of a neural network with dynamical thresholds,
Int. J. of Neural Systems, to be published in the next issue.


                                -- David Horn