Usefulness of chaotic dynamics

[Pankaj Mehra]

Fellow Connectionists:

	It was with fascination that I read Gleick's book on chaos
recently. My initial reaction was that the thoery of chaotic dynamics
of nonlinear systems represents a significant advance in science by
allowing us to model and understand complex phenomena, such as turbulence.

	Yesterday, John Hopfield was in Champaign, talking about ``Neural
Computation'' at the centennial celebration of Physics department. During
his talk, he made a distinction between useful and ``unlikely to be useful''
dynamics of neural networks with feedback. Essentially, dynamics which show
convergent behavior are considered useful, and those that show chaotic or
divergent behavior, not useful.

	The crux of such an argument lies in the fact that ``similar but
unresolvable initial conditions lead to large divergences in trajectories'' [1]
for a chaotic system. This is an antithesis of generalization, which is
considered a defining trait of connectionist models.

	However, a recent paper by Frison [1] shows how the behavior of
chaotic systems may be predicted using neural networks. The applications
suggested in this paper include weather forecasting and securities analysis.

	In my opinion, there is an emerging trend to move away from convergent
dynamics. Section 4 of Hirsch's paper [2] presents arguments for/against using
chaotic dynamics. In a limited sense, the work of Michail Zak [3] shows how
breaking some traditional assumptions of convergent behavior can help solve
the problems of spurious and local minima. I would like to hear more about
the pros and cons of this issue.

- Pankaj Mehra
{p-mehra at uiuc.edu, mehra at cs.uiuc.edu, mehra at aquinas.csl.uiuc.edu}

[1] Frison, T. W., "Predicting Nonlinear and Chaotic Systems Behavior
	Using Neural Networks," Jrnl. Neural Network Computing, pp.
	31-39, Fall 1990.

[2] Hirsch, M. W., "Convergent Activation Dynamics is Continuous Time
	Networks," Neural Networks, vol. 2, pp. 331-349, Pergamon
	Press, 1989.

[3] Zak, M., "Creative Dynamics Approach to Neural Intelligence,"
	Biological Cybernetics, vol. 64, pp. 15-23, Springer-Verlag,
	1990.