What have neural networks achieved?

[Terry Sejnowski]

A footnote to Jay's last post on interference:  One of the best
established facts about memory is the spacing effect -- long term
retention is much better for a wide variety of tasks and materials
if the training is spaced in time rather than massed (cramming may
help for the test tomorrow, but you won't remember it next year).

Charlie Rosenberg showed that NETtalk exhibits a robust spacing
effect when learning a new set of words.  The explanation is similar
to the one Jay has provided:  You don't want to find the nearest
place in weight space that codes the new words, but the nearest
location in weight space that codes the old words and the new ones.

Rosenberg, C. R. and Sejnowski, T. J., The effects of 
distributed vs massed practice on NETtalk, a massively-parallel 
network that learns to read aloud, Proceedings 8th Annual Conference 
of the Cognitive Science Society, Amherst, MA (August 1986). 

Whether the hippocampus is "replaying" recent experiences to the
neocortex during sleep is an open question, though there is some
evidence for this in rats.  For further discussion see:

Sejnowski, T. J., Sleep and memory, Current Biology 5, 832-834 (1995). 

There are high-amplitude thalamocortical rhythms that occur during
sleep whose function is unknown.  The mechanisms underlying these
slow rhythms have been studied at the biophysical level, and 
incorporated into network models, which would qualify for a "success"
story for understanding large-scale dynamical properties of brain
systems:

Steriade, M., McCormick, D. A., Sejnowski, T. J., 
Thalamocortical oscillations in the sleeping and aroused brain, 
Science 262, 679-685 (1993). 

Destexhe, A. and Sejnowski, T. J., Synchronized oscillations in 
thalamic networks: Insights from modeling studies, 
In: M. Steriade, E. G. Jones and D. A. McCormick (Eds.) 
Thalamus, Elsevier, pp 331-371 (1997). 

These models are a first step toward understanding the function of
these oscillations, and perhaps someday the function of sleep, which
remains a deep mystery.

Regarding the comment by Ken Miller, the regions of the cortex that
surround the hippocampus, including the entorhinal cortex, the
perirhinal cortex and the parahippocampal cortex are staging areas
for converging inputs to the hippocampus.  Stuart Zola has shown
that the severity of amnesia folowing lesions of these areas in monkeys is
greater as more surrounding cortical areas are included in the lesion.  
The famous case of HM had surgical removal of the tmeporal lobe which 
included the areas surrounding the hippocampus.  The view in the field is no
longer to think of the hippocampus as the primary site but as part
of a memory system in reciprocal interaction with these cortical
areas.  Other brain areas including frontal cortex and the cerebellum also
are involved:

 Tulving E; Markowitsch HJ.
     Memory beyond the hippocampus.
   Current Opinion in Neurobiology, 1997 Apr, 7(2):209-16.

Functional magentic resonance imaging is a powerful new tool for measuring activity
and has been applied to memory systems -- see the latest results in the 
21 August issue of Science.

Terry

-----