Speedy Alternatives to Back Propagation

[john moody]

At Yale, we have been  studying  two  classes  of  neurally-
inspired  learning  algorithms  which  offer 1000-fold speed
increases over back  propagation  for  learning  real-valued
functions.  These  algorithms  are  "Learning with localized
receptive fields" and  "An  interpolating,  multi-resolution
CMAC",  where  CMAC means Cerebellar Model Articulation Con-
troller.  Both algorithms were presented in  talks  entitled
"Speedy  Alternatives to Back Propagation" given at Snowbird
(April '88), nEuro '88 (Paris, June '88), and INNS  (Boston,
September  '88).  A research report describing the localized
receptive fields approach is now available. Another research
report describing the CMAC models will be available in about
two weeks. To receive copies of these, please send a request
to Judy Terrell at terrell at yalecs.bitnet, terrell at yale.arpa,
or terrell at cs.yale.edu. Be  sure  to  include  your  mailing
address.  There  is  no charge for the research reports, and
they are written in English! An abstract follows.

--John Moody




          Learning with Localized Receptive Fields

              John Moody and Christian Darken
             Yale Computer Science Department
       PO Box 2158 Yale Station, New Haven, CT 06520

              Research Report YALEU/DCS/RR-649
                       September 1988

                          Abstract


We propose  a  network  architecture  based  upon  localized
receptive  field  units and an efficient method for training
such a network which combines self-organized and  supervised
learning.   The  network architecture and learning rules are
appropriate for real-time  adaptive  signal  processing  and
adaptive  control.  For a test problem, predicting a chaotic
timeseries, the network learns 1000 times faster in  digital
simulation  time  than a three layer perceptron trained with
back propagation, but requires about ten times more training
data to achieve comparable prediction accuracy.



This research report will appear in the Proceedings  of  the
1988  Connectionist  Models  Summer School, Morgan Kaufmann,
Publishers 1988.   The  work  was  supported  by  ONR  grant
N00014-86-K-0310,  AFOSR grant F49620-88-C0025, and a Purdue
Army subcontract.

-------