Annoucement of papers extending Delta-Bar-Delta

[Rich Sutton]

Dear Learning Researchers:

I have recently done some work extending that by Jacobs and others on
learning-rate-adaptation methods.  The three papers announced below
extend it in the directions of machine learning, optimal linear
estimation, and psychological modeling, respectively.  Information on
how to obtain copies is given at the end of the message.
                                                          -Rich Sutton

----------------------------------------------------------------------
To appear in the Proceedings of the Tenth National Conference on
Artificial Intelligence, July 1992:

		   ADAPTING BIAS BY GRADIENT DESCENT: 
	       AN INCREMENTAL VERSION OF DELTA-BAR-DELTA

			   Richard S. Sutton
		     GTE Laboratories Incorporated

Appropriate bias is widely viewed as the key to efficient learning and
generalization.  I present a new algorithm, the Incremental
Delta-Bar-Delta (IDBD) algorithm, for the learning of appropriate
biases based on previous learning experience.  The IDBD algorithm is
developed for the case of a simple, linear learning system---the LMS or
delta rule with a separate learning-rate parameter for each input.  The
IDBD algorithm adjusts the learning-rate parameters, which are an
important form of bias for this system.  Because bias in this approach
is adapted based on previous learning experience, the appropriate
testbeds are drifting or non-stationary learning tasks.  For particular
tasks of this type, I show that the IDBD algorithm performs better than
ordinary LMS and in fact finds the optimal learning rates.  The IDBD
algorithm extends and improves over prior work by Jacobs and by me in
that it is fully incremental and has only a single free parameter.  This
paper also extends previous work by presenting a derivation of the IDBD
algorithm as gradient descent in the space of learning-rate parameters.
Finally, I offer a novel interpretation of the IDBD algorithm as an
incremental form of hold-one-out cross validation.

--------------------------------------------------------------------
Appeared in the Proceedings of the Seventh Yale Workshop on Adaptive
and Learning Systems, May 1992, pages 161-166:

		  GAIN ADAPTATION BEATS LEAST SQUARES?

			   Richard S. Sutton
		     GTE Laboratories Incorporated

I present computational results suggesting that gain-adaptation
algorithms based in part on connectionist learning methods may improve
over least squares and other classical parameter-estimation methods for
stochastic time-varying linear systems.  The new algorithms are
evaluated with respect to classical methods along three dimensions:
asymptotic error, computational complexity, and required prior knowledge
about the system.  The new algorithms are all of the same order of
complexity as LMS methods, O(n), where n is the dimensionality of the
system, whereas least-squares methods and the Kalman filter are O(n^2).
The new methods also improve over the Kalman filter in that they do not
require a complete statistical model of how the system varies over time.
In a simple computational experiment, the new methods are shown to
produce asymptotic error levels near that of the optimal Kalman filter
and significantly below those of least-squares and LMS methods.  The new
methods may perform better even than the Kalman filter if there is any
error in the filter's model of how the system varies over time.

------------------------------------------------------------------------
To appear in the Proceedings of the Fourteenth Annual Conference of
the Cognitive Science Society, July 1992:

	       ADAPTATION OF CUE-SPECIFIC LEARNING RATES 
	      IN NETWORK MODELS OF HUMAN CATEGORY LEARNING

		    Mark A. Gluck, Paul T. Glauthier
       Center for Molecular and Behavioral Neuroscience, Rutgers

			 and Richard S. Sutton
		     GTE Laboratories Incorporated

Recent engineering considerations have prompted an improvement to the
least mean squares (LMS) learning rule for training one-layer adaptive
networks: incorporating a dynamically modifiable learning rate for each
associative weight accellerates overall learning and provides a
mechanism for adjusting the salience of individual cues (Sutton, 1992).
Prior research has established that the standard LMS rule can
characterize aspects of animal learning (Rescorla & Wagner, 1972) and
human category learning (Gluck and Bower, 1988).  We illustrate how this
enhanced LMS rule is analogous to adding a cue-salience or attentional
component to the psychological model, giving the network model a means
of distinguishing between relevant and irrelevant cues.  We then
demonstrate the effectiveness of this enhanced LMS rule for modelling
human performance in two non-stationary learning tasks for which the
standard LMS network model fails to account for the data (Hurwitz, 1990;
Gluck, Glauthier & Sutton, in preparation).

------------------------------------------------------------------------

To obtain copies of these papers, please send an email request to
jpierce at gte.com.  Be sure to include your physical mailing address.