New Textbook on Reinforcement Learning

[Rich Sutton]

Dear Colleagues,

This note is to announce the availability of a new textbook on reinforcement
learning by Andy Barto and me.  As many of you know, we have been working on
this book for over four years.  A few weeks ago we received our authors'
copies, and the book is now available by internet/mail order and in bookstores:

Sutton, R.S., Barto, A.G. (1998)  Reinforcement Learning: An Introduction.
MIT Press, Cambridge, MA.

The rest of this note says a little more about the book and points to further
information.

As its title indicates, the book is meant to be an introductory treatment of
reinforcement learning, emphasizing foundations and ideas rather than the
latest developments and mathematical proofs.  We divide the ideas
underlying the
field into a half dozen primary dimensions, consider each in detail, and then
combine them to form a much larger space of possible methods including all the
most popular ones from Q-learning to value iteration and heuristic search.  In
this way we have tried to make the book interesting to both newcomers and
experts alike.  We have tried to make the work accessible to the broadest
possible audiences in artificial intelligence, control engineering,
operations research, psychology, and neuroscience.

If you are a teacher, we urge you to consider creating or altering a course to
use the book.  We have found that the book works very well as the basis for an
independent course on reinforcement learning at the graduate or advanced
undergraduate level.  The eleven chapters can be covered one per week.
Exercises are provided in each chapter to help the students think on their
own about the material.  Answers to the exercises are available to
instructors, for now from me, and probably later from MIT Press in an
instructor's manual.  Programming projects are also suggested throughout
the book.

Of course, the book can also be used to help teach reinforcement learning as
it is most commonly done now, that is, as part of a broader course on machine
learning, artificial intelligence, neural networks, or advanced control.  I
have
taught all the material in the book in as little as four weeks, and of course
subsets can be covered in less time.

Finally, if you are interested in reviewing the book for a major journal or
magazine, please contact our MIT Press publicist, Gita Manaktala
(manak at mit.edu or 617-253-5643), directly.

Further information about the book, including ordering information and
detailed information about its contents, can be obtained from its home page at
http://www.cs.umass.edu/~rich/book/the-book.html.

Rich Sutton
rich at cs.umass.edu