Dissertation on cognitive economy and reinforcement learning

[David J. Finton]

Dear Connectionists:

I am pleased to announce the availability of my Ph.D. dissertation
for download:

____________________________________________________________________

Cognitive Economy and the Role of Representation in On-Line Learning

    PDF:    http://www.cs.wisc.edu/~finton/thesis/main.pdf

    PS:     http://www.cs.wisc.edu/~finton/thesis/main.ps.gz
____________________________________________________________________


The dissertation is 265 pages, and the downloadable files are
1.79 MB (PDF version) and 1.13 MB (gzipped PostScript version).

An abstract follows.

--David Finton

finton at cs.wisc.edu
http://www.cs.wisc.edu/~finton/


Abstract
________

How can an intelligent agent learn an effective representation of
its world?  This dissertation applies the psychological principle of
cognitive economy to the problem of representation in reinforcement
learning.  Psychologists have shown that humans cope with difficult
tasks by simplifying the task domain, focusing on relevant features
and generalizing over states of the world which are ``the same'' with
respect to the task.  This dissertation defines a principled set of
requirements for representations in reinforcement learning, by applying
these principles of cognitive economy to the agent's need to choose the
correct actions in its task.

The dissertation formalizes the principle of cognitive economy into
algorithmic criteria for feature extraction in reinforcement learning.
To do this, it develops mathematical definitions of feature importance,
sound decisions, state compatibility, and necessary distinctions, in
terms of the rewards expected by the agent in the task.  The analysis
shows how the representation determines the apparent values of the agent's
actions, and proves that the state compatibility criteria presented here
result in representations which satisfy a criterion for task learnability.

The dissertation reports on experiments that illustrate one implementation
of these ideas in a system which constructs its representation as it
goes about learning the task.  Results with the puck-on-a-hill task and
the pole-balancing task show that the ideas are sound and can be of
practical benefit.  The principal contributions of this dissertation
are a new framework for thinking about feature extraction in terms
of cognitive economy, and a demonstration of the effectiveness of an
algorithm based on this new framework.