TR on reinforcement learning

[Ron Sun]

Announcing three new papers on combining reinforcement learning with
symbolic methods:


---------------------------------
Autonomous Learning of Sequential Tasks: Experiments and Analyses

by Ron Sun$^{1,2}$, Todd Peterson$^2$

$^1$ NEC Research Institute, 4 Independence Way, Princeton, NJ 08540 \\
$^2$ The University of Alabama,

Appeared in  IEEE Transactions on Neural Networks, Nov.1998

http://cs.ua.edu/~rsun/sun.tnn98.ps

ABSTRACT:
This paper presents  a novel learning model {\sc Clarion},
which is a hybrid model
based on the two-level approach proposed in Sun (1995).
The model integrates neural, reinforcement, and symbolic learning 
methods to perform on-line,  bottom-up learning (i.e.,
learning that goes from neural to symbolic representations).
The model utilizes both procedural and declarative knowledge
(in neural and symbolic representations respectively),
tapping into the  synergy of the two types of processes.
It was applied to deal with sequential decision tasks.
Experiments and analyses  in various ways
are reported that shed light on the advantages of the model.

---------------------------------
Multi-Agent Reinforcement Learning: Weighting and  Partitioning

by  Ron Sun and  Todd Peterson

To appear in: Neural Networks

http://cs.ua.edu/~rsun/sun.NN99.ps

ABSTRACT:
This paper addresses weighting and partitioning
in complex reinforcement learning tasks, with the aim
of facilitating learning.  The paper presents some 
ideas regarding weighting of multiple agents
and extends them into partitioning an input/state space into
multiple regions with differential weighting in these regions,
to exploit the differential  characteristics of regions and
the differential characteristics of agents to
reduce the learning complexity of agents (and their
function approximators) and thus to facilitate the learning 
overall.  It analyzes, in reinforcement learning tasks,
different ways of partitioning a task
and using agents selectively based on partitioning.
Based on the analysis, some heuristic methods are described 
and experimentally tested.  We find that some off-line 
heuristic methods performed the best,
significantly better than single-agent models.

---------------------------------
A Hybrid Architecture for Situated Learning of Reactive Sequential Decision Making

by
Ron Sun 
Todd Peterson
Edward Merrill

To appear in: Applied Intelligence

http://cs.ua.edu/~rsun/sun.apin99.ps

ABSTRACT:
In developing autonomous agents, one usually emphasizes  only (situated) 
procedural knowledge, ignoring more explicit declarative knowledge.
On the other hand, in developing symbolic reasoning models,
one usually emphasizes only declarative knowledge, ignoring procedural
knowledge.  In contrast, we have developed a learning model {\sc Clarion},
which is a hybrid connectionist model consisting of both localist
and distributed representations, based on the two-level approach
proposed in Sun (1995).  {\sc Clarion} learns and utilizes both
procedural and declarative knowledge, tapping into the  synergy of the 
two types of processes, and enables an agent to learn in situated contexts
and generalize  resulting knowledge to different scenarios.
It unifies connectionist, reinforcement, and symbolic learning
in a synergistic way, to perform on-line,  bottom-up learning.
This summary paper presents one version of the architecture
and some results of the experiments.

-----------------------------------------
Dr. Ron Sun
NEC Research Institute
4 Independence Way
Princeton, NJ 08540
phone: 609-520-1550
fax: 609-951-2483
email: rsun at cs.ua.edu,  rsun at research.nj.nec.com 
   (July 1st, 1998 -- July 1st, 1999)
-----------------------------------------

Prof. Ron Sun                                      http://cs.ua.edu/~rsun
Department of Computer Science 
 and Department of Psychology                       phone: (205) 348-6363
The University of Alabama                           fax:   (205) 348-0219
Tuscaloosa, AL 35487                                email: rsun at cs.ua.edu