review article on temporal sequence learning

[Florentin Woergoetter]

Dear Connectionists,

I would like to announce the availability of a review article which 
could be of broader interest at:
http://www.cn.stir.ac.uk/~faw1/Publications/papers/woe_porr_nc2004.pdf

Wrgtter, F. and Porr, B. (2004) Temporal sequence learning, prediction 
and control - A review of different models and their relation to 
biological mechanisms. Neural Comp. 17, 1-75.

Abstract:
In this article we compare methods for temporal sequence learning (TSL) 
across the disciplines machine-control, classical conditioning, neuronal 
models for TSL as well as spike-timing dependent plasticity. This review 
will briefly introduce the most influential models and focus on two 
questions: 1) To what degree are reward-based (e.g. TD-learning) and 
correlation based (hebbian) learning related? and 2) How do the 
different models correspond to possibly underlying biological mechanisms 
of synaptic plasticity? We will first compare the different models in an 
open-loop condition, where behavioral feedback does not alter the 
learning. Here we observe, that reward-based and correlation based 
learning are indeed very similar. Machine-control is then used to 
introduce the problem of closed-loop control (e.g. actor-critic 
architectures). Here the problem of evaluative (rewards) versus 
non-evaluative (correlations) feedback from the environment will be 
discussed showing that both learning approaches are fundamentally 
different in the closed-loop condition. In trying to answer the second 
question we will compare neuronal versions of the different learning 
architectures to the anatomy of the involved brain structures 
(basal-ganglia, thalamus and cortex) and to the molecular biophysics of 
glutamatergic and dopaminergic synapses. Finally we discuss the 
different algorithms used to model spike-timing dependent plasticity 
(STDP) and compare them to reward based learning rules. Certain 
similarities are found in spite of the strongly different time scales. 
Here we focus on the biophysics of the different Calcium-release 
mechanisms known to be involved in STDP.


best wishes

F. Woergoetter