Connectionists: Alternative Time Representation in Dopamine Models

[Francois Rivest]

I am glad to announce the publication of a new model of timing in TD models
of dopamine (see abstract below). 
Feel free to e-mail me comments or questions at [rivestfr [at]
iro.umontreal.ca].

Rivest, Kalaska, & Bengio (2009) Alternative time representation in dopamine
models. Journal of Computational Neuroscience.
http://dx.doi.org/10.1007/s10827-009-0191-1

Abstract:
Dopaminergic neuron activity has been modeled during learning and appetitive
behavior, most commonly using the temporal-difference (TD) algorithm.
However, a proper representation of elapsed time and of the exact task is
usually required for the model to work. Most models use timing elements such
as delay-line representations of time that are not biologically realistic
for intervals in the range of seconds. The interval-timing literature
provides several alternatives. One of them is that timing could emerge from
general network dynamics, instead of coming from a dedicated circuit. Here,
we present a general rate-based learning model based on long short-term
memory (LSTM) networks that learns a time representation when needed. Using
a naïve network learning its environment in conjunction with TD, we
reproduce dopamine activity in appetitive trace conditioning with a constant
CS-US interval, including probe trials with unexpected delays. The proposed
model learns a representation of the environment dynamics in an adaptive
biologically plausible framework, without recourse to delay lines or other
special-purpose circuits. Instead, the model predicts that the
task-dependent representation of time is learned by experience, is encoded
in ramp-like changes in single-neuron activity distributed across small
neural networks, and reflects a temporal integration mechanism resulting
from the inherent dynamics of recurrent loops within the network. The model
also reproduces the known finding that trace conditioning is more difficult
than delay conditioning and that the learned representation of the task can
be highly dependent on the types of trials experienced during training.
Finally, it suggests that the phasic dopaminergic signal could facilitate
learning in the cortex. 

-------------------------------------------------------------------
Francois Rivest, M.Sc.
Candidat au PhD en Informatique (w/Neuroscience) 
Laboratoire d’Informatique des Systèmes Adaptatifs 
Groupe de Recherche sur le Système Nerveux Central Université de Montréal 
http://www-etud.iro.umontreal.ca/~rivestfr/wordpress