Connectionists: PhD thesis available: predictive sequence learning with spiking neurons using rank order coding

[Joy Bose]

Dear connectionists,

This is to announce that my PhD dissertation on the topic of sequence
learning is available for download.

Title: Engineering a sequence machine out of spiking neurons employing rank
order codes

Download URL: http://www.cs.man.ac.uk/~bosej/JoyBose_PhD.pdf (216 pages,
5.23 MB)

With regards,
Joy Bose
APT Research Group
Computer Science
The University of Manchester
Email: bosejATcs.man.ac.uk, joyboseroyATgmail.com
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Abstract:
Sequence memories play an important role in biological systems. For example,
the mammalian brain continuously processes, learns and predicts
spatio-temporal sequences of sensory inputs. The work described in this
dissertation demonstrates how a sequence memory may be built from
biologically plausible spiking neural components. The memory is incorporated
in a sequence machine, an automaton that can perform on-line learning and
prediction of sequences of symbols.

The sequence machine comprises an associative memory which is a variant of
Pentti Kanerva's Sparse Distributed Memory, together with a separate memory
for storing the sequence context or history. The associative memory has at
its core a scalable correlation matrix memory employing a localised learning
rule which can be implemented with spiking neurons.

The symbols constituting a sequence are encoded as rank-ordered N-of-M
codes, each code being implemented as a burst of spikes emitted by a layer
of neurons. When appropriate neural structures are used the spike bursts
maintain coherence and stability as they pass through successive neural
layers. The system is modelled using a representation of order that
abstracts time, and the abstracted system is shown to perform equivalently
to a low-level spiking neural system. The spiking neural implementation of
the  sequence memory
model highlights issues that arise when engineering high-level systems with
asynchronous spiking neurons as building blocks.

Finally, the sequence learning framework is used to simulate different
sequence machine models. The new model proposed here is tested under varied
parameters to characterise its performance in terms of the accuracy of its
sequence predictions.