Connectionists: Temporal Sequencing via Colimits

[mjhealy@ece.unm.edu]

A new technical report available via DspaceUNM at
https://repository.unm.edu/dspace/handle/1928/10424 and also on my
website, http://www.ece.unm.edu/~mjhealy/, describes our initial work in
applying category theory to the modeling of temporal sequences in neural
memories.  With our categorical neural semantic theory (CNST), we take an
alternative to most current approaches dealing with temporal sequencing,
for example in forming episodic memories.  We model the buildup of
temporal sequences as the adaptation of neural structures representing
colimits from a concept category mapped to a neural category by a functor.
A more advanced model would express diagrams of functors connected by
natural transformations,  discussed as a general modeling scheme in prior
papers (also available on my website).  Our initial temporal sequencing
architecture incorporates ART networks for convenience, modified by the
categorical colimit model to form what we call "supertemplates".  These
are templates that embrace multiple ART units in commutative  diagrams.

The terms used above such as "functor" are unfamiliar to many, but are
explained in all of our papers on the CNST to date including the current
technical report.

An abstract and keyword list follows:

M. J. Healy and T. P. Caudell (2010)
"Temporal Sequencing via Supertemplates", UNM Technical Report
EECE-TR-10-0001, DspaceUNM, University of New Mexico.

Abstract

A category-theoretic account of neural network semantics has been used to
characterize incremental concept representation in neural memory. It
involves a category of concepts and concept morphisms together with
categories of objects and morphisms representing the activity in
connectionist structures at different stages of weight adaptation.
Colimits express the more specialized concepts as combinations of abstract
concepts along shared subconcept relationships specified in diagrams. This
provides a mathematical model of concept blending, in which designated
relationships among concepts are preserved in a combination.
Structure-preserving mappings called functors from the concept to neural
categories provide a mathematical model of incremental concept
representation through stages of adaptation. The work reported here
extends these ideas to express temporal sequences of events, such as
episodic memories. This requires an extended notion of neural morphism and
a design principle for diagrams involving concepts in a temporal sequence.
This is tested in a new architecture that involves a notion of
supertemplates, which are ART network templates extending over a
multi-level ART hierarchy with an interposed temporal integrator network.

Keywords

ART, category, colimit, concept, connection path, diagram, episode, event,
functor, morphism, neural, semantics, temporal sequence, theory