Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Michael J Healy]

Dear Colleagues,

Tom Caudell and I are developing a theory of the structure-function relationship in neural networks that implies that the brain is organized into parallel hierarchies. We call this theory the Categorical Neural Semantic Theory (CNST).  It is a mathematical model of the correspondence between the structure of knowledge and neural structure.  The CNST is pretty much theoretical at present, but it is consistent with the data we have been able to obtain.  It implies some interesting structure and phenomena.  A part of this is the parallel hierarchy structure, which you may think of in terms of interconnected modules---or not.  Since the theory posits structures which are mathematically-based, it does not assume evolutionary or physiological constraints---it just produces neural structure-to-function relationships based upon some basic neuroscience and takes off from there, and constraints arise ``naturally''.

Each hierarchy corresponds to a brain region associated with a major function, examples being major subdivisions of visual, motor, association, prefrontal and other cortices.  The hierarchies are built up based upon constructs of category theory, principally colimits and limits.  The latter constructs model concepts obtained by combining simpler ones through ``concept blending'' and by abstracting from more complex, specialized concepts according to their common use in more yet more complex concepts.  Starting with some examples of these constructs in its  initial connectionist structure, the brain builds representations incrementally by ``learning'' further colimits and limits, each step building upon constructs formed in previous steps.  The representations are associated with cells that respond to concepts representing phenomena in the sensor-motor environment or, in some regions (such as pre-frontal regions), they can represent internally-generated concepts.  Bundles of connection paths with shared active states represent  concept relationships. 

The interaction of the hierarchies is modelled via natural transformations between functors.  The  functors are structure-preserving mappings of a category of concepts and relationships between them to a category of neural structures.  The natural transformations model the activity of longer-range connections between regions, which consequently obey a rule we call "knowledge coherence".       

Neither the idea of parallel hierarchies on the one hand, or the use of category theory in brain modelling on the other, is unique to our theory.  However, we are unique in the way we combine these ideas and use them in semantic modelling.  We have some publications and technical reports on this accessible from my web site, http://www.ece.unm.edu/~mjhealy/ .  These include 

M. J. Healy and T. P. Caudell (2010) 
Temporal Sequencing via Supertemplates,
UNM Technical Report EECE-TR-10-0001, DspaceUNM, University of New Mexico. 

M. J. Healy, R. D. Olinger, R. J. Young, S. E. Taylor, T. P. Caudell, and K. W. Larson (2009) 
Applying Category Theory to Improve the Performance of a Neural Architecture, 
Neurocomputing, vol. 72, pp. 3158-3173. 

M. J. Healy, T. P. Caudell, and T. E. Goldsmith (2008) 
A Model of Human Categorization and Similarity Based Upon Category Theory,
UNM Technical Report EECE-TR-08-0010, DSpaceUNM, University of New Mexico. 


M. J. Healy and T. P. Caudell (2006a) 
Ontologies and Worlds in Category Theory: Implications for Neural Systems, 
Axiomathes, vol. 16, nos. 1-2, pp. 165-214.