Connectionists: Neural circuit for sparse coding using optimized matching pursuit

[Fritz Sommer]

A new paper is available that describes a neural network for sparse  
coding of visual input principled on optimized matching pursuit:

Title:
A network that uses few active neurones to code visual input predicts  
the diverse shapes of cortical receptive fields
by M. Rehn and F. T. Sommer

Abstract:
Computational models of primary visual cortex have demonstrated that  
principles of efficient coding and neuronal sparseness can explain  
the emergence of neurones with localised oriented receptive fields.  
Yet, existing models have failed to predict the diverse shapes of  
receptive fields that occur in nature. The existing models used a  
particular “soft” form of sparseness that limits average neuronal  
activity. Here we study models of efficient coding in a broader  
context by comparing soft and “hard” forms of neuronal sparseness.
As a result of our analyses, we propose a novel network model for  
visual cortex. The model forms efficient visual representations in  
which the number of active neurones, rather than mean neuronal  
activity, is limited. This form of hard sparseness also economises  
cortical resources like synaptic memory and metabolic energy.  
Furthermore, our model accurately predicts the distribution of  
receptive field shapes found in the primary visual cortex of cat and  
monkey.

If your institution has access to Journal of Computational  
Neuroscience, you may view your paper at: http://dx.doi.org/10.1007/ 
s10827-006-0003-9
Otherwise the preprint can be downloaded: http://redwood.berkeley.edu/ 
~fsommer/papers/rehnsommer06pre-jcn.pdf
------------------------------------------------------------------------ 
----------------------------------------------
A related recent paper in Neurocomputing explores connections between  
sparse sensory coding and sparse associative memory:

Title:
Storing and restoring visual input with collaborative rank coding and  
associative memory.
by M. Rehn and F. T. Sommer

Abstract:
Associative memory in cortical circuits has been held as a major  
mechanism for content-addressable memory. Hebbian synapses implement  
associative memory efficiently when storing sparse binary activity  
patterns. However, in models of sensory processing, representations  
are graded and not binary. Thus, it has been an unresolved question  
how sensory computation could exploit cortical associative memory.
Here we propose a way how sensory processing could benefit from  
memory in cortical circuitry. We describe a new collaborative method  
of rank coding for converting graded stimuli, such as natural images,  
into sequences of synchronous spike volleys. Such sequences of sparse  
binary patterns can be efficiently processed in associative memory of  
the Willshaw type. We evaluate storage capacity and noise tolerance  
of the proposed system and demonstrate its use in cleanup and fill-in  
for noisy or occluded visual input.

The preprint can be downloaded: http://redwood.berkeley.edu/~fsommer/ 
papers/rehnsommer06neurocomp.pdf
------------------------------------------------------------------------ 
------------------------------------------------

Best,

Fritz Sommer
Redwood Center for Theoretical Neuroscience
University of California, Berkeley
http://redwood.berkeley.edu/wiki/Fritz_Sommer