sparse coding

[Bruno A. Olshausen]

The following paper is available via

	http://redwood.psych.cornell.edu/bruno/papers.html
or
	ftp://redwood.psych.cornell.edu/pub/papers/sparse-coding.ps.Z


	       Sparse coding of natural images produces
	    localized, oriented, bandpass receptive fields

		Bruno A. Olshausen and David J. Field

		 Department of Psychology, Uris Hall
			  Cornell University
			Ithaca, New York 14853

The images we typically view, or natural scenes, constitute a
minuscule fraction of the space of all possible images.  It seems
reasonable that the visual cortex, which has evolved and developed to
effectively cope with these images, has discovered efficient coding
strategies for representing their structure.  Here, we explore the
hypothesis that the coding strategy employed at the earliest stage of
the mammalian visual cortex maximizes the sparseness of the
representation.  We show that a learning algorithm that attempts to
find linear sparse codes for natural scenes will develop receptive
fields that are localized, oriented, and bandpass, much like those in
the visual system.  These receptive fields produce a more efficient
image representation for later stages of processing because sparseness
reduces the entropies of individual outputs, which in turn reduces the
redundancy due to complex statistical dependencies among unit
activities.