shift invariance

[Christopher Lee]

>>>>> "Jerry" == Jerry Feldman <jfeldman at ICSI.Berkeley.EDU> writes:

    Jerry>  Shift invariance is the ability of a neural system to
    Jerry> recognize a pattern independent of where appears on the
    Jerry> retina. It is generally understood that this property can
    Jerry> not be learned by neural network methods, but I have not
    Jerry> seen a published proof. A "local" learning rule is one that
    Jerry> updates the input weights of a unit as a function of the
    Jerry> unit's own activity and some performance measure for the
    Jerry> network on the training example. All biologically plausible
    Jerry> learning rules, as well as all backprop variants, are local
    Jerry> in this sense.
 
Jerry's communique has certainly certainly sparked discussion, but I
feel as if his reference to "neural network methods" needs more
precise definition.  Perhaps Jerry could state more specifically the
class of network architectures and neurons he wishes to consider?
(E.g., Minsky and Papert restricted their proof to order one
perceptrons.)  What sort of resource limitations would you put on this
network relative to the complexity of the task?  (To give an absurd
example of why this is important: for a small "test problem"-like
space, if given an appropriate number of nodes a network could simply
"memorize" all the configurations of an object at all
locations. Clearly, this isn't what one would normally considering
"learning" shift invariance.)

On another vein that might be of interest, it's clear that shift
invariance is a fundamental to the primate visual system in some way,
and a fair amount of interest exists in the neurophysiology community
concerning how this problem is solved; one hypothesis involves the
role of attentional mechanisms in scale and translational invariance
(Olshausen, Anderson, Van Essen, J. of Neuroscience.  13(11):4700-19,
1993).  It is not obvious to me that anything along the lines of
Jerry's proof could be applied to their (the Olshausen et al.) network
model. 

Christopher Lee

-- 
Washington University			
Department of Anatomy and Neurobiology	
email: clee at v1.wustl.edu