shifts

[Juergen Schmidhuber]

Jerry Feldman writes:
>>>
Shift invariance is the ability  of a neural system to recognize a 
pattern independent of where appears on the retina. It is generally 
understood that this property can not be learned by neural network 
methods, but I have not seen a published proof. [...] It is easy to 
show that no local rule can learn shift invariance. [...]  The one 
dimensional case of shift invariance can be handled by treating each 
string as a sequence and learning a finite-state acceptor. But the 
methods that work for this are not local or biologically plausible 
and don't extend to two dimensions.
<<<

It might be of interest to note that the situation changes if the
neural system includes a controller that is able to generate retina-
movements (to change the position of the image on the retina).  There 
are gradient-based controllers that (in certain cases) can *learn*
appropriate, 2-dimensional retina shifts. They are `local' to the extent
backprop through time is `local'. See, e.g., Schmidhuber & Huber (1991):
Learning to generate fovea trajectories for target detection. Int. Journal
of Neural Systems, 2(1 & 2):135-141.

Juergen Schmidhuber, IDSIA