exact shift-invariance from position-independent weights

[Kechen Zhang]

People often do not realize that it is actually possible to get 
shift-invariant responses from position-dependent
weight patterns.  The mechanism may seem counter-intuitive
at the first sight, but the shift-invariance can be rigorously true.

The story begins with the puzzling behaviors of the neurons in
the visual area MST of macaque monkeys. 
For example, some neurons responded very well to a disk rotating
clockwise on a screen no matter where the center of the disk was 
located.  The same neurons would be inhibited if the disk rotated 
counterclockwise, once again, no matter where the disk was 
located on the screen.  Of course, some other cells would prefer 
counterclockwise rotations to clockwise ones, also in 
a shift-invariant manner.  (The same is true for many 
dilation/contraction neurons, and probably also for spiral neurons.)
Recall that MST is just the next processing stage after area MT,
where neurons respond typically to translational movements in a
comparatively small region (receptive field).

One might guess that some nonlinear, higher-order process is
underlying the phenomenon.  But brain has probably found a much 
simpler and more elegant solution.  The plausible solution first 
emerged in a computer simulation experiment by Marty and Margaret Sereno.  
I helped to formalize their findings (hence this message).  
Poggio and colleagues independently arrived at similar conclusion 
via a different path.  In short, rigorously shift-invariant responses 
can be obtained from a simple linear feedforward network whose weight 
pattern is not shift-invariant at all.  The shift-invariance follows
from what Poggio et al. called the Green theorems and we called
the Gauss and Stokes theorems---all special cases of the general
Stokes theorem, which can transform an integral along a closed 
curve into an integral over an area, and vice versa.  
Because the learned weight pattern (considered vector field) has 
a constant curl, the final response depend only on the area of 
that rotating disk.

I think this is a nice example of a counter-intuitive 
neural mechanism for exact shift-invariance.

References: 

[1] Sereno, M. I. and Sereno , M. E. (1991)  Learning to see 
rotation and dilation with a Hebb rule.  In: Advances in 
Neural Information Processing Systems, R. P. Lippman, J. Moody 
and D. S. Touretzky, eds. pp. 320-326.  Morgan Kauffman,
San Mateo, CA.

[2] Zhang, K., Sereno, M. I. and Sereno , M. E. (1993)
Emergence of position-independent detectors of sense of rotation
and dilation with Hebbian learning: an analysis.
Neural Computation 5: 597-612.

[3] Poggio, T., Verri, A. and Torre, V. (1991) Green theorems
and qualitative properties of optical flow.  MIT A.I. Memo,
no. 1289.


-Kechen

________________________________________

Kechen Zhang
Department of Cognitive Science
University of California, San Diego
La Jolla, CA 92093-0515

kzhang at cogsci.ucsd.edu
________________________________________