shift invariance

[hicks@cs.titech.ac.jp]

jfeldman at ICSI.Berkeley.EDU Wed, 21 Feb 1996 08:12:42 wrote::
>Shift invariance is the ability  of a neural system to recognize a pattern
>independent of where appears on the retina. It is generally understood that
>this property can not be learned by neural network methods

I disagree.  As you state later, the network of neurons need only "share" 
weights.  Sharing can be forced, or can occur by independent learnign of the 
same (but translated) data.

> The unlearnability of shift invarince is not a problem in practice because
>people use preprocessing, weight sharing or other techniques to get shift
>invariance where it is known to be needed. However, it does pose a problem for
>the brain and for theories that are overly dependent on learning.

There are two obvious ways in which shift invariance could occur in 
"biological" or other learning systems.  

1) Nature

Some of part of the patterns of connectivity in the low level vision system
are decided genetically and replicated automatically (like many cellular
structures throughout the boody); in effect, a kind of weight sharing.
Natural selection (learning through genetics) favors patterns of connectivity
which will detect frequently appearing patterns; in effect weight learning.

2)  Nurture

The strengths of neuronal connections in the low level vision system 
self-adjust in some Hebbian scheme to frequently occuring patterns.  
The distribution of these patterns, in actual fact, is shift invariant.
(If they aren't shift invariant then there's not much point in learning 
them as though they were shift invariant.)


Respectfully Yours,

	Craig Hicks