Shift Invariance

[Gabriele Scheler]

There should be a difference made between shift-invariance, i.e.
distinguishing between 
T1: {[a,b,c,d,e], [b,c,d,e,a], [c,d,e,a,b]}
T2: {[a,d,b,c,d], [a,d,c,b,e] etc.}
which is more of a purely mathematical problem, and translational invariance,
i.e. detecting a pattern on a plane, no matter where it occurs.
For the latter goal it is sufficient to develop a set of features
in the first layer to detect that pattern in a local field, and
to develop an invariant detector in the next layer, which is ON for
any of the lower-level features. (develop means train for ANN).

In the domain of neural networks the obvious solution to the mathematical
problem would be to train a level of units as sequence encoders:
A1 B1 C1 D1
----- ----
a     b
-------
c
    -------
	d
and classify patterns then on how many of the sequence encoders a-d are ON.
Of course this may be rather wasteful. In another learning approach
called adaptive distance measures, we can reduce training effort considerably
when we use a distance measure which is specifically tuned to problems
of shift invariance. Of course this is nothing else than to have
a class of networks with pre-trained sequence encoders available.
The question here as often is not, which NN can learn this task (backprop
can, Fukushima's Neocognitron can), but
which is most economical in its resources - without requiring
too much knowledge on the type of function to be learned.