Summary (long): pattern recognition comparisons

[Subutai Ahmad]

>But the point is that
>complete connectivity from layer to layer needs O(N**2) links, and the fact that
>"a preprocessing step" reduced the 64x64 array to 80 nodes is a good example of
>how complete connectivity dominates.  Once the preprocessor is handled by the
>net itself it will either need too many links or have ad hoc structure.
>It's surely better to use partial connectivity (e.g., local - which is a very
>general assumption motivated by physical interactions and brain structure)
>than some inevitably ad hoc preprocessing steps of unknown value.

Systems with selective attention mechanisms provide yet another way of
avoiding the combinatorics.  In these models, you can route relevant
feature values from arbitrary locations in the image to a central
processor.  The big advantage is that the central processor can now be
quite complex (possibly fully connected) since it only has to deal
with a relatively small number of inputs.


--Subutai Ahmad
ahmad at icsi.berkeley.edu


References:

Koch, C. and Ullman, S. Shifts in Selective Attention: towards the 
  underlying neural circuitry. Human Neurobiology, Vol 4:219-227,
  1985.

Ahmad, S. and Omohundro, S. Equilateral Triangles: A Challenge for
  Connectionist Vision. In Proceedings of the 12th Annual meeting of the
  Cognitive Science Society, MIT, 1990.

Ahmad, S. and Omohundro, S. A Network for Extracting the Locations of
  Point Clusters Using Selective Attention, ICSI Tech Report 
  No. TR-90-011, 1990.