Local receptive fields

[@neural.att.com]

Steve says
> splitting hairs:  tom, I don't think we need new terms to describe
> what is a continuum...

I disagree.  An RBF unit in a network with 1024 inputs will have 1024 input
lines (and therefore at least 1024 parameters). Now assume these 1024 inputs
are actually the pixels of a 32x32 image....what is local there? certainly not
the connections.

Now here is a really confusing situation: imagine an image-recognition network
with 32x32 inputs (pixels). Suppose that the units in the first layer are
connected to, say, local (2D) 5x5 patches on the input (receptive field
concept #1). Now imagine that these units are gaussian RBF units. Their
activation region (receptive field concept #2) is a hypersphere (or something
similar) in the 25-dimensional space of their input (5x5=25). If these units
were sigmoid units, their activation region (receptive field concept #2) would
be a half space in the 25-D space.

As you see concept #1 and concept #2 are completely orthogonal, and can be
used independently, or even combined. They, therefore, should have different
names.

  -- Yann


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
End of returned mail