Local receptive fields

[Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU]

    I am confused by what appear to be two different usages of the term
    "local receptive fields", and I wonder if anyone can un-confuse me.

I think that the two usages of the term "local receptive field" are more or
less the same idea, but different input encodings change the
implementation.  In both cases, you've got input units encoding some sort
of N-dimensional space, and you've got hidden (or output) units that
respond only to a localized patch of that hyperspace.  That's the basic
idea.

If you think of each input as being a distinct, continous dimension, then
you end up with something like the Moody and Darken units, which respond to
some hyper-sphere or hyper-ellipsoid around an N-dimensional center point.
On the other hand, if you think the individual units as encoding intervals
or patches in this space (as in the speech networks -- each input is a
little patch of time/frequency space or something like that), then you end
up with hidden units that have inputs from a set of these units.  Of
course, there are a few more degrees of freedom in the latter case: within
the "receptive field", the individual weights can encode something more
complex than a simple Gaussian.

So I think that the term "local receptive field" can cover both cases, but
we need to specify what space we are working in and how the inputs map into
that space: continuous orthogonal dimensions, mosaic encoding, or some
hybrid scheme.

-- Scott