local receptive fields

[KRUSCHKE,JOHN,PSY]

> Date: Tue, 30 Oct 90 09:27:36 PST
> From: Tom Dietterich <tgd at turing.CS.ORST.EDU>
> Subject: Local receptive fields
> 
> I am confused by what appear to be two different usages of the term
> "local receptive fields", and I wonder if anyone can un-confuse me.
> 
> In papers about radial basis functions (e.g., Moody and Darken, Poggio
> and Girosi, etc.) the (single) layer of hidden units are described as
> having local receptive fields.  ...
>
> On the other hand, in papers such as those by Waibel et al on phoneme
> recognition or by LeCun et al on handwritten digit recognition, the
> hidden units have connections to only a few of the input units.  These
> hidden units are also described as having local receptive fields.
> 
> From a nervous system point of view, it seems to me that the second
> usage is more correct than the first.  I think we need a better term for
> describing the kind of locality exhibited by RBF networks. 

There is an important difference between the intrinsic topologies of the input
layers in the two situations Dietterich describes. 

In RBF networks, the input nodes are usually assumed to each represent entire
dimensions of variation.  So if there are N input nodes, then the input space
is N-dimensional.  The RBF nodes at the hidden layer are responsive to only a
local region of the N-dimensional input space. 

On the other hand, in the situations that Dietterich describes as more
"neural", the input nodes are not usually assumed to each individually
represent entire dimensions of variation.  For example, each node might
represent a small region of a 2-D image, so that the entire ensemble of N input
nodes actually represents just a 2-D input space.  In such a space there is an
intrinsic topology so that some nodes are closer to each other than others,
whereas in the input space of the RBF network, no input nodes are closer to
each other than any others.

So, both types of "localization in space" make good sense, but in their own
representations of space. 

--John Kruschke