Preprint on Local Receptive Field Neural Networks

[Michael Schmitt]

Dear Connectionists,

a preprint of the paper

"Neural networks with local receptive fields and superlinear VC
dimension"
by Michael Schmitt

is available on-line from

http://www.ruhr-uni-bochum.de/lmi/mschmitt/receptive.ps.gz
(39 pages gzipped PostScript).

This paper has been accepted by Neural Computation in its first
submitted version. (They say that this is the first time that they have
accepted a first time submission.)

Regards,

Michael Schmitt

------------------------------------------------------------

TITLE: Neural networks with local receptive fields and superlinear VC
  dimension

AUTHOR: Michael Schmitt

ABSTRACT
  Local receptive field neurons comprise such well-known and widely
  used unit types as radial basis function neurons and neurons with
  center-surround receptive field. We study the Vapnik-Chervonenkis
  (VC) dimension of feedforward neural networks with one hidden layer
  of these units. For several variants of local receptive field
  neurons we show that the VC dimension of these networks is
  superlinear. In particular, we establish the bound $\Omega(W\log k)$
  for any reasonably sized network with $W$ parameters and $k$ hidden
  nodes.  This bound is shown to hold for discrete center-surround
  receptive field neurons, which are physiologically relevant models
  of cells in the mammalian visual system, for neurons computing a
  difference of Gaussians, which are popular in computational vision,
  and for standard radial basis function (RBF) neurons, a major
  alternative to sigmoidal neurons in artificial neural networks.  The
  result for RBF neural networks is of particular interest since it
  answers a question that has been open for several years. The results
  also give rise to lower bounds for networks with fixed input
  dimension.  Regarding constants all bounds are larger than those
  known thus far for similar architectures with sigmoidal neurons. The
  superlinear lower bounds contrast with linear upper bounds for
  single local receptive field neurons also derived here.


--
Michael Schmitt
LS Mathematik & Informatik, Fakultaet fuer Mathematik
Ruhr-Universitaet Bochum, D-44780 Bochum, Germany
Phone: +49 234 32-23209 , Fax: +49 234 32-14465
http://www.ruhr-uni-bochum.de/lmi/mschmitt/