Technical Report announcement

[Yann le Cun]

Jeff Elman writes:
> This paper rigorously establishes that standard  multi-layer
> feedforward  networks  with as few as one hidden layer using
> arbitrary squashing functions (not  necessarily  continuous)
> at  the  hidden  layer(s)  are  capable of approximating any
> Borel  measurable  function  from  one  Euclidean  space  to
> another  to  any desired degree of accuracy, provided suffi-
> ciently many hidden units are  available.   In  this  sense,
> multi-layer  feedforward  networks  are a class of universal
> approximators.

I showed the same kind of result in my thesis (although probably not as
rigorously). The problem is: if you use monotonic squashing functions, then
you need one more layer (i.e two hidden layers).

reference:
Yann le Cun: "modeles connexionnistes de l'apprentissage" (connectionist
learning models), These de Doctorat, Universite Pierre et Marie Curie 
(Paris 6), June 1987, Paris, France.

- Yann