4 vs 3 layers -- Tech Report available from connectionists archive

[sontag@control.rutgers.edu]

    REPORT AVAILABLE ON CAPABILITIES OF FOUR-LAYER vs THREE-LAYER NETS

At the request of a few people at NIPS, I placed in the connectionists archive
the postscript version of my report describing why TWO hidden layers are
sometimes necessary when solving function-approximation types of problems, a
fact that was mentioned in my poster.  (About 1/2 of the report deals with the
general question, while the other half is devoted to the application to
control that led me to this.)  Below are the abstract and instructions on ftp
retrieval.

I would very much welcome any discussion of the practical implications --if
any-- of the result.  If you want, send email to me and I can summarize later
for the net.

Happy palindromic year to all,
-eduardo

-----------------------------------------------------------------------------

Report SYCON-90-11, Rutgers Center for Systems and Control, October 1990

      FEEDBACK STABILIZATION USING TWO-HIDDEN-LAYER NETS

This report compares the representational capabilities of three-layer
(that is, "one hidden layer") and four-layer ("two hidden layer") nets
consisting of feedforward interconnections of linear threshold units.

It is remarked that for certain problems four layers are required, contrary to
what might be in principle expected from the known approximation theorems.
The differences are not based on numerical accuracy or number of units needed,
nor on capabilities for feature extraction, but rather on a much more basic
classification into "direct" and "inverse" problems.  The former correspond to
the approximation of continuous functions, while the latter are concerned with
approximating one-sided inverses of continuous functions ---and are often
encountered in the context of inverse kinematics determination or in control
questions.

A general result is given showing that nonlinear control systems can be
stabilized using four layers, but not in general using three layers.

-----------------------------------------------------------------------

To obtain copies of the postscript file, please use Jordan Pollack's service:

Example:
unix> ftp cheops.cis.ohio-state.edu          # (or ftp 128.146.8.62)
Name (cheops.cis.ohio-state.edu:): anonymous
Password (cheops.cis.ohio-state.edu:anonymous): <ret>
ftp> cd pub/neuroprose
ftp> binary
ftp> get
(remote-file) sontag.twolayer.ps
(local-file) twolayer.ps.Z
ftp> quit
unix> uncompress twolayer.ps
unix> lpr -P(your_local_postscript_printer) twolayer.ps

----------------------------------------------------------------------------
If you have any difficulties with the above, please send e-mail to
sontag at hilbert.rutgers.edu.   DO NOT "reply" to this message, please.