MIT AI Lab memo 1164

[Tomaso Poggio]

the following technical report is available from the MIT AI Lab
Publication Office (send e-mail to liz at ai.mit.edu)

      Networks and the Best Approximation Property

                        by
          Federico Girosi and Tomaso Poggio

ABSTRACT

Networks can be considered as approximation schemes. Multilayer
networks of the backpropagation type can approximate arbitrarily well
continuous functions (Cybenko, 1989; Funahashi, 1989; Stinchcombe and
White, 1989). We prove that networks derived from regularization
theory and including Radial Basis Functions (Poggio and Girosi, 1989,
AI memo 1140), have a similar property.  From the point of view of
approximation theory, however, the property of approximating
continuous functions arbitrarily well is not sufficient for
characterizing good approximation schemes.  More critical is the
property of {\it best approximation}. The main result of this paper is
that multilayer networks, of the type used in backpropagation, are not
best approximation.  For regularization networks (in particular Radial
Basis Function networks) we prove existence and uniqueness of best
approximation.