RBFs and Generalization

[Thomas H. Hildebrandt]

Begin Wolpert Quote ----------

   Date: Mon, 2 Sep 91 15:20:26 MDT
   From: David Wolpert <dhw at t13.Lanl.GOV>


   John Kruschke writes:

   "One motive for using RBFs has been the promise of better interpolation
   between training examples (i.e., better generalization). 
   "

   Whether or not RBFs result in "better interpolation" is one issue.
   Whether or not they result in "better generalization" is another.
   For some situations the two issues are intimately related, and
   sometimes even identical.
   For other situations they are not.



   David Wolpert (dhw at tweety.lanl.gov)

End Wolpert Quote ----------

I have come to treat interpolation and generalization as the same
animal, since obtaining good generalization is a matter of
interpolating in the right metric space (i.e. the one that best models
the underlying process).

If one obtains good results using RBFs, he may assume that the
underlying metric space is well represented by some combination of
hyperspheres.  If he obtains good results using sigmoidally scaled
linear functionals, he may assume that the underlying metric space is
well represented by some combination of sigmoidal sheets.

If the form of the underlying metric space is unknown, then it is a
toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any
number of other basis functions will work best.

				Thomas H. Hildebrandt
				Visiting Researcher
				CSEE Dept.
				Lehigh University