Papers on Gaussian processes and online learning

[Peter Sollich]

Dear Connectionists,

the following two papers, which I hope may be of interest, are now
available from my web pages:

--------------------------------------------------------------------------

                               Peter Sollich

                  Learning curves for Gaussian processes

http://www.mth.kcl.ac.uk/~psollich/papers/GaussianProcLearningCurveNIPSIX.ps.gz
       (or /~psollich/papers_uncompressed/GaussianProcLearningCurveNIPSIX.ps)

I consider the problem of calculating learning curves (i.e., average
generalization performance) of Gaussian processes used for regression. A
simple expression for the generalization error in terms of the eigenvalue
decomposition of the covariance function is derived, and used as the
starting point for several approximation schemes. I identify where these
become exact, and compare with existing bounds on learning curves; the new
approximations, which can be used for any input space dimension, generally
get substantially closer to the truth.

(In M J Kearns, S A Solla, and D Cohn, editors, Advances in Neural
Information Processing Systems 11, Cambridge, MA. MIT Press. In press.)

--------------------------------------------------------------------------

                 H C Rae, P Sollich, and A C C Coolen

            On-Line Learning with Restricted Training Sets:
           Exact Solution as Benchmark for General Theories

    http://www.mth.kcl.ac.uk/~psollich/papers/HebbOnlineNIPSIX.ps.gz
           (or /~psollich/papers_uncompressed/HebbOnlineNIPSIX.ps)

We solve the dynamics of on-line Hebbian learning in perceptrons exactly,
for the regime where the size of the training set scales linearly with the
number of inputs. We consider both noiseless and noisy teachers. Ouc
calculation cannot be extended to non-Hebbian rules, but the solution
provides a nice benchmark to test more general and advanced theories for
solving the dynamics of learning with restricted training sets.

(In M J Kearns, S A Solla, and D Cohn, editors, Advances in Neural
Information Processing Systems 11, Cambridge, MA. MIT Press. In press.)

--------------------------------------------------------------------------

Any comments and suggestions are welcome. For papers on related topics,
you could also have a look at http://www.mth.kcl.ac.uk/~psollich/publications
for my full publications list.

Merry Christmas!
Peter Sollich

--------------------------------------------------------------------------
 Peter Sollich                               Department of Mathematics
 Phone:  +44 - (0)171 - 873 2875             King's College
 Fax:    +44 - (0)171 - 873 2017             University of London
 E-mail: peter.sollich at kcl.ac.uk             Strand
 WWW:    http://www.mth.kcl.ac.uk/~psollich  London WC2R 2LS, U.K.
--------------------------------------------------------------------------