NIPS*96 post-conference workshop om Model Complexity

[Chris Williams]

Note the call for short presentations near the bottom of this message.

                   NIPS*96 Post-conference Workshop 

                          MODEL COMPLEXITY

                   Snowmass (Aspen), Colorado USA
                        Friday Dec 6th, 1996


ORGANIZERS:      
      Chris Williams (Aston University, UK, c.k.i.williams at aston.ac.uk)  
      Joachim Utans (London Business School, UK, J.Utans at lbs.lon.ac.uk)

OVERVIEW:

One of the most important difficulties in using neural networks for
real-world problems is the issue of model complexity, and how it
affects the generalization performance.

One approach states that model complexity should be tailored to the
amount of training data available, e.g. by using architectures with
small numbers of adaptable parameters, or by penalizing the fit of
larger models (e.g. AIC, BIC, Structural Risk Minimization, GPE).
Alternatively, computationally expensive numerical estimates of the
generalization performance (cross-validation (CV), Bootstrap, and
related methods) can be used to compare and select models (for example
Moody and Utans, 1994). Methods based on regularization restrict model
complexity by reducing the "effective" number of parameters (Moody
1992).

On the other hand, Bayesian methods see no need to limit model
complexity, as overfitting is obviated by marginalization, where
predictions are made by averaging over the posterior weight
distribution. As Neal (1995) has argued, there may be no reason to
believe that neural network models for real-world problems should be
limited to nets containing only a "small" number of hidden units. In
the limit of an infinite number of hidden units neural networks become
Gaussian processes, and hence are closely related to the splines
approach (Wahba, 1990).

Another important aspect of model building is the selection of a
subset of relevant input variables to include in the model, for
instance, in a regression context, the subset of independent
variables, or lagged values for a time series problem.

The aim of this workshop is to present the different ideas on these
topics, and to provide guidance to those confronted with the problem
of model complexity on real-world problems.
 

SPEAKERS:

   Leo Breiman           (University of California Berkeley) 
   Federico Girosi       (MIT) 
   Trevor Hastie         (Stanford) 
   Michael Kearns        (AT&T Laboratories Research)
   John Moody            (Oregon Graduate Institute) 
   Grace Wahba           (University of Wisconsin at Madison) 
   Hal White             (University of California San Diego) 
   Huaiyu Zhu            (Santa Fe Institute) 

WORKSHOP FORMAT:

Of the 6 hours scheduled, about 4 will be taken up with presentations
from the speakers listed above. We also very keen to make sure that
there is time for discussion of the points raised. However, we also
want to provide an opportunity for others to make short presentations
or raise questions; we are considering making available a limited
number of mini-slots of approx. 5-10 minutes (2-3 overheads plus time
for a short discusson) for presentations on relevant topics.

Because the workshop is scheduled for one day only and depending on
the number of proposals received we may schedule the short presentations
to the extend beyond the regular morning session.

CALL FOR PARTICIPATION:

If you would like to make a 5-10 minute presentation please email the 
organizers by Thursday 12 December, giving your name, a title for
your presentation and a short abstract. We will be finalizing the 
program in the following week.

WEB PAGE:
The workshop web page is located at 

http://www.ncrg.aston.ac.uk/nips96/

It includes abstracts for the invited talks.