two papers on interpreting NNs and Guassian process models

[Tony Plate]

The following two papers on interpreting neural networks
and Guassian process models are available for download
from http://www.mcs.vuw.ac.nz/~tap/publications.html

----------

Accuracy versus interpretability in flexible modeling:
implementing a tradeoff using Gaussian process models}

Tony A. Plate
School of Mathematical and Computing Sciences
Victoria University of Wellington, Wellington, New Zealand
Tony.Plate at vuw.ac.nz

To appear in Behaviourmetrika, special issue on Analysis of
knowledge representations in neural network models.

Abstract:
  One of the widely acknowledged drawbacks of flexible
  statistical models is that the fitted models are often
  extremely difficult to interpret.  However, if flexible
  models are constrained to be additive the fitted models are
  much easier to interpret, as each input can be considered
  independently.  The problem with additive models is that
  they cannot provide an accurate model if the phenomenon
  being modeled is not additive.  This paper shows that a
  tradeoff between accuracy and additivity can be implemented
  easily in Gaussian process models, which are a type of
  flexible model closely related to feedforward neural
  networks.  One can fit a series of Gaussian process models
  that begins with the completely flexible and are constrained
  to be progressively more additive, and thus progressively
  more interpretable.  Observations of how the degree of
  non-additivity and the test error change as the models
  become more additive give insight into the importance of
  interactions in a particular model.  Fitted models in the
  series can also be interpreted graphically with a technique
  for visualizing the effects of inputs in non-additive models
  that was adapted from plots for generalized additive models.
  This visualization technique shows the overall effects of
  different inputs and also shows which inputs are involved in
  interactions and how strong those interactions are.

----------

Visualizing the function computed by a feedforward neural network

Tony Plate (Victoria University of Wellington)
Joel Bert (University of British Columbia)
John Grace (University of British Columbia)
Pierre Band (Health Canada)

Computer Science Technical Report CS-TR-98-5
Victoria University of Wellington, Wellington, New Zealand

Abstract:
  A method for visualizing the function computed by a
  feedforward neural network is presented.  It is most
  suitable for models with continuous inputs and a small
  number of outputs, where the output function is reasonably
  smooth, as in regression or probabilistic classification
  tasks.  The visualization makes readily apparent the
  effects of each input and the way in which the functions
  deviates from a linear function.  The visualization can
  also assist in identifying interactions in the fitted
  model.  The method uses only the input-output relationship
  and thus can be applied to any predictive statistical
  model, including bagged and committee models, which are
  otherwise difficult to interpret.  The visualization method
  is demonstrated on a neural-network model of how the risk
  of lung cancer is affected by smoking and drinking.


-- 
Tony Plate, Computer Science                   Voice: +64-4-495-5233 ext 8578
School of Mathematical and Computing Sciences             Fax: +64-4-495-5232
Victoria University, PO Box 600, Wellington, New Zealand    tap at mcs.vuw.ac.nz
http://www.mcs.vuw.ac.nz/~tap