nonparametric reg / nonlin feature extraction

[Edward Malthouse]

The following dissertation is available via anonymous FTP:

Nonlinear Partial Least Squares

By Edward C. Malthouse

Key words: nonparametric regression, partial least squares (PLS),
principal components regression (PCR), projection pursuit regression
(PPR), feedforward neural networks, nonlinear feature extraction,
principal components analysis (PCA), nonlinear principal components
analysis (NLPCA), principal curves and surfaces.

                 A B S T R A C T

We propose a new nonparametric regression method for
high-dimensional data, nonlinear partial least squares (NLPLS). 
NLPLS is motivated by projection-based regression methods, e.g.,
partial least squares (PLS), projection pursuit (PPR), and
feedforward neural networks.  The model takes
the form of a composition of two functions. The first function in the
composition projects the predictor variables onto a lower-dimensional
curve or surface yielding scores, and the second
predicts the response variable from the scores. 
We implement NLPLS with feedforward neural networks.  NLPLS will
often produce a more parsimonious model (fewer score vectors) than
projection-based methods, and the model is well suited for detecting
outliers and future covariates requiring extrapolation.  The scores
are also shown to have useful interpretations. 
We also extend the model for multiple response variables and
discuss situations when multiple response variables should be
modeled simultaneously and when they should be modeled with separate
regressions.  We provide empirical results from mathematical and
chemical engineering examples which evaluate the performances of
PLS, NLPLS, PPR, and three-layer neural networks on (1) response
variable predictions, (2) model parsimony, (3) computational
requirements, and (4) robustness to starting values.

The curves and surfaces used by NLPLS are motivated by
the nonlinear principal components analysis (NLPCA) method of
doing nonlinear feature extraction.  We develop certain properties of
NLPCA and discuss its relation to the principal curve method. 
Both methods attempt to reduce the dimension of a set of multivariate
observations by fitting a curve through the middle of
the observations and projecting the observations onto this
curve.  The two methods fit their models under a similar
objective function, with one important difference:  NLPCA
defines the function which maps observed variables to scores
(projection index) to be continuous.  We show that the effects of
this constraint are (1) NLPCA is unable to model curves and surfaces
which intersect themselves and (2) the NLPCA ``projections'' are
suboptimal producing larger approximation error.  We show how NLPCA
score values can be interpreted and give the results of a small
simulation study comparing the two methods.


The dissertation is 120 pages long (single spaced).

ftp mkt2715.kellogg.nwu.edu
logname:  anonymous
password:  your email address
cd /pub/ecm
binary
get dissert.ps.gz
quit
gzip -d dissert.ps
lp -dps dissert.ps   # or however you print postscript

I'm sorry, but no hardcopies are available.