Variable Selection, Multicategory SVM's

[Grace Wahba]

The following papers are available via 
http://www.stat.wisc.edu/~wahba  -> TRLIST

         TR 1042
Variable Selection via Basis Pursuit for Non-Gaussian Data

Hao Zhang, Grace Wahba,Yi Lin,
Meta Voelker, Michael Ferris
Ronald Klein and Barbara Klein

         Abstract
A simultaneous flexible variable selection procedure is proposed by 
applying a basis pursuit method to the likelihood function. The basis 
functions are chosen to be compatible with variable selection in the 
context of smoothing spline ANOVA models. Since it is a generalized 
LASSO-type method, it enjoys the favorable property of shrinking 
coefficients and gives interpretable results. We derive a Generalized 
Approximate Cross Validation function (GACV), which is an approximate 
leave-out-one cross validation function used to choose smoothing 
parameters. In order to apply the GACV function for a large data set 
situation, we propose a corresponding randomized GACV. A technique 
called `slice modeling' is used to develop an efficient code. Our 
simulation study shows the effectiveness of the proposed approach 
in the Bernoulli case.

         TR 1043
Multicategory Support Vector Machines

Yoonkyung Lee, Yi Lin and Grace Wahba

         Abstract
The Support Vector Machine (SVM) has shown great performance
in practice as a classification methodology.
Oftentimes multicategory problems have been 
treated as a series of binary problems in the SVM paradigm. 
Even though the SVM implements the optimal classification rule
asymptotically in the binary case, solutions to a series of binary problems 
may not be optimal for the original multicategory problem.
We propose multicategory SVMs, which extend the binary SVM to the
multicategory case, and encompass the binary SVM as a special case.
The multicategory SVM implements the optimal classification rule 
as the sample size gets large, overcoming the suboptimality of
the conventional one-versus-rest approach.  
The proposed method deals with the equal misclassification cost and
the unequal cost case in unified way. (Long version of TR 1040)