a technical report

[Duan Kaibo]

Dear Connectionists:
 
We have recently completed a technical report that evaluates some simple
performance measures for tuning hyperparameters of Support Vector Machines.
A pdf file containing this report can be downloaded from: 
 <http://guppy.mpe.nus.edu.sg/~mpessk/comparison.shtml>
http://guppy.mpe.nus.edu.sg/~mpessk/comparison.shtml 
 
Here are the details of the report... 
__________________________________________________________________
 
Title: Evaluation of Simple Performance Measures for Tuning SVM
Hyperparameters
 
Authors: 
                Kaibo Duan ( engp9286 at nus.edu.sg
<mailto:engp9286 at nus.edu.sg> )
                S. Sathiya Keerthi ( mpessk at nus.edu.sg
<mailto:mpessk at nus.edu.sg> )
                Aun Neow Poo ( mpepooan at nus.edu.sg
<mailto:mpepooan at nus.edu.sg> )
 
Abstract:
 
Choosing optimal hyperparameter values for support vector machines is an
important step in SVM design. This is usually done by minimizing either an
estimate of generalization error or some other related performance measure.
In this paper, we empirically study the usefulness of several simple
performance measures that are inexpensive to compute (in the sense that they
do not require expensive matrix operations involving the kernel matrix). The
results point out which of these measures are adequate functionals for
tuning SVM hyperparameters. For SVMs with L1 soft margin formulation, none
of the simple measures yields a performance uniformly as good as k-fold
cross validation; Joachims' Xi-Alpha bound and Wahba et al's GACV come next
and perform reasonably well. For SVMs with L2 soft margin formulation, the
radius margin bound gives a very good prediction of optimal hyperparameter
values. 
__________________________________________________________________
 
 
We are  interested in knowing about the comparitive performance of the
measures that we have considered, on other data sets that we haven't tried. 
 
Best regards,
 
Kaibo