No subject

[Yi Lin]

TR Support Vector Machines and the Bayes Rule in Classification: available via
http://www.stat.wisc.edu/~yilin

Support Vector Machines and the Bayes Rule in Classification

by Yi Lin

UW-Madison statistics department TR 1014, November 1, 1999.

The Bayes rule is the optimal classification rule if the underlying
distribution of the data is known. In practice we do not know the
underlying distribution, and need to ``learn'' classification rules
from the data. One way to derive classification rules in practice is
to implement the Bayes rule approximately by estimating an appropriate
classification function. Traditional statistical methods use estimated
log odds ratio as the classification function. Support vector machines
(SVMs) are one type of large margin classifier, and the relationship
between SVMs and the Bayes rule was not clear. In this paper, it is
shown that SVMs implement the Bayes rule approximately by targeting at
some interesting classification functions. This helps understand the
success of SVMs in many classification studies, and makes it easier to
compare SVMs and traditional statistical methods.