|
NeuroCOLT
Technical Report NC-TR-00-082
On
the Generalisation of Soft Margin Algorithms
John Shawe-Taylor
Nello Cristianini
Abstract
Generalisation bounds depending on the margin of a classifier are
a relatively recent development. They provide an explanation of the
performance of state-of-the-art learning systems such as Support Vector
Machines (SVM) and Adaboost. The difficulty with these bounds has been
either their dependence on the minimal margin or their agnostic form.
The paper presents a technique for correcting those points
that fail to meet a target margin, hence creating a large margin
classifier at the expense of additional functional complexity. Analysis
of this technique leads to bounds that motivate the previously
heuristic soft margin SVM algorithms as well as justifying the use of
the quadratic loss in neural network training algorithms. The results
are extended to give bounds for the probability of failing to achieve a
target accuracy in regression prediction.
Download
Compressed Postscript
|