|
NeuroCOLT
Technical Report NC-TR-96-054
Confidence
Estimates of Classification Accuracy on New Examples
John
Shawe-Taylor
Royal Holloway, University of London
UK
Abstract
Following recent results (NeuroCOLT Technical Report NC-TR-96-053)
showing the importance of the fat shattering dimension in explaining
the beneficial effect of a large margin on generalization performance,
the current paper investigates how the margin on a test example can
be used to give greater certainty of correct classification in the
distribution independent model. The results show that even if the
classifier does not classify all of the training examples correctly,
the fact that a new example has a larger margin than that on the misclassified
examples, can be used to give very good estimates for the generalization
performance in terms of the fat shattering dimension measured at a
scale proportional to the excess margin. The estimate relies on a
sufficiently large number of the correctly classified training examples
having a margin roughly equal to that used to estimate generalization,
indicating that the corresponding output values need to be `well sampled'.
Download Compressed
Postscript
|