|
NeuroCOLT
Technical Report NC-TR-98-019
Generalization
Performance of Regularization Networks and Support Vector Machines
via Entropy Numbers of Compact Operators
Robert C. Williamson
ANU
Alex J. Smola
GMD
Bernhard Sch-olkopf
GMD
Keywords:
ffl-entropy; covering numbers; statistical learning theory; support
vector
machines; linear operators.
Received:
07-JUL-98
Abstract
We derive new bounds for the generalization error of kernel
machines, such as support vector machines and related regularization
networks by obtaining new bounds on their covering numbers. The proofs
make use of a viewpoint that is apparently novel in the field of statistical
learning theory. The hypothesis class is described in terms of a linear
operator mapping from a possibly infinite dimensional unit ball in
feature space into a finite dimensional space. The covering numbers
of the class are then determined via the entropy numbers of the operator.
These numbers, which characterize the degree of compactness of the
operator, can be bounded in terms of the eigenvalues of an integral
operator induced by the kernel function used by the machine. As a
consequence we are able to theoretically explain the effect of the
choice of kernel function on the generalization performance of support
vector machines.
Download Compressed Postscript
|