|
NeuroCOLT
Technical Report NC-TR-97-013
A PAC Analysis
of a Bayesian Estimator
John
Shawe-Taylor
Royal Holloway
University of London, UK
Robert
Williamson
Australian National University
Australia
Abstract
Bayesian analysis of generalization can place a prior distribution
on the hypotheses and estimate the volume of this space that is consistent
with the training data. The larger this volume the greater the confidence
in the classifier obtained. The key feature of such estimators is
that they provide a posteriori estimates of generalization based on
properties of the hypothesis and the training data. This contrasts
with a `classical' PAC analysis which provides only a priori (worst
case) bounds. Following earlier results showing that Data-sensitive
analysis of generalization in the PAC sense is possible, the paper
uses the techniques to give the first PAC style analysis of a Bayesian
inspired estimator of generalization. The estimator concerned is the
size of a ball which can be placed in the consistent region of parameter
space. The ball gives a lower bound on the volume of parameter space
consistent with the training set. The larger the ball the better the
bound on the generalization obtained. In all cases the bounds are
of good generalization with high confidence, hence bounding the tail
of the distribution of generalization errors that might occur.
The
resulting bounds are independent of the complexity of the function
class though they depend weakly on the dimensionality of the parameter
space.
Download Compressed Postscript
|