NeuroCOLT workshop
on
Generalisation Bounds Less than 0.5
Windsor, 29 April - 2 May 2002
Cumberland Lodge

"Learning to Learn"

Andreas Maurer

To facilitate generalization on the basis of very small training sets (a single example per pattern class) a mechanism of transfer learning is proposed, where pattern pairs drawn from a collection of related learning tasks are used to train a preprocessing transformation mapping the input space to a low dimensional representation space. The training criteria is to minimize the Frobenius distance of the empirical instantiation of a standard Gaussian kernel in representation space to the kernel implied by the sample-data for the various tasks. This is shown to approximate the conditional probability that two given pattern belong to the same class for a randomly selected task. Bounds on the estimation error are given for this type of kernel-learning and it is shown how they can be used to derive PAC-bounds for single-example- based classifiers on a new task. It turns out that for transfer- learning it is often better to sample more tasks and fewer examples per task if the total number of training examples is to be held constant. This, as well as the general performance of the proposed mechanism, is experimentally demonstrated in two practical domains of image recognition."