Learning the Kernel with Hyperkernels
Cheng Soon Ong, Alex Smola and Bob Williamson
Journal of Machine Learning Research
This paper addresses the problem of choosing a kernel suitable for estimation with a
Support Vector Machine, hence further automating machine learning. This goal is achieved
by defining a Reproducing Kernel Hilbert Space on the space of kernels itself. Such a
formulation leads to a statistical estimation problem similar to the problem of minimizing
a regularized risk functional.
We state the equivalent representer theorem for the choice of kernels and present
a semidefinite programming formulation of the resulting optimization problem. Several
recipes for constructing hyperkernels are provided, as well as the details of common ma-
chine learning problems. Experimental results for classification, regression and novelty
detection on UCI data show the feasibility of our approach.