Generalized Optimal Reverse Prediction
Martha White and Dale Schuurmans. Generalized Optimal Reverse Prediction. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1305–1313, 2012.
Recently it has been shown that classical supervised and unsupervisedtraining methods can be unified as special cases of so-calledoptimal reverse prediction: predicting inputs from target labelswhile optimizing over both model parameters and missing labels.Although this perspective establishes links betweenclassical training principles, the existing formulation only applies to linear predictors under squared loss, hence is extremely limited.We generalize the formulation of optimal reverse prediction to arbitrary Bregman divergences, and more importantly to non-linear predictors. This extension is achievedby establishing a new, generalized form of forward-reverseminimization equivalence that holds for arbitrary matching losses.Several benefits follow.First, a new variant of Bregman divergence clustering can be recoveredthat incorporates a non-linear data reconstruction model.Second, normalized-cut and kernel-based extensions can be formulated coherently.Finally, a new semi-supervised training principle can be recoveredfor classification problems that demonstrates some advantages over the state of the art.
@InProceedings(12aistats-reverse, Title = "Generalized Optimal Reverse Prediction", Author = "Martha White and Dale Schuurmans", Year = "2012", Booktitle = "Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS)", Pages = "1305--1313", AcceptRate = "< 30\%", bib2html_dl_pdf="../publications/12aistats-reverse.pdf" )