Boosting with diverse base classifiers

S. Dasgupta and P. M. Long. Boosting with diverse base classifiers. Proceedings of the 2003 Conference on Learning Theory.

(Available in Postscript and PDF formats. Software also available.)

Abstract
We establish a new bound on the generalization error rate of the Boost-by-Majority algorithm. The bound holds when the algorithm is applied to a collection of base classifiers that contains a "diverse" subset of "good" classifiers, in a precisely defined sense. We describe cross-validation experiments that suggest that Boost-by-Majority can be the basis of a practically useful learning method, often improving on the generalization of AdaBoost on large datasets.