Fast boosting using adversarial bandits - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2010

Fast boosting using adversarial bandits

Résumé

In this paper we apply multi-armed bandits (MABs) to improve the computational complexity of AdaBoost. AdaBoost constructs a strong classifier in a stepwise fashion by selecting simple base classifiers and using their weighted ''vote'' to determine the final classification. We model this stepwise base classifier selection as a sequential decision problem, and optimize it with MABs where each arm represents a subset of the base classifier set. The MAB gradually learns the ''usefulness'' of the subsets, and selects one of the subsets in each iteration. AdaBoost then searches only this subset instead of optimizing the base classifier over the whole space. The main improvement of this paper over a previous approach is that we use an adversarial bandit algorithm instead of stochastic bandits. This choice allows us to prove a weak-to-strong-learning theorem, which means that the proposed technique remains a boosting algorithm in a formal sense. We demonstrate on benchmark datasets that our technique can achieve a generalization performance similar to standard AdaBoost for a computational cost that is an order of magnitude smaller.
Fichier principal
Vignette du fichier
Busa-Fekete.pdf (1.01 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

in2p3-00614564 , version 1 (12-08-2011)

Identifiants

  • HAL Id : in2p3-00614564 , version 1

Citer

R. Busa-Fekete, Balázs Kégl. Fast boosting using adversarial bandits. 27th International Conference on Machine Learning (ICML 2010), Jun 2010, Haifa, Israel. pp.143-150. ⟨in2p3-00614564⟩
153 Consultations
176 Téléchargements

Partager

Gmail Facebook X LinkedIn More