Contextual Bandit for Active Learning: Active Thompson Sampling

Djallel Bouneffouf (1, 2, 3) , Romain Laroche (4) , Tanguy Urvoy (4) , Raphael Féraud (4) , Robin Allesiardo (5, 4)

1 TSP - INF - Département Informatique
2 SAMOVAR - Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux
3 CNRS - Centre National de la Recherche Scientifique
4 Orange Labs [Lannion]
5 TAO - Machine Learning and Optimisation

The labelling of training examples is a costly task in a supervised classi cation. Active learning strategies answer this problem by selecting the most useful unlabelled examples to train a predictive model. The choice of examples to label can be seen as a dilemma between the exploration and the exploitation over the data space representation. In this paper, a novel active learning strategy manages this compromise by modelling the active learning problem as a contextual bandit problem. We propose a sequential algorithm named Active Thompson Sampling (ATS), which, in each round, assigns a sampling distribution on the pool, samples one point from this distribution, and queries the oracle for this sample point label. Experimental comparison to previously proposed active learning algorithms show superior performance on a real application dataset.

Mots clés: Contextual Bandits; Active learning; Thompson sampling

Domaines: Intelligence artificielle [cs.AI]

Djallel Bouneffouf, Romain Laroche, Tanguy Urvoy, Raphael Féraud, Robin Allesiardo. Contextual Bandit for Active Learning: Active Thompson Sampling. 2014. ⟨hal-01069802⟩