Bandits Corrupted by Nature: Lower Bounds on Regret and Robust Optimistic Algorithm

Debabrota Basu (1) , Odalric-Ambrym Maillard (1) , Timothée Mathieu (1)

1 Scool - Scool

In this paper, we study the stochastic bandits problem with k unknown heavy-tailed and corrupted reward distributions or arms with time-invariant corruption distributions. At each iteration, the player chooses an arm. Given the arm, the environment returns an uncorrupted reward with probability 1−ε and an arbitrarily corrupted reward with probability ε. In our setting, the uncorrupted reward might be heavy-tailed and the corrupted reward might be unbounded. We prove a lower bound on the regret indicating that the corrupted and heavy-tailed bandits are strictly harder than uncorrupted or light-tailed bandits. We observe that the environments can be categorised into hardness regimes depending on the suboptimality gap ∆, variance σ, and corruption proportion ϵ. Following this, we design a UCB-type algorithm, namely HuberUCB, that leverages Huber's estimator for robust mean estimation. HuberUCB leads to tight upper bounds on regret in the proposed corrupted and heavy-tailed setting. To derive the upper bound, we prove a novel concentration inequality for Huber's estimator, which might be of independent interest.

Mots clés: Unbounded corruption; Heavy-tail distributions; Huber's estimator; Regret bounds

Domaines: Mathématiques [math]; Machine Learning [stat.ML]

Debabrota Basu, Odalric-Ambrym Maillard, Timothée Mathieu. Bandits Corrupted by Nature: Lower Bounds on Regret and Robust Optimistic Algorithm. 2022. ⟨hal-03611816⟩