Mixability made efficient: Fast online multiclass logistic regression

1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique - ENS Paris, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
2 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : Mixability has been shown to be a powerful tool to obtain algorithms with optimal regret. However, the resulting methods often suffer from high computational complexity which has reduced their practical applicability. For example, in the case of multiclass logistic regression, the aggregating forecaster (Foster et al. (2018)) achieves a regret of $O(\log(Bn))$ whereas Online Newton Step achieves $O(e^B\log(n))$ obtaining a double exponential gain in $B$ (a bound on the norm of comparative functions). However, this high statistical performance is at the price of a prohibitive computational complexity $O(n^{37})$.
Document type :
Conference papers
Domain :

https://hal.archives-ouvertes.fr/hal-03370530
Contributor : Rémi Jézéquel Connect in order to contact the contributor
Submitted on : Friday, October 8, 2021 - 9:37:35 AM
Last modification on : Wednesday, June 8, 2022 - 12:50:06 PM
Long-term archiving on: : Sunday, January 9, 2022 - 6:23:13 PM

Files

MulticlassLogistic.pdf
Files produced by the author(s)

Identifiers

• HAL Id : hal-03370530, version 1
• ARXIV : 2110.03960

Citation

Rémi Jézéquel, Pierre Gaillard, Alessandro Rudi. Mixability made efficient: Fast online multiclass logistic regression. NeurIPS 2021. Thirty-fifth Conference on Neural Information Processing Systems, Dec 2021, Online, France. ⟨hal-03370530⟩

Record views