Efficient Primal-Dual Algorithms for Large-Scale Multiclass Classification

Dmitry Babichev 1, 2 Dmitrii Ostrovskii 1, 2 Francis Bach 1, 2
1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We develop efficient algorithms to train $\ell_1$-regularized linear classifiers with large dimensionality $d$ of the feature space, number of classes $k$, and sample size $n$. Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several ideas: (i) passing to the equivalent saddle-point problem with a quasi-bilinear objective; (ii) applying stochastic mirror descent with a proper choice of geometry which guarantees a favorable accuracy bound; (iii) devising non-uniform sampling schemes to approximate the matrix products. In particular, for the multiclass hinge loss we propose a \textit{sublinear} algorithm with iterations performed in $O(d+n+k)$ arithmetic operations.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02012569
Contributor : Dmitrii Ostrovskii <>
Submitted on : Friday, February 8, 2019 - 5:58:59 PM
Last modification on : Saturday, February 16, 2019 - 1:23:33 AM
Long-term archiving on : Thursday, May 9, 2019 - 4:01:36 PM

Files

sublinear-eprint.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02012569, version 1
  • ARXIV : 1902.03755

Collections

Citation

Dmitry Babichev, Dmitrii Ostrovskii, Francis Bach. Efficient Primal-Dual Algorithms for Large-Scale Multiclass Classification. 2019. ⟨hal-02012569⟩

Share

Metrics

Record views

50

Files downloads

74