Self-concordant analysis for logistic regression

Francis Bach (1)

1 WILLOW - Models of visual object recognition and scene understanding

Most of the non-asymptotic theoretical work in regression is carried out for the square loss, where estimators can be obtained through closed-form expressions. In this paper, we use and extend tools from the convex optimization literature, namely self-concordant functions, to provide simple extensions of theoretical results for the square loss to the logistic loss. We apply the extension techniques to logistic regression with regularization by the $\ell_2$-norm and regularization by the $\ell_1$-norm, showing that new results for binary classification through logistic regression can be easily derived from corresponding results for least-squares regression.

Mots clés: Convex optimization; sparsity

Domaines: Apprentissage [cs.LG]; Statistiques [math.ST]; Théorie [stat.TH]

Francis Bach. Self-concordant analysis for logistic regression. 2009. ⟨hal-00426227⟩