DC Proximal Newton for Non-Convex Optimization Problems

Alain Rakotomamonjy (1) , Remi Flamary (2, 3) , Gilles Gasso (4)

1 DocApp - LITIS - Equipe Apprentissage
2 LAGRANGE - Joseph Louis LAGRANGE
3 OCA - Observatoire de la Côte d'Azur
4 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes

We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consists in obtaining a descent direction from an approximation of the loss function and then in performing a line search to ensure sufficient descent. A theoretical analysis is provided showing that the iterates of the proposed algorithm {admit} as limit points stationary points of the DC objective function. Numerical experiments show that our approach is more efficient than current state of the art for a problem with a convex loss functions and non-convex regularizer. We have also illustrated the benefit of our algorithm in high-dimensional transductive learning problem where both loss function and regularizers are non-convex.

Mots clés: proximal Newton; non-convex regularization; sparse logistic regression; Difference of convex functions; sparse logistic regression.

Domaines: Apprentissage [cs.LG]; Informatique

Alain Rakotomamonjy, Remi Flamary, Gilles Gasso. DC Proximal Newton for Non-Convex Optimization Problems. IEEE Transactions on Neural Networks, 2016. ⟨hal-00952445v3⟩