Gentle Nearest Neighbors Boosting over Proper Scoring Rules

Abstract : Tailoring nearest neighbors algorithms to boosting is an important problem. Recent papers study an approach, UNN, which provably minimizes particular convex surrogates under weak assumptions. However, numerical issues make it necessary to experimentally tweak parts of the UNN algorithm, at the possible expense of the algorithm's convergence and performance. In this paper, we propose a lightweight alternative algorithm optimizing proper scoring rules from a very broad set, and establish formal convergence rates under the boosting framework that surprisingly compete with those known for UNN. It is an adaptive Newton-Raphson algorithm, which belongs to the same lineage as the popular Gentle Adaboost. To the best of our knowledge, no such boosting-compliant convergence rates were previously known for these algorithms. We provide experiments on a dozen domains, including the challenging Caltech and SUN computer vision databases. They display that GNNB significantly outperforms UNN, both in terms of convergence rate and quality of the solution obtained, and GNNB provides a simple and efficient contender to techniques that can be used on very large domains, like stochastic gradient descent -- for which little is known to date. Experiments include a divide-and-conquer improvement of GNNB which exploits the link with proper scoring rules optimization.
Type de document :
Article dans une revue
IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2014, pp.14. <10.1109/TPAMI.2014.2307877>


https://hal.archives-ouvertes.fr/hal-00958809
Contributeur : Estelle Nivault <>
Soumis le : jeudi 13 mars 2014 - 14:16:45
Dernière modification le : mardi 28 juin 2016 - 07:48:51
Document(s) archivé(s) le : vendredi 13 juin 2014 - 11:25:23

Fichier

PAMI.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

UNICE | I3S | BNRMI | UGA

Citation

Richard Nock, Wafa Bel Haj Ali, Roberto D'Ambrosio, Franck Nielsen, Michel Barlaud. Gentle Nearest Neighbors Boosting over Proper Scoring Rules. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2014, pp.14. <10.1109/TPAMI.2014.2307877>. <hal-00958809>

Partager

Métriques

Consultations de
la notice

272

Téléchargements du document

251