Calibration and regret bounds for order-preserving surrogate losses in learning to rank

Abstract : Learning to rank is usually reduced to learning to score individual objects, leaving the "ranking" step to a sorting algorithm. In that context, the surrogate loss used for training the scoring function needs to behave well with respect to the target performance measure which only sees the final ranking. A characterization of such a good behavior is the notion of calibration, which guarantees that minimizing (over the set of measurable functions) the surrogate risk allows us to maximize the true performance. In this paper, we consider the family of order-preserving (OP) losses which includes popular surrogate losses for ranking such as the squared error and pairwise losses. We show that they are calibrated with performance measures like the Discounted Cumulative Gain (DCG), but also that they are not calibrated with respect to the widely used Mean Average Precision and Expected Reciprocal Rank. We also derive, for some widely used OP losses, quantitative surrogate regret bounds with respect to several DCG-like evaluation measures.
Type de document :
Article dans une revue
Machine Learning, Springer Verlag, 2013, 93 (2-3), pp.227-260. 〈10.1007/s10994-013-5382-3〉
Liste complète des métadonnées

Littérature citée [27 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-00834230
Contributeur : Clément Calauzènes <>
Soumis le : vendredi 14 juin 2013 - 15:10:07
Dernière modification le : lundi 17 décembre 2018 - 01:31:18
Document(s) archivé(s) le : mardi 4 avril 2017 - 21:59:03

Fichier

final_main.pdf
Fichiers éditeurs autorisés sur une archive ouverte

Identifiants

Citation

Clément Calauzènes, Nicolas Usunier, Patrick Gallinari. Calibration and regret bounds for order-preserving surrogate losses in learning to rank. Machine Learning, Springer Verlag, 2013, 93 (2-3), pp.227-260. 〈10.1007/s10994-013-5382-3〉. 〈hal-00834230〉

Partager

Métriques

Consultations de la notice

349

Téléchargements de fichiers

69