Calibration and regret bounds for order-preserving surrogate losses in learning to rank

Abstract : Learning to rank is usually reduced to learning to score individual objects, leaving the "ranking" step to a sorting algorithm. In that context, the surrogate loss used for training the scoring function needs to behave well with respect to the target performance measure which only sees the final ranking. A characterization of such a good behavior is the notion of calibration, which guarantees that minimizing (over the set of measurable functions) the surrogate risk allows us to maximize the true performance. In this paper, we consider the family of order-preserving (OP) losses which includes popular surrogate losses for ranking such as the squared error and pairwise losses. We show that they are calibrated with performance measures like the Discounted Cumulative Gain (DCG), but also that they are not calibrated with respect to the widely used Mean Average Precision and Expected Reciprocal Rank. We also derive, for some widely used OP losses, quantitative surrogate regret bounds with respect to several DCG-like evaluation measures.
Document type :
Journal articles
Liste complète des métadonnées

Cited literature [27 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00834230
Contributor : Clément Calauzènes <>
Submitted on : Friday, June 14, 2013 - 3:10:07 PM
Last modification on : Thursday, March 21, 2019 - 2:34:28 PM
Document(s) archivé(s) le : Tuesday, April 4, 2017 - 9:59:03 PM

File

final_main.pdf
Publisher files allowed on an open archive

Identifiers

Citation

Clément Calauzènes, Nicolas Usunier, Patrick Gallinari. Calibration and regret bounds for order-preserving surrogate losses in learning to rank. Machine Learning, Springer Verlag, 2013, 93 (2-3), pp.227-260. ⟨10.1007/s10994-013-5382-3⟩. ⟨hal-00834230⟩

Share

Metrics

Record views

356

Files downloads

88