Skip to Main content Skip to Navigation
Journal articles

Calibration and regret bounds for order-preserving surrogate losses in learning to rank

Abstract : Learning to rank is usually reduced to learning to score individual objects, leaving the "ranking" step to a sorting algorithm. In that context, the surrogate loss used for training the scoring function needs to behave well with respect to the target performance measure which only sees the final ranking. A characterization of such a good behavior is the notion of calibration, which guarantees that minimizing (over the set of measurable functions) the surrogate risk allows us to maximize the true performance. In this paper, we consider the family of order-preserving (OP) losses which includes popular surrogate losses for ranking such as the squared error and pairwise losses. We show that they are calibrated with performance measures like the Discounted Cumulative Gain (DCG), but also that they are not calibrated with respect to the widely used Mean Average Precision and Expected Reciprocal Rank. We also derive, for some widely used OP losses, quantitative surrogate regret bounds with respect to several DCG-like evaluation measures.
Document type :
Journal articles
Complete list of metadata

Cited literature [27 references]  Display  Hide  Download
Contributor : Clément Calauzènes <>
Submitted on : Friday, June 14, 2013 - 3:10:07 PM
Last modification on : Friday, January 8, 2021 - 5:34:09 PM
Long-term archiving on: : Tuesday, April 4, 2017 - 9:59:03 PM


Publisher files allowed on an open archive



Clément Calauzènes, Nicolas Usunier, Patrick Gallinari. Calibration and regret bounds for order-preserving surrogate losses in learning to rank. Machine Learning, Springer Verlag, 2013, 93 (2-3), pp.227-260. ⟨10.1007/s10994-013-5382-3⟩. ⟨hal-00834230⟩



Record views


Files downloads