Skip to Main content Skip to Navigation
New interface
Journal articles

Model Based Comparison of Discounted Cumulative Gain and Average Precision

Georges Dupret Benjamin Piwowarski 1 
1 BD - Bases de Données
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : In this paper, we propose to explain Discounted Cumulative Gain (DCG) as the expectation of the total utility collected by a user given a generative probabilistic model on how users browse the result page ranking list of a search engine. We contrast this with a generalization of Average Precision, pAP, that has been defined in Dupret and Piwowarski (2010). In both cases, user decision models coupled with Web search logs allow to estimate some parameters that are usually left to the designer of a metric. In this paper, we compare the user models for DCG and pAP at the interpretation and experimental level. DCG and AP are metrics computed before a ranking function is exposed to users and as such, their role is to predict the function performance. In counterpart to prognostic metric, a diagnostic metric is computed after observing the user interactions with the result list. A commonly used diagnostic metric is the clickthrough rate at position 1, for example. In this work we show that the same user model developed for DCG can be used to derive a diagnostic version of this metric. The same hold for pAP and any metric with a proper user model. We show that not only does this diagnostic view provide new information, it also allows to define a new criterion for assessing a metric. In previous works based on user decision modeling, the performance of different metrics were compared indirectly in terms of the ability of the associated user model to predict future user actions. Here we propose a new and more direct criterion based on the ability of the prognostic version of the metric to predict the diagnostic performance.
Document type :
Journal articles
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-01185457
Contributor : Lip6 Publications Connect in order to contact the contributor
Submitted on : Thursday, August 20, 2015 - 11:22:20 AM
Last modification on : Sunday, June 26, 2022 - 9:40:27 AM

Links full text

Identifiers

Citation

Georges Dupret, Benjamin Piwowarski. Model Based Comparison of Discounted Cumulative Gain and Average Precision. Journal of Discrete Algorithms, 2013, 18, pp.49-62. ⟨10.1016/j.jda.2012.10.002⟩. ⟨hal-01185457⟩

Share

Metrics

Record views

77