A Minimax Optimal Algorithm for Crowdsourcing

Abstract : We consider the problem of accurately estimating the reliability of workers based on noisy labels they provide, which is a fundamental question in crowdsourcing. We propose a novel lower bound on the minimax estimation error which applies to any estimation procedure. We further propose Triangular Estimation (TE), an algorithm for estimating the reliability of workers. TE has low complexity, may be implemented in a streaming setting when labels are provided by workers in real time, and does not rely on an iterative procedure. We prove that TE is minimax optimal and matches our lower bound. We conclude by assessing the performance of TE and other state-of-the-art algorithms on both synthetic and real-world data.
Type de document :
Communication dans un congrès
NIPS, 2017, Los Angeles, United States
Liste complète des métadonnées

Littérature citée [20 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01592105
Contributeur : Thomas Bonald <>
Soumis le : vendredi 22 septembre 2017 - 16:00:22
Dernière modification le : vendredi 31 août 2018 - 09:12:07
Document(s) archivé(s) le : samedi 23 décembre 2017 - 14:00:16

Fichier

nips17.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01592105, version 1

Citation

Thomas Bonald, Richard Combes. A Minimax Optimal Algorithm for Crowdsourcing. NIPS, 2017, Los Angeles, United States. 〈hal-01592105〉

Partager

Métriques

Consultations de la notice

347

Téléchargements de fichiers

206