Characterization of experts in crowdsourcing platforms

Abstract : Crowdsourcing platforms enable to propose simple human intelligence tasks to a large number of participants who realise these tasks. The workers often receive a small amount of money or the platforms include some other incentive mechanisms, for example they can increase the workers reputation score, if they complete the tasks correctly. We address the problem of identifying experts among participants, that is, workers, who tend to answer the questions correctly. Knowing who are the reliable workers could improve the quality of knowledge one can extract from responses. As opposed to other works in the literature, we assume that participants can give partial or incomplete responses, in case they are not sure that their answers are correct. We model such partial or incomplete responses with the help of belief functions, and we derive a measure that characterizes the expertise level of each participant. This measure is based on precise and exactitude degrees that represent two parts of the expertise level. The precision degree reflects the reliability level of the participants and the exactitude degree reflects the knowledge level of the participants. We also analyze our model through simulation and demonstrate that our richer model can lead to more reliable identification of experts.
Type de document :
Communication dans un congrès
The 4th International Conference on Belief Functions, Sep 2016, Prague, Czech Republic
Liste complète des métadonnées

Littérature citée [14 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01372142
Contributeur : Arnaud Martin <>
Soumis le : lundi 26 septembre 2016 - 21:25:32
Dernière modification le : mercredi 2 août 2017 - 10:08:33

Fichiers

Article-Belief 2016-VF.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01372142, version 1
  • ARXIV : 1609.09748

Citation

Amal Ben Rjab, Mouloud Kharoune, Zoltan Miklos, Arnaud Martin. Characterization of experts in crowdsourcing platforms. The 4th International Conference on Belief Functions, Sep 2016, Prague, Czech Republic. 〈hal-01372142〉

Partager

Métriques

Consultations de
la notice

213

Téléchargements du document

142