Efficient Worker Selection Through History-based Learning in Crowdsourcing

Abstract : Crowdsourcing has emerged as a promising approach for obtaining services and data in a short time and at a reasonable budget. However, the quality of the output provided by the crowd is not guaranteed, and must be controlled. This quality control usually relies on worker screening or contribution reviewing at the cost of additional time and budget overheads. In this paper, we propose to reduce these overheads by leveraging the system history. We describe an offline learning algorithm that groups tasks from history into homogeneous clusters and learns for each cluster the worker features that optimize the contribution quality. These features are then used by the online targeting algorithm to select reliable workers for each incoming task. The proposed method is compared to the state of the art selection methods using real world datasets. Results show that we achieve comparable, and in some cases better, output quality for a smaller budget and shorter time.
Document type :
Conference papers
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01567896
Contributor : Tarek Awwad <>
Submitted on : Monday, July 24, 2017 - 4:25:56 PM
Last modification on : Wednesday, January 23, 2019 - 9:03:04 AM

Identifiers

Citation

Tarek Awwad, Nadia Bennani, Konstantin Ziegler, Veronika Rehn Sonigo, Lionel Brunie, et al.. Efficient Worker Selection Through History-based Learning in Crowdsourcing. Computer Software and Applications Conference COMPSAC, IEEE, Jul 2017, Turin, Italy. pp.923-928, ⟨10.1109/COMPSAC.2017.275⟩. ⟨hal-01567896⟩

Share

Metrics

Record views

490