Getting by with a Little Help from the Crowd: Practical Approaches to Social Image Labeling

Abstract : Validating user tags helps to refine them, making them more useful for finding images. In the case of interpretation-sensitive tags, however, automatic (i.e., pixel-based) approaches cannot be expected to deliver optimal results. Instead, human input is the key. This paper studies how crowdsourcing-based approaches to image tag validation can achieve parsimony in their use of human input from the crowd, in the form of votes collected from workers on a crowdsourcing platform. Experiments in the domain of social fashion images are carried out using the dataset published by the Crowdsourcing Task of the Mediaeval 2013 Multimedia Benchmark. Experimental results reveal that when a larger number of crowd-contributed votes are available, it is difficult to beat a majority vote. However, additional information sources, i.e., crowdworker history and visual image features, allow us to maintain similar validation performance while making use of less crowd-contributed input. Further, investing in expensive experts who collaborate to create definitions of interpretation-sensitive concepts does not necessarily pay off. Instead, experts can cause interpretations of concepts to drift away from conventional wisdom. In short, validation of interpretation-sensitive user tags for social images is possible, with " just a little help from the crowd".
Document type :
Conference papers
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01318338
Contributor : Bibliothèque Universitaire Déposants Hal-Avignon <>
Submitted on : Thursday, May 19, 2016 - 2:48:42 PM
Last modification on : Saturday, June 8, 2019 - 2:20:03 PM

Links full text

Identifiers

Collections

Citation

Babak Loni, Jonathon Hare, Mihai Georgescu, Michael Riegler, Xiaofei Zhu, et al.. Getting by with a Little Help from the Crowd: Practical Approaches to Social Image Labeling. CrowdMM '14 International ACM Workshop on Crowdsourcing for Multimedia , Nov 2014, Orlando, United States. ⟨10.1145/2660114.2660123⟩. ⟨hal-01318338⟩

Share

Metrics

Record views

46