S. Basu and J. Christensen, Teaching classification boundaries to humans, Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI'13, pp.109-115, 2013.

I. Basu-roy, S. Lykourentzou, S. Thirumuruganathan, G. Amer-yahia, and . Das, Task assignment optimization in knowledge-intensive crowdsourcing, The VLDB Journal, vol.9, issue.3, pp.467-491, 2015.
DOI : 10.1145/2442657.2442661

J. Biel and D. Gatica-perez, The YouTube Lens: Crowdsourced Personality Impressions and Audiovisual Analysis of Vlogs, IEEE Transactions on Multimedia, vol.15, issue.1, pp.41-55, 2013.
DOI : 10.1109/TMM.2012.2225032

P. Bonnet, W. Vellinga, R. Planqué, A. Rauber, S. Palazzo et al., Lifeclef 2015: Multimedia life species identification challenges, Experimental IR Meets Multilinguality, Multimodality, and Interaction: 6th International Conference of the CLEF Association, CLEF'15 Proceedings, p.462, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01182782

K. Borne and Z. Team, The zooniverse: A framework for knowledge discovery from citizen science data, AGU Fall Meeting Abstracts, p.650, 2011.

J. Bragg, D. S. Mausam, and . Weld, Optimal testing for crowd workers, Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS '16 International Foundation for Autonomous Agents and Multiagent Systems, pp.966-974

J. Champ, A. Joly, and P. Bonnet, Fine-grained Visual Faceted Search, Proceedings of the ACM International Conference on Multimedia, MM '14, pp.721-722, 2014.
DOI : 10.1145/1991996.1992045
URL : https://hal.archives-ouvertes.fr/hal-01088822

J. Champ, T. Lorieul, M. Servajean, and A. Joly, A comparative study of fine-grained classification methods in the context of the lifeclef plant identification challenge 2015, CLEF 2015, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01182788

G. B. Dantzig, Discrete-Variable Extremum Problems, Operations Research, vol.5, issue.2, pp.266-288, 1957.
DOI : 10.1287/opre.5.2.266

A. P. Dawid and A. M. Skene, Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm, Applied Statistics, vol.28, issue.1, pp.20-28, 1979.
DOI : 10.2307/2346806

J. Deng, W. Dong, R. Socher, L. Li, K. Li et al., Imagenet: A large-scale hierarchical image database, Computer Vision and Pattern Recognition CVPR 2009. IEEE Conference on, pp.248-255, 2009.

M. Fang, X. Zhu, B. Li, W. Ding, and X. Wu, Self-Taught Active Learning from Crowds, 2012 IEEE 12th International Conference on Data Mining, pp.858-863, 2012.
DOI : 10.1109/ICDM.2012.64
URL : http://www.cs.umb.edu/%7Eding/papers/icdm2012.pdf

L. Gottlieb, G. Friedland, J. Choi, P. Kelm, and T. Sikora, Creating experts from the crowd: Techniques for finding workers for difficult tasks, IEEE Transactions on Multimedia, vol.16, issue.7, pp.2075-2079, 2014.
DOI : 10.1109/TMM.2014.2347268

P. Ipeirotis, Crowdsourcing using Mechanical Turk, Proceedings of the 8th International Workshop on Information Integration on the Web in conjunction with WWW 2011, IIWeb '11, p.1, 2011.
DOI : 10.1145/1982624.1982625

A. Joly, H. Goëau, P. Bonnet, V. Baki´cbaki´c, J. Barbe et al., Interactive plant identification based on social image data, Ecological Informatics, vol.23, pp.22-34, 2014.
DOI : 10.1016/j.ecoinf.2013.07.006
URL : https://hal.archives-ouvertes.fr/hal-00908872

E. Kamar, S. Hacker, and E. Horvitz, Combining human and machine intelligence in large-scale crowdsourcing, AAMAS '12 Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, pp.467-474, 2012.

H. Kim and Z. Ghahramani, Bayesian classifier combination, International conference on artificial intelligence and statistics, pp.619-627, 2012.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, Advances in neural information processing systems, pp.1097-1105, 2012.
DOI : 10.1162/neco.2009.10-08-881
URL : http://dl.acm.org/ft_gateway.cfm?id=3065386&type=pdf

D. Liu, S. Yan, X. Hua, and H. Zhang, Image Retagging Using Collaborative Tag Propagation, IEEE Transactions on Multimedia, vol.13, issue.4, pp.702-712, 2011.
DOI : 10.1109/TMM.2011.2134078
URL : http://www.ee.columbia.edu/%7Edongliu/Papers/RetaggingTMM.pdf

A. Parameswaran, A. D. Sarma, H. Garcia-molina, N. Polyzotis, and J. Widom, Human-assisted graph search, Proceedings of the VLDB Endowment, pp.267-278, 2011.
DOI : 10.14778/1952376.1952377

A. G. Parameswaran, H. Garcia-molina, H. Park, N. Polyzotis, A. Ramesh et al., CrowdScreen, Proceedings of the 2012 international conference on Management of Data, SIGMOD '12, pp.361-372, 2012.
DOI : 10.1145/2213836.2213878

A. Quinn, B. Bederson, T. Yeh, and J. Lin, CrowdFlow: Integrating machine learning with mechanical turk for speedcost-quality flexibility. Better performance over iterations, 2010.

H. Rahman, L. Joppa, and S. B. Roy, Feature based task recommendation in crowdsourcing with implicit observations

V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin et al., Learning from crowds, The Journal of Machine Learning Research, vol.11, pp.1297-1322, 2010.

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, LabelMe: A Database and Web-Based Tool for Image Annotation, International Journal of Computer Vision, vol.3, issue.1, pp.1-3157, 2008.
DOI : 10.1007/s11263-007-0090-8
URL : http://staff.science.uva.nl/%7Ecgmsnoek/pub/readinggroup/LabelMe.pdf

S. Sahni and T. Gonzalez, P-Complete Approximation Problems, Journal of the ACM, vol.23, issue.3, pp.555-565, 1976.
DOI : 10.1145/321958.321975

J. Sang, C. Xu, and J. Liu, User-Aware Image Tag Refinement via Ternary Semantic Analysis, IEEE Transactions on Multimedia, vol.14, issue.3, pp.883-895, 2012.
DOI : 10.1109/TMM.2012.2188782

V. S. Sheng, F. Provost, and P. G. Ipeirotis, Get another label? improving data quality and data mining using multiple, noisy labelers, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD 08, pp.614-622, 2008.
DOI : 10.1145/1401890.1401965
URL : http://pages.stern.nyu.edu/~panos/publications/kdd2008.pdf

E. Simpson and S. Roberts, Bayesian Methods for Intelligent Task Assignment in Crowdsourcing Systems, Decision Making: Uncertainty, Imperfection, Deliberation and Scalability, pp.1-32, 2015.
DOI : 10.1007/978-3-319-15144-1_1

E. Simpson, S. Roberts, I. Psorakis, and A. Smith, Dynamic Bayesian Combination of Multiple Imperfect Classifiers, Decision Making and Imperfection, pp.1-35, 2013.
DOI : 10.1007/978-3-642-36406-8_1
URL : http://arxiv.org/abs/1206.1831

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed et al., Going deeper with convolutions. arXiv preprint, 2014.
DOI : 10.1109/cvpr.2015.7298594
URL : http://arxiv.org/pdf/1409.4842

B. Thomee, D. A. Shamma, G. Friedland, B. Elizalde, K. Ni et al., The new data and new challenges in multimedia research, 2015.

L. Tran-thanh, M. Venanzi, A. Rogers, and N. R. Jennings, Efficient budget allocation with accuracy guarantees for crowdsourcing classification tasks, Proceedings of the 2013 International Conference on Autonomous Agents and Multiagent Systems, AAMAS '13 International Foundation for Autonomous Agents and Multiagent Systems, pp.901-908, 2013.

S. Tulyakov, S. Jaeger, V. Govindaraju, and D. Doermann, Review of Classifier Combination Methods, Studies in Computational Intelligence, vol.90, issue.1, pp.361-386, 2008.
DOI : 10.1007/978-3-540-76280-5_14
URL : http://lampsrv02.umiacs.umd.edu/pubs/Papers/Tulyakov_MLDAR/Tulyakov_MLDAR.pdf

M. Venanzi, J. Guiver, and G. Kazai, Community-based bayesian aggregation models for crowdsourcing, Proceedings of the 23rd international conference on World wide web, WWW '14, pp.63-87, 2014.
DOI : 10.1145/2566486.2567989

C. Wang and D. M. Blei, Variational Inference in Nonconjugate Models, Journal of Machine Learning Research, vol.14, pp.1005-1031, 2013.

P. Welinder and P. Perona, Online crowdsourcing: Rating annotators and obtaining cost-effective labels, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Workshops, pp.25-32, 2010.
DOI : 10.1109/CVPRW.2010.5543189
URL : http://vision.caltech.edu/publications/WelinderPerona10.pdf