Towards Engineering aWeb-Scale Multimedia Service: A Case Study Using Spark

Abstract : Computing power has now become abundant with multi-core machines, grids and clouds, but it remains a challenge to harness the available power and move towards gracefully handling web-scale datasets. Several researchers have used automatically distributed computing frameworks, notably Hadoop and Spark, for processing multimedia material, but mostly using small collections on small clusters. In this paper, we describe the engineering process for a prototype of a (near) web-scale multimedia service using the Spark framework running on the AWS cloud service. We present experimental results using up to 43 billion SIFT feature vectors from the public YFCC 100M collection, making this the largest high-dimensional feature vector collection reported in the literature. The design of the prototype and performance results demonstrate both the flexibility and scalability of the Spark framework for implementing multimedia services.
Type de document :
Communication dans un congrès
Multimedia Systems Conference, Jun 2017, Taipei, Taiwan. acm, MMSys'17 Proceedings of the 8th ACM on Multimedia Systems Conference, 2017, MMSys'17 Proceedings of the 8th ACM on Multimedia Systems Conference. <10.1145/3083187.3083200>
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01565196
Contributeur : Laurent Amsaleg <>
Soumis le : mercredi 19 juillet 2017 - 15:41:46
Dernière modification le : jeudi 20 juillet 2017 - 01:11:20

Identifiants

Collections

Citation

Gylfi Þór Guðmundsson, Laurent Amsaleg, Björn Þór Jónsson, Michael Franklin. Towards Engineering aWeb-Scale Multimedia Service: A Case Study Using Spark. Multimedia Systems Conference, Jun 2017, Taipei, Taiwan. acm, MMSys'17 Proceedings of the 8th ACM on Multimedia Systems Conference, 2017, MMSys'17 Proceedings of the 8th ACM on Multimedia Systems Conference. <10.1145/3083187.3083200>. <hal-01565196>

Partager

Métriques

Consultations de la notice

65