A Performance Evaluation of Apache Kafka in Support of Big Data Streaming Applications

Paul Le Noac'H 1 Alexandru Costan 1 Luc Bougé 1
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA_D1 - SYSTÈMES LARGE ÉCHELLE
Abstract : Producer performances when modifying batch size for several number of nodes and a message size of 50B 7. Take-aways • The variation of the batch size shows that there is a range of batches with a better performance. • When varying the number of nodes in some scenarios: a sudden performance drop (probably due to the internal Kafka synchronizations as well as the underlying network). • Future work : evaluating reference processing frameworks (Apache Spark and Flink) Parameters : • Message size • Batch size • Acquirement strategy • Network and disk I/O threads • Message replication • Hardware 2. Contribution • Isolate the performance of each Kafka component
Type de document :
Poster
IEEE Big Data 2017, Dec 2017, Boston, United States. 2017
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01647229
Contributeur : Paul Le Noac'H <>
Soumis le : vendredi 24 novembre 2017 - 12:12:17
Dernière modification le : jeudi 15 novembre 2018 - 11:58:57

Identifiants

  • HAL Id : hal-01647229, version 1

Citation

Paul Le Noac'H, Alexandru Costan, Luc Bougé. A Performance Evaluation of Apache Kafka in Support of Big Data Streaming Applications. IEEE Big Data 2017, Dec 2017, Boston, United States. 2017. 〈hal-01647229〉

Partager

Métriques

Consultations de la notice

2172

Téléchargements de fichiers

371