Skip to Main content Skip to Navigation
Poster communications

A Performance Evaluation of Apache Kafka in Support of Big Data Streaming Applications

Paul Le Noac'H 1 Alexandru Costan 1 Luc Bougé 1
1 KerData - Scalable Storage for Clouds and Beyond
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
Abstract : Producer performances when modifying batch size for several number of nodes and a message size of 50B 7. Take-aways • The variation of the batch size shows that there is a range of batches with a better performance. • When varying the number of nodes in some scenarios: a sudden performance drop (probably due to the internal Kafka synchronizations as well as the underlying network). • Future work : evaluating reference processing frameworks (Apache Spark and Flink) Parameters : • Message size • Batch size • Acquirement strategy • Network and disk I/O threads • Message replication • Hardware 2. Contribution • Isolate the performance of each Kafka component
Document type :
Poster communications
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01647229
Contributor : Paul Le Noac'H <>
Submitted on : Friday, November 24, 2017 - 12:12:17 PM
Last modification on : Wednesday, June 24, 2020 - 4:19:45 PM

Identifiers

  • HAL Id : hal-01647229, version 1

Citation

Paul Le Noac'H, Alexandru Costan, Luc Bougé. A Performance Evaluation of Apache Kafka in Support of Big Data Streaming Applications. IEEE Big Data 2017, Dec 2017, Boston, United States. 2017. ⟨hal-01647229⟩

Share

Metrics

Record views

2614

Files downloads

960