Multi-Level Elasticity for Data Stream Processing

Abstract : This paper investigates reactive elasticity in stream processing environments where the performance goal is to analyze large amounts of data with low latency and minimum resources. Working in the context of Apache Storm, we propose an elastic management strategy which modulates the parallelism degree of applications' components while explicitly addressing the hierarchy of execution containers (virtual machines, processes and threads). We show that provisioning the wrong kind of container may lead to performance degradation and propose a solution that provisions the least expensive container (with minimum resources) to increase performance. We describe our monitoring metrics and show how we take into account the specifics of an execution environment. We provide an experimental evaluation with real-world applications which validates the applicability of our approach.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02143440
Contributor : Vania Marangozova-Martin <>
Submitted on : Wednesday, May 29, 2019 - 2:01:59 PM
Last modification on : Thursday, June 6, 2019 - 1:26:16 AM

Identifiers

Collections

Citation

Vania Marangozova-Martin, Ahmed El Rheddane, Noel de Palma. Multi-Level Elasticity for Data Stream Processing. IEEE Transactions on Parallel and Distributed Systems, Institute of Electrical and Electronics Engineers, 2019, pp.1-12. ⟨10.1109/TPDS.2019.2907950⟩. ⟨hal-02143440⟩

Share

Metrics

Record views

9