Cost-Efficient and Latency-Aware Event Consuming in Workload-Skewed Distributed Event Queues

Mazen Ezzeddine (1) , Gael Migliorini , Françoise Baude (2, 3) , Fabrice Huet (1)

1 I3S - Laboratoire d'Informatique, Signaux, et Systèmes de Sophia Antipolis
2 Laboratoire I3S - COMRED - COMmunications, Réseaux, systèmes Embarqués et Distribués
3 SCALE - Safe Composition of Autonomous applications with Large-SCALE Execution environment

Distributed event queues have emerged as a central component in building large scale cloud applications. In distributed event queues, guaranteeing a maximum event processing latency for high percentile of events in a cost-efficient manner is of paramount interest. This is achieved through efficient and accurate solutions to autoscale event consumers to meet the incoming workload. However, most of current solutions to autoscale event consumers are threshold-based that add/remove consumer replicas based on a metric of interest. These autoscalers lack an accurate estimation on the number of replicas that is just enough to keep up with the arrival rate of events and are not cost-efficient. Moreover, threshold-based autoscalers are not designed with workload-skewness in mind. When the workload is skewed few partitions of the distributed queue will receive higher percentile of the events produced. In such cases, the autoscale process must be complemented with a load-aware assignment of event consumer replicas to queue partitions. However, load-aware assignment is not performed by threshold-based autoscalers as they assume a uniform event load across the partitions of the queue. Hence, in this work, we first express the problem of cost-efficient scaling of event consumers to achieve a desired latency as a bin pack problem. This bin pack problem depends on the arrival rate of events, consumption rate of consumers, and on the events backlog in the queues. Next, we show that the process of scaling event consumers in face of skewed workload is performed by a controller/autoscaler and by one of the consumer replicas namely the leader. The controller monitors the cluster state and launches the appropriate number of consumer replicas. Next, the leader consumer performs a load-aware assignment of partitions to consumer replicas. In face of skewed workloads, observed results show order of magnitude gains in terms of latency guarantee as compared to an autoscale methodology that is not complemented by a load-aware assignment

Mots clés: Distributed Event Queue; Autoscale; Rebalancing; Producer; Consumer; Event Driven Microservices; Kubernetes; SLA; Kafka; Bin Packing; Microservice

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Mazen Ezzeddine, Gael Migliorini, Françoise Baude, Fabrice Huet. Cost-Efficient and Latency-Aware Event Consuming in Workload-Skewed Distributed Event Queues. 6th International Conference on Cloud and Big Data Computing (ICCBDC’2022), Aug 2022, Birmingham, United Kingdom. ⟨hal-03778255⟩