Efficiently Summarizing Distributed Data Streams over Sliding Windows
Résumé
Estimating the frequency of any piece of information in large-scale distributed data streams became of utmost importance in the last decade (\emph{e.g.}, in the context of network monitoring, big data, \emph{etc.}). If some elegant solutions have been proposed recently, their approximation is computed from the inception of the stream. In a runtime distributed context, one would prefer to gather information only about the recent past. In this paper, we consider the \emph{sliding window functional monitoring} model and propose two different (on-line) algorithms that $(\varepsilon,\delta)$-approximate the items frequency in the active window. They use a very small amount of memory with respect to the size of the window $N$ and the number of distinct items $n$ of the stream: namely $O(\frac{1}{\varepsilon} \log \frac{1}{\delta} (\log N + \log n))$ and $O(\frac{1}{\tau\varepsilon} \log \frac{1}{\delta} (\log N + \log n))$ bits of space, where $\tau$ is a parameter limiting memory usage. We also provide their distributed variant with a communication cost of $O(\frac{k}{\varepsilon^2} \log \frac{1}{\delta} \log N)$ bits per window (where $k$ is the number of nodes). Experiments on synthetic traces and real data sets validate the robustness and accuracy of our algorithms.
Origine : Fichiers produits par l'(les) auteur(s)