Estimating the Frequency of Data Items in Massive Distributed Streams

Emmanuelle Anceaume 1 Yann Busnel 2, 3 Nicolò Rivetti 4
1 CIDRE - Confidentialité, Intégrité, Disponibilité et Répartition
CentraleSupélec, Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
3 DIONYSOS - Dependability Interoperability and perfOrmance aNalYsiS Of networkS
Inria Rennes – Bretagne Atlantique , IRISA-D2 - RÉSEAUX, TÉLÉCOMMUNICATION ET SERVICES
4 GDD - Gestion de Données Distribuées [Nantes]
LINA - Laboratoire d'Informatique de Nantes Atlantique
Abstract : —We investigate the problem of estimating on the fly the frequency at which items recur in large scale distributed data streams, which has become the norm in cloud-based application. This paper presents CASE, a combination of tools and probabilistic algorithms from the data streaming model. In this model, functions are estimated on a huge sequence of data items, in an online fashion, and with a very small amount of memory with respect to both the size of the input stream and the values domain from which data items are drawn. We derive upper and lower bounds on the quality of CASE, improving upon the Count-Min sketch algorithm which has, so far, been the best algorithm in terms of space and time performance to estimate the frequency of data items. We prove that CASE guarantees an (ε, δ)-approximation of the frequency of all the items, provided they are not rare, in a space-efficient way and for any input stream. Experiments on both synthetic and real datasets confirm our analysis. Index Terms—Data stream model; frequency estimation; randomized approximation algorithm.
Complete list of metadatas

Cited literature [17 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01194529
Contributor : Emmanuelle Anceaume <>
Submitted on : Monday, September 7, 2015 - 10:54:18 AM
Last modification on : Friday, November 16, 2018 - 1:39:08 AM
Long-term archiving on : Tuesday, December 8, 2015 - 11:42:25 AM

File

NCCA2015.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01194529, version 1

Citation

Emmanuelle Anceaume, Yann Busnel, Nicolò Rivetti. Estimating the Frequency of Data Items in Massive Distributed Streams. IEEE 4th Symposium on Network Cloud Computing and Applications (NCCA), Jun 2015, Munich, Germany. pp.9. ⟨hal-01194529⟩

Share

Metrics

Record views

1827

Files downloads

325