Efficient estimation of the cardinality of large data sets

Philippe Chassaing (1) , Lucas Gerin (1)

1 IECN - Institut Élie Cartan de Nancy

F.Giroire has recently proposed an algorithm which returns the approximate number of distincts elements in a large sequence of words, under strong constraints coming from the analysis of large data bases. His estimation is based on statistical properties of uniform random variables in $[0,1]$. In this note we propose an optimal estimation, using Kullback information and estimation theory.

Mots clés: cardinality; large multiset; approximate counting; data stream algorithms

Domaines: Probabilités [math.PR]

Philippe Chassaing, Lucas Gerin. Efficient estimation of the cardinality of large data sets. Fourth Colloquium on Mathematics and Computer Science Algorithms, Trees, Combinatorics and Probabilities, 2006, Nancy, France. pp.419-422. ⟨hal-00095370v4⟩