A Lower Bound on the Sample Size needed to perform a Significant Frequent Pattern Mining Task

Stéphanie Jacquemont (1) , François Jacquenet (1) , Marc Sebban (1)

1 LHC - Laboratoire Hubert Curien

During the past few years, the problem of assessing the statistical significance of frequent patterns extracted from a given set S of data has received much attention. Considering that S always consists of a sample drawn from an unknown underlying distribution, two types of risks can arise during a frequent pattern mining process: accepting a false frequent pattern or rejecting a true one. In this context, many approaches presented in the literature assume that the dataset size is an application-dependent parameter. In this case, there is a trade-off between both errors leading to solutions that only control one risk to the detriment of the other one. On the other hand, many sampling-based methods have attempted to determine the optimal size of S ensuring a good approximation of the original (potentially infinite) database from which S is drawn. However, these approaches often resort to Chernoff bounds that do not allow the independent control of the two risks. In this paper, we overcome the mentioned drawbacks by providing a lower bound on the sample size required to control both risks and achieve a significant frequent pattern mining task.

Domains: Machine Learning [cs.LG]

Stéphanie Jacquemont, François Jacquenet, Marc Sebban. A Lower Bound on the Sample Size needed to perform a Significant Frequent Pattern Mining Task. Pattern Recognition Letters, 2009, 30 (2009), pp.960-967. ⟨10.1016/j.patrec.2009.05.002⟩. ⟨hal-00381667v2⟩