Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

A non-parametric k-nearest neighbour entropy estimator

Damiano Lombardi 1 Sanjay Pant 1
1 REO - Numerical simulation of biological flows
LJLL - Laboratoire Jacques-Louis Lions, Inria Paris-Rocquencourt, UPMC - Université Pierre et Marie Curie - Paris 6
Abstract : A non-parametric k-nearest neighbour based entropy estimator is proposed. It improves on the classical Kozachenko-Leonenko estimator by considering non-uniform probability densities in the region of k-nearest neighbours around each sample point. It aims at improving the classical estima-tors in three situations: first, when the dimensionality of the random variable is large; second, when near-functional relationships leading to high correlation between components of the random variable are present; and third, when the marginal variances of random variable components vary significantly with respect to each other. Heuristics on the error of the proposed and classical estimators are presented. Finally, the proposed estimator is tested for a variety of distributions in successively increasing dimensions and in the presence of a near-functional relationship. Its performance is compared with a classical estimator and shown to be a significant improvement.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [5 references]  Display  Hide  Download
Contributor : Damiano Lombardi <>
Submitted on : Monday, June 22, 2015 - 8:59:41 AM
Last modification on : Friday, March 27, 2020 - 3:38:44 AM
Long-term archiving on: : Tuesday, April 25, 2017 - 5:37:19 PM


Files produced by the author(s)


  • HAL Id : hal-01166056, version 1


Damiano Lombardi, Sanjay Pant. A non-parametric k-nearest neighbour entropy estimator. 2015. ⟨hal-01166056⟩



Record views


Files downloads