Mixing and non-mixing local minima of the entropy contrast for blind source separation

Frédéric Vrins (1) , Dinh-Tuan Pham (2) , Michel Verleysen (1)

1 DICE - MLG - Dispositifs Intégrés et Circuits Electroniques Machine Learning Group
2 SMS - Statistique et Modélisation Stochatisque

In this paper, both non-mixing and mixing local minima of the entropy are analyzed from the viewpoint of blind source separation (BSS); they correspond respectively to acceptable and spurious solutions of the BSS problem. The contribution of this work is twofold. First, a Taylor development is used to show that the exact output entropy cost function has a non-mixing minimum when this output is proportional to any of the non-Gaussian sources, and not only when the output is proportional to the lowest entropic source. Second, in order to prove that mixing entropy minima exist when the source densities are strongly multimodal, an entropy approximator is proposed. The latter has the major advantage that an error bound can be provided. Even if this approximator (and the associated bound) is used here in the BSS context, it can be applied for estimating the entropy of any random variable with multimodal density.

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Frédéric Vrins, Dinh-Tuan Pham, Michel Verleysen. Mixing and non-mixing local minima of the entropy contrast for blind source separation. IEEE Transactions on Information Theory, 2007, 53 (3), pp.1030-1042. ⟨10.1109/TIT.2006.890716⟩. ⟨hal-00853136⟩