Is the general form of Renyi's entropy a contrast for source separation?

Frédéric Vrins (1) , Dinh-Tuan Pham (2) , Michel Verleysen (1)

1 DICE - MLG - Dispositifs Intégrés et Circuits Electroniques Machine Learning Group
2 SMS - Statistique et Modélisation Stochatisque

Renyi's entropy-based criterion has been proposed as an objective function for independent component analysis because of its relationship with Shannon's entropy and its computational advantages in specific cases. These criteria were suggested based on "convincing" experiments. However, there is no theoretical proof that globally maximizing those functions would lead to separate the sources; actually, this was implicitly conjectured. In this paper, the problem is tackled in a theoretical way; it is shown that globally maximizing the Renyi's entropy-based criterion, in its general form, does not necessarily provide the expected independent signals. The contrast function property of the corresponding criteria simultaneously depend on the value of the Renyi parameter, and on the (unknown) source densities.

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Frédéric Vrins, Dinh-Tuan Pham, Michel Verleysen. Is the general form of Renyi's entropy a contrast for source separation?. 7th International Conference of Independent Component Analysis and Signal Separation, ICA 2007, Sep 2007, London, United Kingdom. pp.129-136, ⟨10.1007/978-3-540-74494-8_17⟩. ⟨hal-00853922⟩