de Bruijn identities : from Shannon, Kullback–Leibler and Fisher to generalized φ-entropies, φ-divergences and φ-Fisher informations

Steeve Zozor 1 Jean-Marc Brossier 1
1 GIPSA-CICS - CICS
GIPSA-DIS - Département Images et Signal
Abstract : In this paper we propose a generalization of the usual deBruijn identity that links the Shannon differential entropy (or the Kullback–Leibler divergence) and the Fisher information (or the Fisher divergence) of the output of a Gaussian channel. The generalization makes use of φ-entropies on the one hand, and of φ-divergences (of the Csizàr class) on the other hand, as generalizations of the Shannon entropy and of the Kullback–Leibler divergence respectively. The generalized deBruijn identities induce the definition of generalized Fisher informations and generalized Fisher divergences; some of such generalizations exist in the literature. Moreover, we provide results that go beyond the Gaussian channel: we are then able to characterize a noisy channel using general measures of mutual information, both for Gaussian and non-Gaussian channels.
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01220709
Contributor : Steeve Zozor <>
Submitted on : Monday, October 26, 2015 - 6:01:07 PM
Last modification on : Monday, April 9, 2018 - 12:22:48 PM

Identifiers

Collections

Citation

Steeve Zozor, Jean-Marc Brossier. de Bruijn identities : from Shannon, Kullback–Leibler and Fisher to generalized φ-entropies, φ-divergences and φ-Fisher informations. 34th International Workshop on Bayesian Inference and Maximun Entropy Methods in Science and Engineering (MaxEnt'14), Sep 2014, Amboise, France. pp.522-529, ⟨10.1063/1.4906018⟩. ⟨hal-01220709⟩

Share

Metrics

Record views

440