Computation and Estimation of Generalized Entropy Rates for Denumerable Markov Chains

Gabriela Ciuperca 1 Valérie Girardin 2 Loïck Lhote 3
1 LAPCS - Laboratoire de Probabilités, Combinatoire et Statistique
2 LMNO - Laboratoire de Mathématiques Nicolas Oresme
3 Equipe AMACC - Laboratoire GREYC - UMR6072
GREYC - Groupe de Recherche en Informatique, Image, Automatique et Instrumentation de Caen
Abstract : —We study entropy rates of random sequences for general entropy functionals including the classical Shannon and Rényi entropies and the more recent Tsallis and Sharma-Mittal ones. In the first part, we obtain an explicit formula for the entropy rate for a large class of entropy functionals, as soon as the process satisfies a regularity property known in dynamical systems theory as the quasi-power property. Independent and identically distributed sequence of random variables naturally satisfy this property. Markov chains are proven to satisfy it too, under simple explicit conditions on their transition probabilities. All the entropy rates under study are thus shown to be either infinite or zero except at a threshold where they are equal to Shannon or Rényi entropy rates up to a multiplicative constant. In the second part, we focus on the estimation of the marginal generalized entropy and entropy rate for parametric Markov chains. Estimators with good asymptotic properties are built through a plug-in procedure using a maximum likelihood es-timation of the parameter.
Keywords : Rényi entropy Tsallis entropy plug-in estimation Index Terms—entropy rate entropy functional parametric Markov chain
Type de document :
Article dans une revue
IEEE Transactions on Information Theory, Institute of Electrical and Electronics Engineers, 2011, 57, pp.4026 - 4034. 〈10.1109/TIT.2011.2133710〉
Liste complète des métadonnées

Littérature citée [24 références]  Voir  Masquer  Télécharger
https://hal.archives-ouvertes.fr/hal-01082088
Contributeur : Greyc Référent <>
Soumis le : mercredi 12 novembre 2014 - 15:33:31
Dernière modification le : jeudi 7 février 2019 - 17:33:30
Document(s) archivé(s) le : vendredi 13 février 2015 - 11:16:15

Fichier

RIACL-CIUPERCA-2011-1.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

GREYC | GREYC-AMACC | ICJ | COMUE-NORMANDIE | UNIV-LYON1 | ENSICAEN | UNICAEN

Citation

Gabriela Ciuperca, Valérie Girardin, Loïck Lhote. Computation and Estimation of Generalized Entropy Rates for Denumerable Markov Chains. IEEE Transactions on Information Theory, Institute of Electrical and Electronics Engineers, 2011, 57, pp.4026 - 4034. 〈10.1109/TIT.2011.2133710〉. 〈hal-01082088〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

330

Téléchargements de fichiers

339

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD