HAL : inria-00486840, version 4

DOI : 10.1109/TSP.2011.2107908

Voir la fiche détaillée

BibTeX,EndNote,... BibTeX - EndNote - TEI - RefWorks -

IEEE Transactions on Signal Processing 59, 5 (2011) 2405-2410

Versions disponibles

v1 (27-05-2010)

v2 (02-12-2010)

v3 (14-12-2010)

v4 (12-03-2011)

Should penalized least squares regression be interpreted as Maximum A Posteriori estimation?

Rémi Gribonval 1

(05/2011)

Penalized least squares regression is often used for signal denoising and inverse problems, and is commonly interpreted in a Bayesian framework as a Maximum A Posteriori (MAP) estimator, the penalty function being the negative logarithm of the prior. For example, the widely used quadratic program (with an $\ell^1$ penalty) associated to the LASSO / Basis Pursuit Denoising is very often considered as MAP estimation under a Laplacian prior in the context of additive white Gaussian noise (AWGN) reduction. This paper highlights the fact that, while this is {\em one} possible Bayesian interpretation, there can be other equally acceptable Bayesian interpretations. Therefore, solving a penalized least squares regression problem with penalty $\phi(x)$ need not be interpreted as assuming a prior $C\cdot \exp(-\phi(x))$ and using the MAP estimator. In particular, it is shown that for {\em any} prior $P_X$, the minimum mean square error (MMSE) estimator is the solution of a penalized least square problem with some penalty $\phi(x)$, which can be interpreted as the MAP estimator with the prior $C \cdot \exp(-\phi(x))$. Vice-versa, for {\em certain} penalties $\phi(x)$, the solution of the penalized least squares problem is indeed the MMSE estimator, with a certain prior $P_X$. In general $dP_X(x) \neq C \cdot \exp(-\phi(x))dx$.

1 :	METISS (INRIA - IRISA)
	CNRS : UMR6074 – INRIA – Institut National des Sciences Appliquées (INSA) - Rennes – Université de Rennes 1

Domaine

:

Informatique/Traitement du signal et de l'image Mathématiques/Statistiques Statistiques/Théorie Sciences de l'ingénieur/Traitement du signal et de l'image

Mots-clés : Bayesian estimation – Maximum A Posteriori – Minimum Mean Square Error – MAP – MMSE – penalized least squares regression – LASSO – Basis Pursuit – nonconvex optimization – proximity operator

inria-00486840, version 4

http://hal.inria.fr/inria-00486840

oai:hal.inria.fr:inria-00486840

Contributeur : Rémi Gribonval <>

Soumis le : Vendredi 11 Mars 2011, 17:52:30

Dernière modification le : Mardi 14 Juin 2011, 20:00:53