4462 articles – 2360 Notices  [english version]
 HAL : inria-00486840, version 4
 DOI : 10.1109/TSP.2011.2107908
 IEEE Transactions on Signal Processing 59, 5 (2011) 2405-2410
 Versions disponibles v1 (27-05-2010) v2 (02-12-2010) v3 (14-12-2010) v4 (12-03-2011)
 Should penalized least squares regression be interpreted as Maximum A Posteriori estimation?
 (05/2011)
 Penalized least squares regression is often used for signal denoising and inverse problems, and is commonly interpreted in a Bayesian framework as a Maximum A Posteriori (MAP) estimator, the penalty function being the negative logarithm of the prior. For example, the widely used quadratic program (with an $\ell^1$ penalty) associated to the LASSO / Basis Pursuit Denoising is very often considered as MAP estimation under a Laplacian prior in the context of additive white Gaussian noise (AWGN) reduction. This paper highlights the fact that, while this is {\em one} possible Bayesian interpretation, there can be other equally acceptable Bayesian interpretations. Therefore, solving a penalized least squares regression problem with penalty $\phi(x)$ need not be interpreted as assuming a prior $C\cdot \exp(-\phi(x))$ and using the MAP estimator. In particular, it is shown that for {\em any} prior $P_X$, the minimum mean square error (MMSE) estimator is the solution of a penalized least square problem with some penalty $\phi(x)$, which can be interpreted as the MAP estimator with the prior $C \cdot \exp(-\phi(x))$. Vice-versa, for {\em certain} penalties $\phi(x)$, the solution of the penalized least squares problem is indeed the MMSE estimator, with a certain prior $P_X$. In general $dP_X(x) \neq C \cdot \exp(-\phi(x))dx$.
 1 : METISS (INRIA - IRISA) CNRS : UMR6074 – INRIA – Institut National des Sciences Appliquées (INSA) - Rennes – Université de Rennes 1
 Domaine : Informatique/Traitement du signal et de l'imageMathématiques/StatistiquesStatistiques/ThéorieSciences de l'ingénieur/Traitement du signal et de l'image
 Mots-clés : Bayesian estimation – Maximum A Posteriori – Minimum Mean Square Error – MAP – MMSE – penalized least squares regression – LASSO – Basis Pursuit – nonconvex optimization – proximity operator
Liste des fichiers attachés à ce document :
 ANNEX
 MMSEvsMAP.m(2.7 KB)
 PDF
 MAPvsOptim-revision-HAL.pdf(248.6 KB)
 inria-00486840, version 4 http://hal.inria.fr/inria-00486840 oai:hal.inria.fr:inria-00486840 Contributeur : Rémi Gribonval <> Soumis le : Vendredi 11 Mars 2011, 17:52:30 Dernière modification le : Mardi 14 Juin 2011, 20:00:53