Closed-Form MMSE Estimation for Signal Denoising Under Sparse Representation Modeling Over a Unitary Dictionary

Matan Protter (1) , Irad Yavneh (1) , Michael Elad (1)

1 Department of Computer Science [Haifa]

This paper deals with the Bayesian signal denoising problem, assuming a prior based on a sparse representation modeling over a unitary dictionary. It is well known that the maximum a posteriori probability (MAP) estimator in such a case has a closed-form solution based on a simple shrinkage. The focus in this paper is on the better performing and less familiar minimum-mean-squared-error (MMSE) estimator. We show that this estimator also leads to a simple formula, in the form of a plain recursive expression for evaluating the contribution of every atom in the solution. An extension of the model to real-world signals is also offered, considering heteroscedastic nonzero entries in the representation, and allowing varying probabilities for the chosen atoms and the overall cardinality of the sparse representation. The MAP and MMSE estimators are redeveloped for this extended model, again resulting in closed-form simple algorithms. Finally, the superiority of the MMSE estimator is demonstrated both on synthetically generated signals and on real-world signals (image patches).

Mots clés: Bayesian signal denoising; MAP estimator; MMSE estimator; closed-form MMSE estimation; maximum a posteriori probability estimator; minimum-mean-squared-error estimator; plain recursive expression; sparse representation; unitary dictionary; Bayes methods; least mean squares methods; recursive estimation; signal denoising

Matan Protter, Irad Yavneh, Michael Elad. Closed-Form MMSE Estimation for Signal Denoising Under Sparse Representation Modeling Over a Unitary Dictionary. IEEE Transactions on Signal Processing, 2010, 58 (7), pp.3471 -3484. ⟨10.1109/TSP.2010.2046596⟩. ⟨inria-00577220⟩