A. Dempster, N. M. Laird, and D. B. Rubin, Maximum-likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc, vol.39, pp.1-17, 1977.

R. Neal and G. Hinton, A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants, Learning in Graphical Models, M. I. Jordan, pp.355-368, 1998.
DOI : 10.1007/978-94-011-5014-9_12

H. Attias, A variational Bayesian framework for graphical models, Proc. Neural Inf. Process. Syst. Conf, pp.209-216, 2000.

C. Robert and G. Casella, Monte Carlo Statistical Methods, 2004.

P. Green, K. Latuszy?ski, M. Pereyra, and C. P. Robert, Bayesian computation: a summary of the current state, and samples backwards and forwards, Statistics and Computing, vol.91, issue.14, 2015.
DOI : 10.1007/s11222-015-9574-5
URL : https://hal.archives-ouvertes.fr/hal-01409252

A. Doucet and A. M. Johansen, A tutorial on particle filtering and smoothing: Fifteen years later, The Oxford Handbook of Nonlinear Filtering, 2011.

A. Beskos, A. Jasra, E. A. Muzaffer, and A. M. Stuart, Sequential Monte Carlo methods for Bayesian elliptic inverse problems, Statistics and Computing, vol.30, issue.4, 2015.
DOI : 10.1007/s11222-015-9556-7

J. Marin, P. Pudlo, C. Robert, and R. Ryder, Approximate Bayesian computational methods, Statistics and Computing, vol.6, issue.31, pp.1-14, 2011.
DOI : 10.1007/s11222-011-9288-2
URL : https://hal.archives-ouvertes.fr/hal-00567240

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller, Equation of State Calculations by Fast Computing Machines, The Journal of Chemical Physics, vol.21, issue.6, pp.1087-1091, 1953.
DOI : 10.1063/1.1699114

W. Hastings, Monte Carlo sampling methods using Markov chains and their applications, Biometrika, vol.57, issue.1, pp.97-109, 1970.
DOI : 10.1093/biomet/57.1.97

S. Chib and E. Greenberg, Understanding the Metropolis-Hastings algorithm, Ann. Math. Statist, vol.49, pp.327-335, 1995.

M. Bédard, Weak convergence of Metropolis algorithms for non-i.i.d. target distributions, The Annals of Applied Probability, vol.17, issue.4, pp.1222-1244, 2007.
DOI : 10.1214/105051607000000096

G. O. Roberts and J. S. Rosenthal, Optimal scaling for various Metropolis-Hastings algorithms, Statistical Science, vol.16, issue.4, pp.351-367, 2001.
DOI : 10.1214/ss/1015346320

C. Andrieu and G. Roberts, The pseudo-marginal approach for efficient Monte Carlo computations, The Annals of Statistics, vol.37, issue.2, pp.697-725, 2009.
DOI : 10.1214/07-AOS574

C. Andrieu, A. Doucet, and R. Holenstein, Particle Markov chain Monte Carlo methods, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.50, issue.3, pp.269-342, 2011.
DOI : 10.1111/j.1467-9868.2009.00736.x
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.156.7033

M. Pereyra, N. Dobigeon, H. Batatia, and J. Tourneret, Estimating the Granularity Coefficient of a Potts-Markov Random Field Within a Markov Chain Monte Carlo Algorithm, IEEE Transactions on Image Processing, vol.22, issue.6, pp.2385-2397, 2013.
DOI : 10.1109/TIP.2013.2249076

S. Jarner and E. Hansen, Geometric ergodicity of Metropolis algorithms, Stochastic Processes and their Applications, vol.85, issue.2, pp.341-361, 2000.
DOI : 10.1016/S0304-4149(99)00082-4

A. Beskos, G. Roberts, and A. Stuart, Optimal scalings for local Metropolis???Hastings chains on nonproduct targets in high dimensions, The Annals of Applied Probability, vol.19, issue.3, pp.863-898, 2009.
DOI : 10.1214/08-AAP563

G. Roberts and R. Tweedie, Exponential Convergence of Langevin Distributions and Their Discrete Approximations, Bernoulli, vol.2, issue.4, pp.341-363, 1996.
DOI : 10.2307/3318418

M. Girolami and B. Calderhead, Riemann manifold Langevin and Hamiltonian Monte Carlo methods, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.13, issue.10, pp.123-214, 2011.
DOI : 10.1111/j.1467-9868.2010.00765.x

M. Betancourt, F. Nielsen, and F. Barbaresco, A General Metric for Riemannian Manifold Hamiltonian Monte Carlo, Proc. Nat. Conf. Geometric Sci. Inf, pp.327-334, 2013.
DOI : 10.1007/978-3-642-40020-9_35

Y. Atchadé, An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift, Methodology and Computing in Applied Probability, vol.22, issue.2, pp.235-254, 2006.
DOI : 10.1007/s11009-006-8550-0

N. S. Pillai, A. M. Stuart, and A. H. Thiéry, Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions, The Annals of Applied Probability, vol.22, issue.6, pp.2320-2356, 2012.
DOI : 10.1214/11-AAP828

M. Pereyra, Proximal Markov chain Monte Carlo algorithms, Statistics and Computing, vol.1, issue.4, 2015.
DOI : 10.1007/s11222-015-9567-4

T. Xifara, C. Sherlock, S. Livingstone, S. Byrne, and M. Girolami, Langevin diffusions and the Metropolis-adjusted Langevin algorithm, Statistics & Probability Letters, vol.91, pp.14-19, 2014.
DOI : 10.1016/j.spl.2014.04.002

B. Casella, G. Roberts, and O. Stramer, Stability of Partially Implicit Langevin Schemes and Their MCMC Variants, Methodology and Computing in Applied Probability, vol.1, issue.4
DOI : 10.1007/s11009-010-9196-5

A. Schreck, G. Fort, S. L. Corff, and E. Moulines, A shrinkage-thresholding metropolis adjusted Langevin algorithm for Bayesian variable selection ArXiv Available: arXiv:1312, p.2013

R. Neal, MCMC Using Hamiltonian Dynamics, pp.113-162, 2013.
DOI : 10.1201/b10905-6

M. J. Betancourt, S. Byrne, S. Livingstone, and M. Girolami, The geometric foundations of Hamiltonian Monte Carlo, 2014.

A. Beskos, N. Pillai, G. Roberts, J. Sanz-serna, and A. Stuart, Optimal tuning of the hybrid Monte Carlo algorithm, Bernoulli, vol.19, issue.5A, pp.1501-1534, 2013.
DOI : 10.3150/12-BEJ414

Y. Zhang, C. A. Sutton, J. Shawe-taylor, R. Zemel, P. Bartlett et al., Quasi-Newton methods for Markov chain Monte Carlo, Adv. Neural Inf. . Process. Syst, vol.24, issue.11, pp.2393-2401, 2011.

M. D. Hoffman and A. Gelman, The no-u-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo, J. Mach. Learn. Res, vol.15, pp.1593-1623, 2014.

M. Betancourt, S. Byrne, and M. Girolami, Optimizing the Integrator Step Size for Hamiltonian Monte Carlo ArXiv Available: arXiv:1411, p.2014

K. Latuszy?ski, G. Roberts, and J. Rosenthal, Adaptive Gibbs samplers and related MCMC methods, The Annals of Applied Probability, vol.23, issue.1, pp.66-98, 2013.
DOI : 10.1214/11-AAP806

C. Bazot, N. Dobigeon, J. Tourneret, A. K. Zaas, G. S. Ginsburg et al., Unsupervised Bayesian linear unmixing of gene expression microarrays, BMC Bioinformatics, vol.14, issue.1, 2013.
DOI : 10.1093/biomet/82.4.711
URL : https://hal.archives-ouvertes.fr/hal-00858969

D. A. Van-dyk and T. Park, Partially Collapsed Gibbs Samplers, Journal of the American Statistical Association, vol.103, issue.482, pp.790-796, 2008.
DOI : 10.1198/016214508000000409

T. Park and D. A. Van-dyk, Partially Collapsed Gibbs Samplers: Illustrations and Applications, Journal of Computational and Graphical Statistics, vol.18, issue.2, pp.283-305, 2009.
DOI : 10.1198/jcgs.2009.08108

G. Kail, J. Tourneret, F. Hlawatsch, and N. Dobigeon, Blind Deconvolution of Sparse Pulse Sequences Under a Minimum Distance Constraint: A Partially Collapsed Gibbs Sampler Method, IEEE Transactions on Signal Processing, vol.60, issue.6, pp.2727-2743, 2012.
DOI : 10.1109/TSP.2012.2190066

D. A. Van-dyk and X. Jiao, Metropolis-Hastings Within Partially Collapsed Gibbs Samplers, Journal of Computational and Graphical Statistics, vol.794, issue.2
DOI : 10.1080/10618600.2014.930041

M. Jordan, Z. Gharamani, T. Jaakkola, and L. Saul, An Introduction to Variational Methods for Graphical Models, Mach. Learn, vol.37, pp.183-233, 1999.
DOI : 10.1007/978-94-011-5014-9_5

C. M. Bishop, Pattern Recognition and Machine Learning, 2007.

R. Weinstock, Calculus of Variations, 1974.

C. Peterson and J. R. Anderson, A mean field theory learning algorithm for neural networks, Complex Syst, vol.1, pp.995-1019, 1987.

G. Parisi, Statistical Field Theory, 1988.

M. J. Wainwright and M. I. Jordan, Graphical Models, Exponential Families, and Variational Inference, Foundations and Trends?? in Machine Learning, vol.1, issue.1???2, 2008.
DOI : 10.1561/2200000001

J. S. Yedidia, W. T. Freeman, and Y. Weiss, Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms, IEEE Transactions on Information Theory, vol.51, issue.7, pp.2282-2312, 2005.
DOI : 10.1109/TIT.2005.850085

A. L. Yuille and A. Rangarajan, The concave-convex procedure (CCCP), Proc. Neural Inf. Process. Syst. Conf, pp.1033-1040, 2002.

R. G. Gallager, Low-density parity-check codes, IEEE Transactions on Information Theory, vol.8, issue.1, pp.21-28, 1962.
DOI : 10.1109/TIT.1962.1057683

J. R. Pearl51-]-f, B. J. Kschischang, H. Frey, and . Loeliger, Probabilistic Reasoning in Intelligent Systems Factor graphs and the sum-product algorithm, IEEE Trans. Inf. Theory, vol.47, issue.2, pp.498-519, 1988.

L. R. Rabiner, A tutorial on hidden Markov models and SELECTED applications in speech recognition, Proc. IEEE, pp.257-286, 1989.

G. F. Cooper, The computational complexity of probabilistic inference using bayesian belief networks, Artificial Intelligence, vol.42, issue.2-3, pp.393-405, 1990.
DOI : 10.1016/0004-3702(90)90060-D

K. P. Murphy, Y. Weiss, and M. I. Jordan, Loopy belief propagation for approximate inference: An empirical study, Proc. Uncert, pp.467-475, 1999.

R. J. Mceliece, D. J. Mackay, and J. Cheng, Turbo decoding as an instance of Pearl's "belief propagation" algorithm, IEEE Journal on Selected Areas in Communications, vol.16, issue.2, pp.140-152, 1998.
DOI : 10.1109/49.661103

D. J. Mackay, Information Theory, Inference, and Learning Algorithms, 2003.

W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, Learning low-level vision, Proceedings of the Seventh IEEE International Conference on Computer Vision, pp.25-47, 2000.
DOI : 10.1109/ICCV.1999.790414

J. Boutros and G. Caire, Iterative multiuser joint decoding: unified framework and asymptotic analysis, IEEE Transactions on Information Theory, vol.48, issue.7, pp.1772-1793, 2002.
DOI : 10.1109/TIT.2002.1013125

D. L. Donoho, A. Maleki, and A. Montanari, Message passing algorithms for compressed sensing: I. motivation and construction, IEEE Information Theory Workshop 2010 (ITW 2010), pp.1-5, 2010.
DOI : 10.1109/ITWKSPS.2010.5503193

S. Rangan, Generalized approximate message passing for estimation with random linear mixing, 2011 IEEE International Symposium on Information Theory Proceedings, pp.2168-2172, 2011.
DOI : 10.1109/ISIT.2011.6033942

T. Heskes, M. Opper, W. Wiegerinck, O. Winther, and O. Zoeter, Approximate inference techniques with expectation constraints, Journal of Statistical Mechanics: Theory and Experiment, vol.2005, issue.11
DOI : 10.1088/1742-5468/2005/11/P11015

T. Heskes, O. Zoeter, and W. Wiegerinck, Approximate expectation maximization, Proc. Neural Inf. Process. Syst. Conf, pp.353-360, 2004.

T. Minka, A family of approximate algorithms for Bayesian inference, 2001.

M. Seeger, Expectation propagation for exponential families, 2005.

M. Opper and O. Winther, Expectation consistent free energies for approximate inference, Proc. Neural Inf. Process. Syst. Conf, pp.1001-1008, 2005.

T. Heskes and O. Zoeter, Expectation propagation for approximate inference in dynamic Bayesian networks, Proc. Uncert, pp.313-320, 2002.

M. Bayati and A. Montanari, The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing, IEEE Transactions on Information Theory, vol.57, issue.2, pp.764-785, 2011.
DOI : 10.1109/TIT.2010.2094817

A. Javanmard and A. Montanari, State evolution for general approximate message passing algorithms, with applications to spatial coupling, Information and Inference, vol.2, issue.2, pp.115-144, 2013.
DOI : 10.1093/imaiai/iat004

S. Rangan, P. Schniter, E. Riegler, A. Fletcher, and V. Cevher, Fixed points of generalized approximate message passing with arbitrary matrices, Proc. IEEE Int. Symp. Inf. Theory, pp.664-668, 2013.

F. Krzakala, A. Manoel, E. W. Tramel, and L. Zdeborová, Variational free energies for compressed sensing, 2014 IEEE International Symposium on Information Theory, pp.1499-1503, 2014.
DOI : 10.1109/ISIT.2014.6875083

J. P. Vila and P. Schniter, An Empirical-Bayes Approach to Recovering Linearly Constrained Non-Negative Sparse Signals, IEEE Transactions on Signal Processing, vol.62, issue.18, pp.4689-4703, 2014.
DOI : 10.1109/TSP.2014.2337841

S. Rangan, P. Schniter, and A. Fletcher, On the convergence of generalized approximate message passing with arbitrary matrices, Proc. IEEE Int. Symp. Inf. Theory, pp.236-240, 2014.

J. Vila, P. Schniter, S. Rangan, F. Krzakala, and L. Zdeborová, Adaptive damping and mean removal for the generalized approximate message passing algorithm, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2021-2025, 2015.
DOI : 10.1109/ICASSP.2015.7178325
URL : https://hal.archives-ouvertes.fr/cea-01140721

S. Rangan, A. K. Fletcher, P. Schniter, and U. Kamilov, Inference for feneralized linear models via alternating directions and Bethe free energy minimization, p.2015

L. Bottou and Y. Lecun, Large scale online learning, Proc. Annu. Conf. Neur. Inf. Process. Syst, p.8, 2004.

S. Theodoridis, Machine Learning: A Bayesian and Optimization Perspective, 2015.

S. O. Haykin, Adaptive Filter Theory, 2002.

A. H. Sayed, Adaptive Filters, 2011.
DOI : 10.1002/9780470374122

P. L. Combettes, J. Pesquet, H. H. Bauschke, R. Burachik, and P. L. Combettes, Proximal Splitting Methods in Signal Processing, Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp.185-212, 2010.
DOI : 10.1007/978-1-4419-9569-8_10
URL : https://hal.archives-ouvertes.fr/hal-00643807

J. Duchi and Y. Singer, Efficient online and batch learning using forward backward splitting, J. Mach. Learn. Res, vol.10, pp.2899-2934, 2009.

Y. F. Atchadé, G. Fort, and E. Moulines, On stochastic proximal gradient algorithms, p.2014

L. Rosasco, S. Villa, and B. C. V?, A stochastic forward-backward splitting method for solving monotone inclusions in Hilbert spaces, p.2014

P. L. Combettes and J. Pesquet, Stochastic Quasi-Fej??r Block-Coordinate Fixed Point Iterations with Random Sweeping, SIAM Journal on Optimization, vol.25, issue.2, pp.1221-1248, 2015.
DOI : 10.1137/140971233

H. Robbins and S. Monro, A Stochastic Approximation Method, The Annals of Mathematical Statistics, vol.22, issue.3, pp.400-407, 1951.
DOI : 10.1214/aoms/1177729586

J. M. Ermoliev and Z. V. Nekrylova, The method of stochastic gradients and its application, Seminar: Theory of Optimal Solutions, pp.24-47, 1967.

O. V. Guseva, Convergence rate of the method of generalized stochastic gradients, Cybernetics, vol.7, issue.4, pp.143-145, 1971.
DOI : 10.1007/BF01071048

D. P. Bertsekas and J. N. Tsitsiklis, Gradient Convergence in Gradient methods with Errors, SIAM Journal on Optimization, vol.10, issue.3, pp.627-642, 2000.
DOI : 10.1137/S1052623497331063

F. Bach and E. Moulines, Non-asymptotic analysis of stochastic approximation algorithms for machine learning, Proc. Ann. Conf. Neur. Inf. Proc. Syst., Granada, Spain, pp.451-459, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00608041

A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, Robust Stochastic Approximation Approach to Stochastic Programming, SIAM Journal on Optimization, vol.19, issue.4, pp.1574-1609, 2008.
DOI : 10.1137/070704277
URL : https://hal.archives-ouvertes.fr/hal-00976649

B. T. Polyak and A. B. Juditsky, Acceleration of Stochastic Approximation by Averaging, SIAM Journal on Control and Optimization, vol.30, issue.4, pp.838-855, 1992.
DOI : 10.1137/0330046

S. Shalev-shwartz, Y. Singer, and N. Srebro, Pegasos, Proceedings of the 24th international conference on Machine learning, ICML '07, pp.807-814, 2007.
DOI : 10.1145/1273496.1273598

L. Xiao, Dual averaging methods for regularized stochastic learning and online optimization, J. Mach. Learn. Res, vol.11, pp.2543-2596, 2010.

J. Duchi, S. Shalev-shwartz, Y. Singer, and A. Tewari, Composite objective mirror descent, Proc. Conf. Learn. Theory, pp.14-26, 2010.

L. W. Zhong and J. T. Kwok, Accelerated stochastic gradient method for composite regularization, J. Mach. Learn. Rech, vol.33, pp.1086-1094, 2014.

H. Ouyang, N. He, L. Tran, and A. Gray, Stochastic alternating direction method of multipliers, Proc. 30th Int. Conf, pp.80-88, 2013.

T. Suzuki, Dual averaging and proximal gradient descent for online alternating direction multiplier method, Proc. 30th Int. Conf, pp.392-400, 2013.

W. Zhong and J. Kwok, Fast stochastic alternating direction method of multipliers, Tech. Rep. Beijing, 2014.

Y. Xu and W. Yin, Block Stochastic Gradient Iteration for Convex and Nonconvex Optimization, SIAM Journal on Optimization, vol.25, issue.3, p.2014
DOI : 10.1137/140983938

C. Hu, J. T. Kwok, and W. Pan, Accelerated gradient methods for stochastic optimization and online learning, Proc. Ann. Conf. Neur. Inf. Process. Syst, pp.11-12, 2009.

G. Lan, An optimal method for stochastic composite optimization, Mathematical Programming, vol.24, issue.1-2, pp.365-397, 2012.
DOI : 10.1007/s10107-010-0434-y

Q. Lin, X. Chen, and J. Peña, A sparsity preserving stochastic gradient methods for sparse regression, Computational Optimization and Applications, vol.68, issue.3, pp.455-482, 2014.
DOI : 10.1007/s10589-013-9633-9

J. Duchi, E. Hazan, and Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization, J. Mach. Learn. Res, vol.12, pp.2121-2159, 2011.

S. Amari, Natural Gradient Works Efficiently in Learning, Neural Computation, vol.37, issue.2, pp.251-276, 1998.
DOI : 10.1103/PhysRevLett.76.2188

E. Chouzenoux, J. Pesquet, and A. Florescu, A stochastic 3MG algorithm with application to 2D filter identification, Proc. 22nd Eur. Signal Process. Conf, pp.1587-1591, 2014.
URL : https://hal.archives-ouvertes.fr/hal-01077261

B. Widrow and S. D. Stearns, Adaptive Signal Processing, 1985.

H. J. Kushner and G. G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, 2003.

S. L. Gay and S. Tavathia, The fast affine projection algorithm, Proc. Int. Conf. Acoust., Speech Signal Process, pp.3023-3026, 1995.

A. W. Khong and P. A. Naylor, Efficient Use Of Sparse Adaptive Filters, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers, pp.1375-1379, 2006.
DOI : 10.1109/ACSSC.2006.354982

C. Paleologu, J. Benesty, and S. Ciochina, An improved proportionate NLMS algorithm based on the norm, Proc. Int. Conf. Acoust., Speech Signal Process, 2010.

O. Hoshuyama, R. A. Goubran, and A. Sugiyama, A generalized proportionate variable step-size algorithm for fast changing acoustic environments, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, pp.161-164, 2004.
DOI : 10.1109/ICASSP.2004.1326788

C. Paleologu, J. Benesty, and F. Albu, Regularization of the improved proportionate affine projection algorithm, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.169-172, 2012.
DOI : 10.1109/ICASSP.2012.6287844

Y. Chen, Y. Gu, and A. O. Hero, Sparse LMS for system identification, Proc. Int. Conf. Acoust., Speech Signal Process, pp.3125-3128, 2009.

Y. Chen, Y. Gu, and A. O. Hero, Regularized least-mean-square algorithms, 2010.

R. Meng, R. C. De-lamare, and V. H. Nascimento, Sparsity-aware affine projection adaptive algorithms for system identification, Proc. Sensor Signal Process. Defence, pp.1-5, 2011.

L. V. Markus, W. A. Martins, and P. S. Diniz, Affine projection algorithms for sparse system identification, Proc. Int. Conf. Acoust., Speech, Signal Process, pp.5666-5670, 2013.

L. V. Markus, T. N. Tadeu, N. Ferreira, W. A. Martins, and P. S. Diniz, Sparsity-aware data-selective adaptive filters, IEEE Trans. Signal Process, vol.62, issue.17, pp.4557-4572, 2014.

Y. Murakami, M. Yamagishi, M. Yukawa, and I. Yamada, A sparse adaptive filtering using time-varying soft-thresholding techniques, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.3734-3737, 2010.
DOI : 10.1109/ICASSP.2010.5495870

M. Yamagishi, M. Yukawa, and I. Yamada, Acceleration of adaptive proximal forward-backward splitting method and its application to sparse system identification, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.4296-4299, 2011.
DOI : 10.1109/ICASSP.2011.5947303

S. Ono, M. Yamagishi, and I. Yamada, A sparse system identification by using adaptively-weighted total variation via a primal-dual splitting approach, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.6029-6033, 2013.
DOI : 10.1109/ICASSP.2013.6638822

D. Angelosante, J. A. Bazerque, and G. B. Giannakis, Online Adaptive Estimation of Sparse Signals: Where RLS Meets the <formula formulatype="inline"><tex Notation="TeX">$\ell_1$</tex> </formula>-Norm, IEEE Transactions on Signal Processing, vol.58, issue.7, pp.3436-3447, 2010.
DOI : 10.1109/TSP.2010.2046897

B. Babadi, N. Kalouptsidis, and V. Tarokh, SPARLS: The Sparse RLS Algorithm, IEEE Transactions on Signal Processing, vol.58, issue.8, pp.4013-4025, 2010.
DOI : 10.1109/TSP.2010.2048103

Y. Kopsinis, K. Slavakis, and S. Theodoridis, Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted <formula formulatype="inline"> <tex Notation="TeX">$\ell_{1}$</tex></formula> Balls, IEEE Transactions on Signal Processing, vol.59, issue.3, pp.936-952, 2011.
DOI : 10.1109/TSP.2010.2090874

K. Slavakis, Y. Kopsinis, S. Theodoridis, and S. Mclaughlin, Generalized Thresholding and Online Sparsity-Aware Learning in a Union of Subspaces, IEEE Transactions on Signal Processing, vol.61, issue.15, pp.3760-3773, 2013.
DOI : 10.1109/TSP.2013.2264464

A. Repetti, E. Chouzenoux, and J. Pesquet, A parallel block-coordinate approach for primal-dual splitting with arbitrary random block selection, 2015 23rd European Signal Processing Conference (EUSIPCO), 2015.
DOI : 10.1109/EUSIPCO.2015.7362380
URL : https://hal.archives-ouvertes.fr/hal-01370498

D. Blatt, A. O. Hero, and H. Gauchman, A Convergent Incremental Gradient Method with a Constant Step Size, SIAM Journal on Optimization, vol.18, issue.1, pp.29-51, 1998.
DOI : 10.1137/040615961

D. P. Bertsekas, Incremental gradient, subgradient, and proximal methods for convex optimization: A survey, 2010.

M. Schmidt, N. L. Roux, and F. Bach, Minimizing finite sums with the stochastic average gradient, p.2013
URL : https://hal.archives-ouvertes.fr/hal-00860051

A. Defazio, J. Domke, and T. Cartano, Finito: A faster, permutable incremental gradient method for big data problems, Proc. 31st Int. Conf, pp.1125-1133, 2014.

A. Defazio, F. Bach, and S. Lacoste, SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives, Proc. Ann. Conf. Neur. Inf. Proc. Syst, pp.1646-1654, 2014.
URL : https://hal.archives-ouvertes.fr/hal-01016843

R. Johnson and T. Zhang, Accelerating stochastic gradient descent using predictive variance reduction, Proc. Annu. Conf. Neur. Inf. Process. Syst, pp.315-323, 2013.

J. Kone?ný, J. Liu, P. Richtárik, and M. Taká?, mS2GD: Mini-batch semi-stochastic gradient descent in the proximal setting, p.2014

P. Tseng, Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization, Journal of Optimization Theory and Applications, vol.109, issue.3, pp.475-494, 2001.
DOI : 10.1023/A:1017501703105

E. Chouzenoux, J. Pesquet, and A. Repetti, A block coordinate variable metric forward-backward algorithm Available: http://www.optimization-online.org/DB_HTML, p.2014, 2013.

J. Pesquet and A. Repetti, A class of randomized primal-dual algorithms for distributed optimization, J. Nonlinear Convex Anal, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01077615

N. Komodakis and J. Pesquet, Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems, IEEE Signal Processing Magazine, vol.32, issue.6, pp.31-54, 2014.
DOI : 10.1109/MSP.2014.2377273
URL : https://hal.archives-ouvertes.fr/hal-01246610

P. Richtárik and M. Taká?, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Mathematical Programming, vol.67, issue.1, pp.1-38, 2014.
DOI : 10.1007/s10107-012-0614-z

I. Necoara and A. Patrascu, A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints, Computational Optimization and Applications, vol.25, issue.1???3, pp.307-337, 2014.
DOI : 10.1007/s10589-013-9598-8

Z. Lu and L. Xiao, On the complexity analysis of randomized blockcoordinate descent methods, Math. Program, 2015.

O. Fercoq and P. Richtárik, Accelerated, Parallel, and Proximal Coordinate Descent, SIAM Journal on Optimization, vol.25, issue.4, p.2014
DOI : 10.1137/130949993

Y. Nesterov, Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems, SIAM Journal on Optimization, vol.22, issue.2, pp.341-362, 2012.
DOI : 10.1137/100802001

P. L. Combettes, D. D?ng, and B. C. V?, Dualization of signal recovery problems Set-Valued Var, Anal, vol.18, pp.373-404, 2010.

S. Shalev-shwartz and T. Zhang, Stochastic dual coordinate ascent methods for regularized loss minimization, J. Mach. Learn. Res, vol.14, pp.567-599, 2013.

S. Shalev-shwartz and T. Zhang, Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization, Mathematical Programming, vol.46, issue.6, 2014.
DOI : 10.1007/s10107-014-0839-0

M. Jaggi, V. Smith, J. Taká?, S. Terhorst, T. Krishnan et al., Communication-efficient distributed dual coordinate ascent, Proc. Annu Conf. Neur. Inf. Process. Syst, pp.3068-3076, 2014.

Z. Qu, P. Richtárik, and T. Zhang, Randomized dual coordinate ascent with arbitrary sampling, p.2014

F. Iutzeler, P. Bianchi, P. Ciblat, and W. Hachem, Asynchronous distributed optimization using a randomized alternating direction method of multipliers, 52nd IEEE Conference on Decision and Control, pp.3671-3676, 2013.
DOI : 10.1109/CDC.2013.6760448
URL : https://hal.archives-ouvertes.fr/hal-00868412

]. P. Bianchi, W. Hachem, and F. Iutzeler, A stochastic coordinate descent primal-dual algorithm and applications to large-scale composite optimization, p.2014

S. Allassonniere and E. Kuhn, Convergent stochastic Expectation Maximization algorithm with efficient sampling in high dimension. Application to deformable template model estimation, Computational Statistics & Data Analysis, vol.91, 2012.
DOI : 10.1016/j.csda.2015.04.011
URL : https://hal.archives-ouvertes.fr/hal-00720617

Y. Marnissi, A. Benazza-benyahia, E. Chouzenoux, and J. Pesquet, Majorize-minimize adapted Metropolis Hastings algorithm. Application to multichannel image recovery, Proc. Eur. Signal Process. Conf. (EUSIPCO'14), pp.1332-1336, 2014.
URL : https://hal.archives-ouvertes.fr/hal-01077273

J. J. Moreau, Fonctions convexes duales et points proximaux dans un espace Hilbertien, C. R. Acad. Sci. Paris Sér. A, vol.255, pp.2897-2899, 1962.

L. Chaari, J. Tourneret, C. Chaux, and H. Batatia, A Hamiltonian Monte Carlo Method for Non-Smooth Energy Sampling, IEEE Transactions on Signal Processing, vol.64, issue.21, 2014.
DOI : 10.1109/TSP.2016.2585120
URL : https://hal.archives-ouvertes.fr/hal-01376544

H. Rue, Fast sampling of Gaussian Markov random fields, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.63, issue.2, pp.325-338, 2001.
DOI : 10.1111/1467-9868.00288

W. F. Trench, An Algorithm for the Inversion of Finite Toeplitz Matrices, Journal of the Society for Industrial and Applied Mathematics, vol.12, issue.3, pp.515-522, 1964.
DOI : 10.1137/0112045

D. Geman and C. Yang, Nonlinear image recovery with half-quadratic regularization, IEEE Transactions on Image Processing, vol.4, issue.7, pp.932-946, 1995.
DOI : 10.1109/83.392335

G. Papandreou, A. L. Yuille, J. Lafferty, C. Williams, J. Shawe-taylor et al., Gaussian sampling by local perturbations, Adv. Neural Inf. Process. Syst. 23 (NIPS'10), pp.1858-1866, 2010.

F. Orieux, O. Feron, and J. Giovannelli, Sampling High-Dimensional Gaussian Distributions for General Linear Inverse Problems, IEEE Signal Processing Letters, vol.19, issue.5, pp.251-254, 2012.
DOI : 10.1109/LSP.2012.2189104
URL : https://hal.archives-ouvertes.fr/hal-00779449

C. Gilavert, S. Moussaoui, and J. Idier, Efficient Gaussian Sampling for Solving Large-Scale Inverse Problems Using MCMC, IEEE Transactions on Signal Processing, vol.63, issue.1, pp.70-80, 2015.
DOI : 10.1109/TSP.2014.2367457
URL : https://hal.archives-ouvertes.fr/hal-01059414

O. Alfred, both in electrical engineering Since 1984, he has been with the University of Michigan, Ann Arbor, where he is the R. Jamison and Betty Williams Professor of Engineering and co-director of the Michigan Institute for Data Science (MIDAS) His primary appointment is in the Department of Electrical Engineering and Computer Science and he also has appointments, by courtesy, in the Department of Biomedical Engineering and the Department of Statistics. From he held the Digiteo Chaire d'Excellence, sponsored by Digiteo Research Park in Paris, located at the Ecole Superieure d'Electricite, Gif-sur-Yvette, France. He has held other visiting positions at LIDS Massachusetts Institute of He is a Fellow of the He received the University of Michigan Distinguished Faculty Achievement Award He has been plenary and keynote speaker at several workshops and conferences. He has received several best paper awards including: an, III (S'79?M'84?SM'98?F'98) received the B.S. (summa cum laude) from Boston University Ecole Normale Supérieure de Lyon Ecole Nationale Supérieure des Télécommunications Lucent Bell Laboratories Scientific Research Labs of the Ford Motor Company Ecole Nationale Superieure des Techniques Avancees (ENSTA), Ecole Superieure d'Electricite Best Original Paper Award from the Journal of Flow Cytometry Best Magazine Paper Award from the IEEE Signal Processing Society an IEEE ICASSP Best Student Paper Award, 1980.