Performance analysis of greedy algorithms for minimising a Maximum Mean Discrepancy

Luc Pronzato (1, 2)

1 Laboratoire d'Informatique, Signaux, et Systèmes de Sophia-Antipolis (I3S) / Equipe SYSTEMES
2 GdR MASCOT-NUM - Méthodes d'Analyse Stochastique des Codes et Traitements Numériques

We analyse the performance of several iterative algorithms for the quantisation of a probability measure µ, based on the minimisation of a Maximum Mean Discrepancy (MMD). Our analysis includes kernel herding, greedy MMD minimisation and Sequential Bayesian Quadrature (SBQ). We show that the finite-sample-size approximation error, measured by the MMD, decreases as 1/n for SBQ and also for kernel herding and greedy MMD minimisation when using a suitable step-size sequence. The upper bound on the approximation error is slightly better for SBQ, but the other methods are significantly faster, with a computational cost that increases only linearly with the number of points selected. This is illustrated by two numerical examples, with the target measure µ being uniform (a space-filling design application) and with µ a Gaussian mixture. They suggest that the bounds derived in the paper are overly pessimistic, in particular for SBQ. The sources of this pessimism are identified but seem difficult to counter.

Mots clés: Maximum Mean Discrepancy; quantisation; greedy algorithm; sequential Bayesian quadrature; kernel herding; space-filling design; computer experiments

Domaines: Statistiques [math.ST]

Luc Pronzato. Performance analysis of greedy algorithms for minimising a Maximum Mean Discrepancy. Statistics and Computing, In press. ⟨hal-03114891v2⟩