On proportional volume sampling for experimental design in general spaces
Résumé
Optimal experimental design for linear regression is a fundamental research topic in statistics and related fields. For finite design spaces, recent progress has shown that random designs drawn using proportional volume sampling (PVS) lead to approximation guarantees for A-optimal design. Proportional volume sampling strikes the balance between forcing the design nodes to jointly fill the design space, while marginally staying in regions of high mass under the solution of a relaxed convex version of the optimal problem. In this paper, we examine some of the statistical implications of PVS. First, we extend PVS to generic Polish design spaces, and we show that not only are the A-optimality approximation guarantees preserved, but we obtain similar guarantees for D-optimal design. Through a connection with determinantal point processes, another type of repulsive random design, we show that PVS can be sampled in polynomial time. Unfortunately, in spite of its mathematical elegance and computational tractability, we demonstrate on a simple example that the practical implications of general PVS are likely limited. In the second part of the paper, we focus on applications and rather investigate the use of PVS as a subroutine for stochastic search heuristics. We demonstrate that PVS is a robust addition to the practitioner's toolbox, especially when the regression functions are nonstandard and the design space is low-dimensional but has a complicated shape (e.g., nonlinear boundaries, several connected components).