Occam's hammer: a link between randomized learning and multiple testing FDR control

Gilles Blanchard (1, 2, 3) , François Fleuret (4, 5)

1 Fraunhofer FIRST.IDA - Department Intelligent Data Analysis [Berlin]
2 LMO - Laboratoire de Mathématiques d'Orsay
3 CNRS - Centre National de la Recherche Scientifique
4 CVLAB - Computer Vision Laboratory [EPFL]
5 IDIAP Research Institute

We establish a generic theoretical tool to construct probabilistic bounds for algorithms where the output is a subset of objects from an initial pool of candidates (or more generally, a probability distribution on said pool). This general device, dubbed "Occam's hammer'', acts as a meta layer when a probabilistic bound is already known on the objects of the pool taken individually, and aims at controlling the proportion of the objects in the set output not satisfying their individual bound. In this regard, it can be seen as a non-trivial generalization of the ``union bound with a prior'' ("Occam's razor''), a familiar tool in learning theory. We give applications of this principle to randomized classifiers (providing an interesting alternative approach to PAC-Bayes bounds) and multiple testing (where it allows to retrieve exactly and extend the so-called Benjamini-Yekutieli testing procedure).

Mots clés: multiple testing; randomized estimation; PAC-Bayes; False Discovery Rate (FDR); learning theory

Domaines: Statistiques [math.ST]; Apprentissage [cs.LG]

Gilles Blanchard, François Fleuret. Occam's hammer: a link between randomized learning and multiple testing FDR control. Learning Theory -- 20th Annual Conference on Learning Theory (COLT 2007), Jun 2007, San Diego, United States. pp.112-126, ⟨10.1007/978-3-540-72927-3_10⟩. ⟨hal-00090244⟩