Survey of machine-learning experimental methods at NeurIPS2019 and ICLR2020

Xavier Bouthillier (1) , Gaël Varoquaux (1, 2)

1 MILA - Montreal Institute for Learning Algorithms [Montréal]
2 Inria Saclay - Ile de France

How do machine-learning researchers run their empirical validation? In the context of a push for improved reproducibility and benchmarking, this question is important to develop new tools for model comparison. This document summarizes a simple survey about experimental procedures, sent to authors of published papers at two leading conferences, NeurIPS 2019 and ICLR 2020. It gives a simple picture of how hyper-parameters are set, how many baselines and datasets are included, or how seeds are used.

Domaines: Intelligence artificielle [cs.AI]

Xavier Bouthillier, Gaël Varoquaux. Survey of machine-learning experimental methods at NeurIPS2019 and ICLR2020. [Research Report] Inria Saclay Ile de France. 2020. ⟨hal-02447823⟩