Skip to Main content Skip to Navigation
Book sections

Analysis of the AutoML Challenge series 2015-2018

Abstract : The ChaLearn AutoML Challenge 1 (NIPS 2015-ICML 2016) consisted of six rounds of a machine learning competition of progressive difficulty, subject to limited computational resources. It was followed by one round of AutoML challenge (PAKDD 2018). The AutoML setting differs from former model selection/hyper-parameter selection challenges, such as the one we previously organized for NIPS 2006: the participants aim to develop fully automated and computationally efficient systems, capable of being trained and tested without human intervention, with code submission. This paper analyzes the results of these competitions and provides details about the datasets, which were not revealed to the participants. The solutions of the winners are systematically benchmarked over all datasets of all rounds and compared with canonical machine learning algorithms available in scikit-learn. All materials discussed in this paper (data and code) have been made publicly available at
Document type :
Book sections
Complete list of metadata
Contributor : Zhengying Liu Connect in order to contact the contributor
Submitted on : Friday, October 26, 2018 - 3:06:42 PM
Last modification on : Thursday, July 8, 2021 - 3:50:32 AM
Long-term archiving on: : Sunday, January 27, 2019 - 3:01:44 PM


Files produced by the author(s)


  • HAL Id : hal-01906197, version 1


Isabelle Guyon, Lisheng Sun-Hosoya, Marc Boullé, Hugo Escalante, Sergio Escalera, et al.. Analysis of the AutoML Challenge series 2015-2018. Frank Hutter; Lars Kotthoff; Joaquin Vanschoren. AutoML: Methods, Systems, Challenges, Springer Verlag, In press, The Springer Series on Challenges in Machine Learning. ⟨hal-01906197⟩



Les métriques sont temporairement indisponibles