Analysis of the AutoML Challenge series 2015-2018

Abstract : The ChaLearn AutoML Challenge 1 (NIPS 2015-ICML 2016) consisted of six rounds of a machine learning competition of progressive difficulty, subject to limited computational resources. It was followed by one round of AutoML challenge (PAKDD 2018). The AutoML setting differs from former model selection/hyper-parameter selection challenges, such as the one we previously organized for NIPS 2006: the participants aim to develop fully automated and computationally efficient systems, capable of being trained and tested without human intervention, with code submission. This paper analyzes the results of these competitions and provides details about the datasets, which were not revealed to the participants. The solutions of the winners are systematically benchmarked over all datasets of all rounds and compared with canonical machine learning algorithms available in scikit-learn. All materials discussed in this paper (data and code) have been made publicly available at http://automl.chalearn.org/.
Type de document :
Chapitre d'ouvrage
Frank Hutter; Lars Kotthoff; Joaquin Vanschoren. AutoML: Methods, Systems, Challenges, Springer Verlag, In press, The Springer Series on Challenges in Machine Learning, 〈https://www.automl.org/book/〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01906197
Contributeur : Zhengying Liu <>
Soumis le : vendredi 26 octobre 2018 - 15:06:42
Dernière modification le : jeudi 7 février 2019 - 17:21:35
Document(s) archivé(s) le : dimanche 27 janvier 2019 - 15:01:44

Fichier

automl-challenges.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01906197, version 1

Citation

Isabelle Guyon, Lisheng Sun-Hosoya, Marc Boullé, Hugo Escalante, Sergio Escalera, et al.. Analysis of the AutoML Challenge series 2015-2018. Frank Hutter; Lars Kotthoff; Joaquin Vanschoren. AutoML: Methods, Systems, Challenges, Springer Verlag, In press, The Springer Series on Challenges in Machine Learning, 〈https://www.automl.org/book/〉. 〈hal-01906197〉

Partager

Métriques

Consultations de la notice

161

Téléchargements de fichiers

278