A note on phase transition and computational pitfalls of learning from sequences

Antoine Cornuéjols (1) , Michèle Sebag (2)

1 MIA-Paris - Mathématiques et Informatique Appliquées
2 LRI - Laboratoire de Recherche en Informatique

An ever greater range of applications call for learning from sequences. Gram- mar induction is one prominent tool for sequence learning, it is therefore important to know its properties and limits. This paper presents a new type of analysis for inductive learning. A few years ago, the discovery of a phase transition phenomenon in inductive logic program- ming proved that fundamental characteristics of the learning problems may affect the very possibility of learning under very general conditions. We show that, in the case of grammatical inference, while there is no phase transition when considering the whole hypothesis space, there is a much more severe “gap” phenomenon affecting the effective search space of standard gram- matical induction algorithms for deterministic finite automata (DFA). Focusing on standard search heuristics, we show that they overcome this difficulty to some ex- tent, but that they are subject to overgeneralization. The paper last suggests some directions to alleviate this problem.

Mots clés: grammatical inference; sequence learning; supervised learning

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]

Antoine Cornuéjols, Michèle Sebag. A note on phase transition and computational pitfalls of learning from sequences. Journal of Intelligent Information Systems, 2008, 31 (2), pp.177-189. ⟨10.1007/s10844-008-0063-6⟩. ⟨hal-02480319⟩