Phase Transitions within Grammatical Inference

Nicolas Pernot , Antoine Cornuéjols (1) , Michèle Sebag (2)

1 MIA-Paris - Mathématiques et Informatique Appliquées
2 LRI - Laboratoire de Recherche en Informatique

It is now well-known that the feasibility of induc-tive learning is ruled by statistical properties linking the empirical risk minimization principle and the "capacity" of the hypothesis space. The discovery , a few years ago, of a phase transition phenomenon in inductive logic programming proves that other fundamental characteristics of the learning problems may similarly affect the very possibility of learning under very general conditions. Our work examines the case of grammatical inference. We show that while there is no phase transition when considering the whole hypothesis space, there is a much more severe "gap" phenomenon affecting the effective search space of standard grammatical induction algorithms for de-terministic finite automata (DFA). Focusing on the search heuristics of the RPNI and RED-BLUE algorithms , we show that they overcome this problem to some extent, but that they are subject to over-generalization. The paper last suggests some directions for new generalization operators, suited to this Phase Transition phenomenon.

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]

Nicolas Pernot, Antoine Cornuéjols, Michèle Sebag. Phase Transitions within Grammatical Inference. Int. Joint Conf. on Artificial Intelligence (IJCAI), Jul 2005, Edinburgh, United Kingdom. ⟨hal-02482160⟩