New Recurrent Neural Network Variants for Sequence Labeling

Marco Dinarelli (1, 2, 3, 4) , Isabelle Tellier (1, 2, 3, 4)

1 ENS-PSL - École normale supérieure - Paris
2 USPC - Université Sorbonne Paris Cité
3 PSL - Université Paris Sciences et Lettres
4 Lattice - Lattice - Langues, Textes, Traitements informatiques, Cognition - UMR 8094

In this paper we study different architectures of Recurrent Neural Networks (RNN) for sequence labeling tasks. We propose two new variants of RNN and we compare them to the more traditional RNN architectures of Elman and Jordan. We explain in details the advantages of these new variants of RNNs with respect to Elman's and Jordan's RNN. We evaluate all models, either new or traditional, on three different tasks: POS-tagging of the French Treebank, and two tasks of Spoken Language Understanding (SLU), namely ATIS and MEDIA. The results we obtain clearly show that the new variants of RNN are more effective than the traditional ones.

Mots clés: Machine Learning; Recurrent Neural Networks; Spoken Language Understanding; Slot Filling

Domaines: Informatique et langage [cs.CL]

Marco Dinarelli, Isabelle Tellier. New Recurrent Neural Network Variants for Sequence Labeling. 17th International Conference on Intelligent Text Processing and Computational Linguistics, Apr 2016, Konya, Turkey. ⟨hal-01489955⟩