The contribution of the notion of hapax legomena to word alignment

Adrien Lardilleux (1) , Yves Lepage (2)

1 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
2 Equipe Hultech - Laboratoire GREYC - UMR6072

Current techniques in word alignment disregard words with a low frequency because they would not be useful. Against this belief, this paper shows that, in particular, the notion of hapax legomena may contribute to word alignment to a large extent. In an experiment, we show that pairs of corpus hapaxes contribute the majority of the best word alignments. In addition, we show that the notion of sentence hapax justifies a practical and common simplification of a standard alignment method.

Mots clés: Sub-sentential alignment; low frequency terms; hapax

Domaines: Autre [cs.OH]; Informatique et langage [cs.CL]; Traitement du texte et du document

Adrien Lardilleux, Yves Lepage. The contribution of the notion of hapax legomena to word alignment. The 3rd Language and Technology Conference (LTC'07), Oct 2007, Poznań, Poland. pp.458-462. ⟨hal-00252026v2⟩