Detecting Derivatives using Specific and Invariant Descriptors

Fabien Poulard , Nicolas Hernandez (1) , Béatrice Daille (2)

1 LINA - Laboratoire d'Informatique de Nantes Atlantique
2 LS2N - équipe TALN - Traitement Automatique du Langage Naturel

This paper explores the detection of derivation linksbetween texts (otherwise called plagiarism, near-duplication,revision, etc.) at the document level. We evaluate the use of textualelements implementing the ideas of specificity and invariance aswell as their combination to characterize derivatives. We builta French press corpus based on Wikinews revisions to run thisevaluation. We obtain performances similar to the state of theart method (n-grams overlap) while reducing the signature sizeand so, the processing costs. In order to ensure the verifiabilityand the reproducibility of our results we make our code as wellas our corpus available to the community.

Mots clés: extual derivatives; detection of derivations; near-duplicates; revisions; linguistic descriptors; French corpus

Domaines: Informatique et langage [cs.CL]

Fabien Poulard, Nicolas Hernandez, Béatrice Daille. Detecting Derivatives using Specific and Invariant Descriptors. Twelveth International Conference on Computational Linguistics and Intelligent Text Processing (CICLING 2010), Mar 2010, Tokyo, Japan. pp.7--13. ⟨hal-01160955⟩