A Novel Gradient Accumulation Method for Calibration of Named Entity Recognition Models

Grégor Jouet (1, 2, 3) , Clément Duhart (2) , Jacopo Staiano (3) , Francis Rousseaux (4) , Cyril de Runz (1)

1 BDTLN - Bases de données et traitement des langues naturelles
2 PULV - Pôle Universitaire Léonard de Vinci
3 reciTAL
4 CRESTIC - Centre de Recherche en Sciences et Technologies de l'Information et de la Communication - EA 3804

The adoption of deep learning models has brought significant performance improvements across several research fields, such as computer vision and natural language processing. However, their "black-box" nature yields the downside of poor explainability: in particular, several real-world applications require-to varying extents-reliable confidence scores associated to a model's prediction. The relation between a model's accuracy and confidence is typically referred to as calibration. In this work, we propose a novel calibration method based on gradient accumulation in conjunction with existing loss regularization techniques. Our experiments on the Named Entity Recognition task show an improvement of the performance/calibration ratio compared to the current methods.

Mots clés: calibration; ner; uncertainty; noise injection

Domaines: Informatique [cs]

Grégor Jouet, Clément Duhart, Jacopo Staiano, Francis Rousseaux, Cyril de Runz. A Novel Gradient Accumulation Method for Calibration of Named Entity Recognition Models. International Joint Conference on Neural Networks (IJCNN), 2022, Padoue, Italy. pp.1-8, ⟨10.1109/IJCNN55064.2022.9892324⟩. ⟨hal-03792800⟩