Validated Intraclass Correlation Statistics to Test Item Performance Models

Pierre Courrieu (1) , Muriele Brand-d'Abrescia (2) , Ronald Peereman (3) , Daniel Spieler (4) , Arnaud Rey (1)

1 LPC - Laboratoire de psychologie cognitive
2 LEAD - Laboratoire d'Etude de l'Apprentissage et du Développement [Dijon]
3 LPNC - Laboratoire de Psychologie et NeuroCognition
4 School of Psychology - Georgia Institute of Technology

A new method, with an application program in Matlab code, is proposed for testing item performance models on empirical databases. This method uses data intraclass correlation statistics as expected correlations to which one compares simple functions of correlations between model predictions and observed item performance. The method rests on a data population model whose validity for the considered data is suitably tested, and has been verified for three behavioural measure databases. Contrarily to usual model selection criteria, this method provides an effective way of testing under-fitting and over-fitting, answering the usually neglected question "does this model suitably account for these data?"

Mots clés: Model test; misfit detection; intraclass correlation; item performance databases

Domaines: Méthodologie [stat.ME]

Pierre Courrieu, Muriele Brand-d'Abrescia, Ronald Peereman, Daniel Spieler, Arnaud Rey. Validated Intraclass Correlation Statistics to Test Item Performance Models. Behavior Research Methods, 2011, 43 (1), pp. 37-55, DOI 10.3758/s13428-010-0020-5. ⟨10.3758/s13428-010-0020-5⟩. ⟨hal-00522759v2⟩