General or Idiosyncratic Item Effects: What Is the Good Target for Models?

Pierre Courrieu (1) , Arnaud Rey (1)

1 LPC - Laboratoire de psychologie cognitive

Recently, Adelman, Marquis, Sabatos-DeVito, and Estes (2013) formulated severe criticisms about approaches based on averaging item response times (RTs) over participants and associated methods for estimating the amount of item variance that models should try to account for. Their main argument was that item effects include stable idiosyncratic effects. In this comment, we provide supplementary empirical evidence that this assertion is indeed valid. However, the actual implications of this result are not those defended in Adelman et al. (2013), where there seems to be confusion about the precision of measures and the nature of target effects. Indeed, basic statistical considerations show that any arbitrary data precision level can be achieved in all cases using an appropriate number of observations per item, whereas general and idiosyncratic item effects are both targets of interest for modeling but in different objectives.

Keywords: intraclass correlation coefficient; large-scale behavioral databases; idiosyncratic item effects; systematic item variance; reading model testing

Domains: Methodology [stat.ME]; Psychology

Pierre Courrieu, Arnaud Rey. General or Idiosyncratic Item Effects: What Is the Good Target for Models?. Journal of Experimental Psychology: Learning, Memory, and Cognition, 2015, 41 (5), pp.1597-1601. ⟨10.1037/xlm0000062⟩. ⟨hal-02438489⟩