Examining the range of variations in self-presentation bias using Monte-Carlo simulation
Résumé
The extent to which response distortion – such as social desirability responding (SDR) – is present in self-report personality inventories is an issue of concern and debate, as it may impede such measures’ psychometric indices, including predictive validity. In spite of general agreement on the possibility of this SDR bias, authors highlight inconsistencies in its reported effect sizes (spreading from small to large). Using Monte Carlo simulations, research has provided some evidence that SDR bias does only have a small effect on predictive validity even when - what the authors qualify as - large amounts of SDR bias are added (i.e., one standard deviation unit added to the true score; Paunonen & LeBel, 2012).
We argue that the criteria underlying the simulation procedure of this research may be, in part, psychometrically and psychologically unwarranted. Specifically, although SDR bias is conceived of as an individual differences variable, the authors systematically add an amount of bias which is only and linearly related to the true score on a 1 to 9-point response scale. While random error is added to the original true score, no variability reflecting individual differences is considered.
Using the same Monte Carlo procedure, relaxing some of the initial assumptions, such as linearity of the true score-bias relationship or normal distribution of errors, we aim at relying on a more psychologically and psychometrically valid procedure. Specifically, we evaluate how a bias sampled from a half normal distribution, which reflects positivity bias towards positive inflation of scores, influences the effect on predictive validity. Also, through nonlinear relationship, we consider the possibility for the bias to remain large for high values of the true score, thus leading to a saturation of the scale. Indeed, at the measurement and latent levels, the whole continuum of scale values ought to be used; i.e., on the 9-point scale, responses might range from 1 to 9 regardless of the true score. Extreme cases – although rare – are made possible when sampling the bias from a half-normal distribution.
Finally, we evaluate the relevance of the various models on real participants’ data, by testing the fit between the simulated distributions and the human distributions for each of the Big Five dimensions. Discussion focuses on the implication for self-presentation and motivated social cognition.