Developing and using statistical tools to estimate observer effect for ordered class data: The case of the IBP (Index of Biodiversity Potential)
Résumé
Ecological indicators based on measurements made by individual observers often include extra noise associated
with the observer herself (observer effect). The literature on this subject has used a variety of analytical tools to
account for this variation, but few analyses have used a hierarchical statistical model approach that can account
for the variation and bias among observers on both the mean and variance of the data. We used the Index of
Biodiversity Potential (IBP), a rapid habitat assessment method widely used in France, in a case study to propose
latent variable hierarchical models to assess the impact of observer effect while accounting for other sources of
variation. For seven of the ten factors constituting the IBP, we used a sequential model selection procedure to
analyze observers’ scores, then analyzed the final models. The structure of the best models varied according to
the related factor. Our analyses confirm: (i) that the expert reference observer provided lower variability in his
observations than did the other observers for all factors except the one related to standing deadwood; (ii) that all
but two factors showed at least a moderate level of systematic random observer variations; (iii) that the absence
of leaves in winter led to increased variations when tree-species identification was required; and (iv) that
evaluation of stand openness was highly variable and should therefore be assessed by an expert. Overall, the data
were coherent with our best models from various points of view for which we diagnosed goodness-of-fit. Our
work is a further illustration of the interest of adopting fully parametric statistical modelling of observer-based
variations.