Fixed-effect variance and the estimation of repeatabilities and heritabilities: issues and solutions

Pierre de Villemereuil (1) , Michael B. Morrissey (2) , Shinichi Nakagawa (3) , Holger Schielzeth (4)

1 University of Auckland [Auckland]
2 University of St Andrews [Scotland]
3 UNSW - University of New South Wales [Sydney]
4 Friedrich-Schiller-Universität = Friedrich Schiller University Jena [Jena, Germany]

Linear mixed effects models are frequently used for estimating quantitative genetic parameters, including the heritability, as well as the repeatability, of traits. Heritability acts as a filter that determines how efficiently phenotypic selection translates into evolutionary change, while repeatability informs us about the individual consistency of phenotypic traits. As quantities of biological interest, it is important that the denominator, the phenotypic variance in both cases, reflects the amount of phenotypic variance in the relevant ecological setting. The current practice of quantifying heritabilities and repeatabilities from mixed effects models frequently deprives their denominator of variance explained by fixed effects (often leading to upward-bias of heritabilities and repeatabilities) and it has been suggested to omit fixed effects when estimating heritabilities in particular. We advocate an alternative option of fitting models incorporating all relevant effects, while including the variance explained by fixed effects into the estimation of the phenotypic variance. The approach is easily implemented and allows optimising the estimation of phenotypic variance, for example by the exclusion of variance arising from experimental design effects while still including all biologically relevant sources of variation. We address the estimation and interpretation of heritabilities in situations in which potential covariates are themselves heritable traits of the organism. Furthermore, we discuss complications that arise in generalised and non-linear mixed models with fixed effects. In these cases, the variance parameters on the data scale depend on the location of the intercept and hence on the scaling of the fixed effects. Integration over the biologically relevant range of fixed effects offers a preferred solution in those situations.

Mots clés: heritability; linear mixed modelling; fixed effects; quantitative genetics

Domaines: Evolution [q-bio.PE]

Pierre de Villemereuil, Michael B. Morrissey, Shinichi Nakagawa, Holger Schielzeth. Fixed-effect variance and the estimation of repeatabilities and heritabilities: issues and solutions. Journal of Evolutionary Biology, 2018, 31 (4), pp.621-632. ⟨10.1111/jeb.13232⟩. ⟨hal-03043390⟩