Comparison between Principal Component Analysis and Factor Analysis: an Informational Perspective.
Résumé
Principal Components Analysis (PCA) and Factor Analysis (FA) have been the two most widely used methods for dimension reduction over the years. Though the theoretical bases for both methods are clearly established and rather different (to cut short; PCA is based on an algebraical criterion: the retained variance, while FA is established through a statistical model of latent variables and leads to retain the maximal covariance), there has been a large attempt, specially among users of such dimension reduction techniques, to provide some comparisons of these methods from a very applied point of view. Behavioural and Psychological research is probably one of the most interested field by these comparisons, but statistical softwares are also concerned. Even recently in advanced sciences the subject is a still living issue. The link between PCA and FA in terms of model and sub-model has already been established. The restriction concerning PCA is therefore obvious but some work is need to get all the consequences of this result. We propose to focus on a new trend in multivariate analysis: dimensionality or more precisely redundancy, because this result provides a way to compare both methods from an informational point of view. The presentation will first point out some restrictions of classical PCA with respect to redundancy between variables. Some guidelines to an informational approach of multivariate analysis will then be proposed and connected in some way with the FA paradigm. The result is then used to highlight the informational approach of multivariate analysis and the comparison between both methods.