Independent Component Analysis and Multi-Way Factor Analysis
Résumé
The problem of identifying linear mixtures of independent random variables only from outputs can be traced back to 1953 with the works of Darmois or Skitovich. They pointed out that when data are non Gaussian, a lot more can be said about the mixture. In practice, Blind Identification of linear mixtures is useful especially in Factor Analysis, in addition to many other application areas (including signal & image processing, digital communications, biomedical, or complexity theory). Harshman and Carroll provided independently numerical algorithms to decompose a data record stored in a 3-way array into elementary arrays, each representing the contribution of a single underlying factor. The main difference with the well known Principal Component Analysis is that the mixture is not imposed to be a unitary matrix. This is very relevant because the actual mixture often has no reason to have orthogonal columns. The Parafac algorithm, widely used since that time, theoretically does not converge for topological reasons, but yields very usable results after a finite number of iterations under mild conditions. Independently, the problem of Blind Source Separation (BSS) arose around 1985 and was solved -explicitly or implicitly- with the help of High-Order Statistics (HOS), which are actually tensor objects. It gave rapidly birth to the more general problem of Independent Component Anlalysis (ICA) in 1991. ICA is a tool that can be used to extract factors when the physical diversity does not allow to store efficiently the data in tensor format, in other words when the Parafac algorithm cannot be used. This tutorial provides a very accessible background on Statistical Independence, High-Order Statistics, and Tensors. Simple examples are given throughout the talk in order to illustrate various concepts and properties. It emphasizes both the usefulness and limitations of Parafac and ICA algorithms. Mathematically advanced topics are only tackled, but striking tensor properties that are not satisfied by matrices are still touched upon. Some reported results show how strange and attractive this research area can be.