Independent Component Analysis: Principles and Practice

Citation

Andrew, A.M. (2002), "Independent Component Analysis: Principles and Practice", Kybernetes, Vol. 31 No. 2. https://doi.org/10.1108/k.2002.06731bae.004

Independent Component Analysis: Principles and Practice

Stephen Roberts and Richard Everson (Editors)Cambridge University Press2001xii + 338 pp.ISBN 0-521-79298-3hardback, £45.00

This deals with an important new development in statistical mathematics, related to the much older topic of Principal Component Analysis. Each of them operates on a set of data inputs, and PCA provides a means of attributing these to components that are uncorrelated, where correlation is defined as usual in terms of second-order statistical characteristics of the data. As the editors of this work put it in their introductory chapter: "ICA aims at a loftier goal: it seeks a transformation to coordinates in which the data are maximally statistically independent, not merely decorrelated".

The theory and applications of ICA are developed in twelve chapters, from different authors or groups of authors. The contributions have been specially commissioned with an attempt at uniform nomenclature throughout, and with a common set of references that is surprisingly lengthy for a relatively new topic. The longest of the chapters, by a substantial margin, is the first in which the two editors give a valuable Introduction, and there is a useful Preface giving an overview of the coverage of each chapter.

In the first chapter the general idea is introduced by reference to the "cocktail party problem" which is that of understanding how a person can follow a conversation under cocktail-party conditions. Where inputs from a number of sources are mixed, to give several mixture signals containing the original components in different proportions, the method is remarkably effective in recovering the original signals from the mixtures. Results of a test are shown, where there were three original components, two of them music of very different kinds and the third white noise.

The introductory example is rather simpler than the full cocktail party problem since the components are mixed without the time differences that would be introduced by the spatial separation of ears or microphones. The full problem with time separation also receives attention. The methods utilise the fact that the probability density functions of the components may be other than Gaussian. In this they contrast with standard statistical methods in which Gaussian distributions are assumed (and methods are then rated for their robustness against violation of assumptions, particularly of Gaussianity).

Although the point is not stressed in the book, it is worth noting that a portable cocktail-party analyser could be an enormous boon-to many elderly people who have difficulty in heating speech in such situations. The ear- trumpet was an early but inelegant solution.

Independence of components is defined in terms of mutual information between them, but with a specially rigorous definition of this taking account of probability density functions. Remarkably, efficient algorithms have been developed that operate with modest sample sizes. In the later chapters variations are described that operate in non-stationary situations.

Apart from its use in isolating sources in cocktail parties and other acoustically noisy environments, the technique has various important areas of application. A fairly obvious extension of the application to cocktail parties is to the elimination of crosstalk in communication networks. Another area of application is in biomedicine, with electroencephalography particularly mentioned. In electroencephalography, signals are picked up from a number of electrodes placed over the head and the technique allows isolation of distinct sources of electrical variations contributing to these. Applications have also been made to financial analysis, to identify sources of variation in investment returns.

An application with a less parametric character is to the classification of groups of text documents with independent themes. There are also various applications to scene analysis, including a controversial attempt to analyse human faces as aggregates of a number of basic types. In connection with scene analysis it is pleasant to find references to an early pioneer of cybernetics.

Horace Barlow, who has long argued (Barlow 1959) that a function of the nervous system is to reduce redundancy, and hence mutual information between channels. The method also allows the removal of noise from signals and images in ways that seem to contravene basic ideas of information theory since they operate without a correction channel. The auxiliary information allowing this refers only to an assumed nature of the original signal orimage. An example is shown where an image containing printed text and a well-defined decorative pattern is first corrupted by random substitution of pixels and then substantially restored by an application of ICA. The book is intended to be a self-contained introduction and overview of this important development and it appears to meet the requirement admirably.

Alex M. Andrew

ReferenceBarlow, H.B. (1959) "Sensory mechanisms, the reduction of redundancy and intelligence", in: Mechanisation of Thought Processes, HMSO, London, pp. 535-74.