tailieunhanh - Independent component analysis P16

ICA with Overcomplete Bases A difficult problem in independent component analysis (ICA) is encountered if the number of mixtures xi is smaller than the number of independent components si. This means that the mixing system is not invertible: We cannot obtain the independent components (ICs) by simply inverting the mixing matrix . Therefore, even if we knew the mixing matrix exactly, we could not recover the exact values of the independent components. This is because information is lost in the mixing process. This situation is often called ICA with overcomplete bases. This is because we have in the ICA model A x =. | Independent Component Analysis. Aapo Hyvarinen Juha Karhunen Erkki Oja Copyright 2001 John Wiley Sons Inc. ISBNs 0-471-40540-X Hardback 0-471-22131-7 Electronic 16 ICA with Overcomplete Bases A difficult problem in independent component analysis ICA is encountered if the number of mixtures a is smaller than the number of independent components s . This means that the mixing system is not invertible We cannot obtain the independent components ICs by simply inverting the mixing matrix A. Therefore even if we knew the mixing matrix exactly we could not recover the exact values of the independent components. This is because information is lost in the mixing process. This situation is often called ICA with overcomplete bases. This is because we have in the ICA model x As i where the number of basis vectors a is larger than the dimension of the space of x thus this basis is too large or overcomplete. Such a situation sometimes occurs in feature extraction of images for example. As with noisy ICA we actually have two different problems. First how to estimate the mixing matrix and second how to estimate the realizations of the independent components. This is in stark contrast to ordinary ICA where these two problems are solved at the same time. This problem is similar to the noisy ICA in another respect as well It is much more difficult than the basic ICA problem and the estimation methods are less developed. 305 306 ICA WITH OVERCOMPLETE BASES ESTIMATION OF THE INDEPENDENT COMPONENTS Maximum likelihood estimation Many methods for estimating the mixing matrix use as subroutines methods that estimate the independent components for a known mixing matrix. Therefore we shall first treat methods for reconstructing the independent components assuming that we know the mixing matrix. Let us denote bym the number of mixtures and by n the number of independent components. Thus the mixing matrix has size m x n with n m and therefore it is not invertible. The simplest