tailieunhanh - Bài 9: ICA by Maximum Likelihood Estimation

A very popular approach for estimating the independent component analysis (ICA) model is maximum likelihood (ML) estimation. Maximum likelihood estimation is a fundamental method of statistical estimation; a short introduction was provided in Section . One interpretation of ML estimation is that we take those parameter values as estimates that give the highest probability for the observations. In this section, we show how to apply ML estimation to ICA estimation. | Independent Component Analysis. Aapo Hyvarinen Juha Karhunen Erkki Oja Copyright 2001 John Wiley Sons Inc. ISBNs 0-471-40540-X Hardback 0-471-22131-7 Electronic 9 ICA by Maximum Likelihood Estimation A very popular approach for estimating the independent component analysis ICA model is maximum likelihood ML estimation. Maximum likelihood estimation is a fundamental method of statistical estimation a short introduction was provided in Section . One interpretation of ML estimation is that we take those parameter values as estimates that give the highest probability for the observations. In this section we show how to apply ML estimation to ICA estimation. We also show its close connection to the neural network principle of maximization of information flow infomax . THE LIKELIHOOD OF THE ICA MODEL Deriving the likelihood It is not difficult to derive the likelihood in the noise-free ICA model. This is based on using the well-known result on the density of a linear transform given in . According to this result the density of the mixture vector x As can be formulated as p4x detB pg s detB i 203 204 ICA BYMAXIMUM LIKELIHOOD ESTIMATION where B A 1 and the p denote the densities of the independent components. This can be expressed as a function of B b . b r and x giving IV x detB JJpj bfx i Assume that we have T observations obi denoted by x l x 2 . x T . Then the likelihood can be obtained see Section as the product of this density evaluated at the T points. This is denoted by L and considered as a function of B T n B niLil x rB t i i i Very often it is more practical to use the logarithm of the likelihood since it is algebraically simpler. This does not make any difference here since the maximum of the logarithm is obtained at the same point as the maximum of the likelihood. The log-likelihood is given by T n logL B EE logpj bfx i Tlog detB t i i i The basis of the logarithm makes no difference though in the .