Hi everyone,
I just started doing a project based on ICA. After reading the book "Independent
component analysis" by Hyvarinen A ect., I have some questions which confused me
a lot.
1. negentroy is defined as J(X)=H(Xgauss)-H(X). It has the advantage that it is
scale invariant. J(X) is used as a criterion of find the mimimum of H(X) so that
the signal is as non-gaussian as possible.
2. whitening is applied before ICA algorithm (non-gaussian algorithm) so that
the covariance matrix is always Identity matrix. This means that in J(X),
H(Xgauss) and H(X) have covariance matrix equals to I.
3. It is also mentioned in this book (page 113), H(Y)=H(X)+log|det(A)| if A is
invertible linear transform Y=AX. This means any orthogonal matrix A does not
change the entropy of X. In other words, the entropy of Y is the same as that of
X.
In ICA algorithm, it tries to find an orthogonal matrix which maximize J(X).
Since entropy H(X) does not change at all (according to 3) by applying any
orthogonal transform, how can it be used as a criterion?
Thanks for helping.
