Forums

Covariance Matrix diagonal or not?

Started by matlab2015 4 months ago20 replieslatest reply 4 months ago109 views
The literature says that when three signals are uncorrelated i.e., they have different frequencies, then their COVARIANCE MATRIX Rxx is diagoanl. I wanted to implement it in Matlab like the following but I don't get what I should expect. So where am I not doing correct?


% Case for Un-correlated signals

w=[pi/4 pi/3 pi/2]'; % Un-correlated signals of different frequencies

 N=5;

 xx=2*exp(1j*(w*[1:N])); 

 RxxUn=xx*xx'


% Case for Fully correlated signals of same frequencies

 w=[pi/4 pi/4 pi/4]'; % Correlated signals

 N=5;

 xx=2*exp(1j*(w*[1:N])); 

 RxxCo=xx*xx'


% Case for Partially correlated signals

 w=[pi/4 pi/4 pi/2]'; % Partially Correlated signals

 N=5;

 xx=2*exp(1j*(w*[1:N])); 

 RxxPar=xx*xx'


[ - ]
Reply by Tim WescottMay 25, 2021

You should show us your correlation matrices.

I would expect, looking at that code, that none of the correlation matrices are diagonal, and that is because the signals *as you have defined them* are, in fact, correlated.

Sinusoidal signals of different frequencies, like the ones you're working with, are, in general, uncorrelated *as the duration of the signal goes to infinity*.  Slightly more specifically, if the frequencies are related by rational ratios, then two signals of different frequencies will be uncorrelated *over a time period that captures an integer number of cycles of all the signals involved*.

A signal with a radian frequency of pi/2 repeats every four steps; a signal with a radian frequency of pi/3 repeats every 6 steps.  A signal with a radian frequency of pi/4 repeats every 8 steps.

So N needs to be equal to the least common multiple of 4, 6, and 8 -- i.e., 24.  Try that.

(and show us the correlation matrices, please -- it'll be interesting).

[ - ]
Reply by matlab2015May 26, 2021

Still I don't get what is expected even after changes as below:


clear all

close all

clc

% UnCorrelated

w=[pi/4 pi/3 pi/2]'; % Un-correlated signals

 N=24; 

 xx=2*exp(1j*(w*[1:N])); 

%  xxMean=mean(xx(1,:));

 RxxUn=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

 %RxxUn=xx*xx'/N

  disp('RxxUn should be diagoanl matrix')

 [VxxUn,DxxUn]=eig(RxxUn);

 % Fully correlated

  w=[pi/4 pi/4 pi/4]'; % Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxCo=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

% RxxCo=xx*xx'/N

 Singular=det(RxxCo)

 disp('RxxCo should be Non-diagoanl and singular')

 [VxxCo,DxxCo]=eig(RxxCo);

 % Partially correlated

  w=[pi/4 pi/4 pi/2]'; % Partially Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxPar=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

%  RxxPar=xx*xx'/N

 NonSingular=det(RxxPar)

 disp('RxxPar should be  Non-diagoanl and Non-singular')

[VxxPar,DxxPar]=eig(RxxPar);

[ - ]
Reply by adiduaMay 25, 2021

For this example: w=[pi/4 pi/3 pi/2]' try N=24 (or some multiple of 24) and check if you get the expected result.

Complex exponentials decorrelate over full cycles. 

[ - ]
Reply by matlab2015May 26, 2021

I tried it for N=24 also but the last result is still wrong.

[ - ]
Reply by adiduaMay 26, 2021

I should have clarified. The N=24 suggestion was for your first case only. For the last case, try N=8.

Are you starting to see a pattern here? You need to pick an N such that N*f is a multiple of 2*pi (i.e. full cycle) for all frequencies (f) of interest.

I strongly suggest you try different values of N and f and make some observations on your own. Just posting MATLAB code and claiming it doesn't work won't lead to a very productive discussion (as @Tim Wescott also pointed out).

[ - ]
Reply by matlab2015May 26, 2021

No I am not interested in any pattern. I just want to verify the literature.

[ - ]
Reply by adiduaMay 27, 2021

One example is not going to verify the literature. And calling a result "wrong" doesn't make sense if the example you have constructed is itself wrong. 


Anyway, enough said, because this is not a particularly meaningful discussion if you are unwilling or unable to explore the concept you are trying to understand.

[ - ]
Reply by omersayliMay 25, 2021

Hi,


Since you use 5 samples for the 3 frequencies in complex exponential functions, you should use below for calculating covariance function (Resembling Matlab's covariance fnc definition) 

Rxx=conj(xx - mean(xx,2)) * (xx-mean(xx,2)).' / 4


or you may use 


cov(xx.') (Matlab assumes samples are in each row, you have them in columns). 



Omer 


[ - ]
Reply by Tim WescottMay 26, 2021

The sequences still won't be orthogonal.

[ - ]
Reply by TreefarmerMay 25, 2021

Omer, I believe you would have to take the absolute values of xx - mean(xx). We are interested in the distance either way from the mean. In my explanation above, the distance is the same from St. Louis to Kansas City as from Kansas City to St. Louis. Covariance does not care about which direction we go.

[ - ]
Reply by omersayliMay 26, 2021

Hi, 

No, there is no absolute value operation in covariance definition, but transposition and conjugation. 

[ - ]
Reply by matlab2015May 26, 2021

Even after the following changes didn't give me the correct result:


clear all

close all

clc

% UnCorrelated

w=[pi/4 pi/3 pi/2]'; % Un-correlated signals

 N=24; 

 xx=2*exp(1j*(w*[1:N])); 

%  xxMean=mean(xx(1,:));

 RxxUn=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

 %RxxUn=xx*xx'/N

  disp('RxxUn should be diagoanl matrix')

 [VxxUn,DxxUn]=eig(RxxUn);

 % Fully correlated

  w=[pi/4 pi/4 pi/4]'; % Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxCo=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

% RxxCo=xx*xx'/N

 Singular=det(RxxCo)

 disp('RxxCo should be Non-diagoanl and singular')

 [VxxCo,DxxCo]=eig(RxxCo);

 % Partially correlated

  w=[pi/4 pi/4 pi/2]'; % Partially Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxPar=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

%  RxxPar=xx*xx'/N

 NonSingular=det(RxxPar)

 disp('RxxPar should be  Non-diagoanl and Non-singular')

[VxxPar,DxxPar]=eig(RxxPar);

[ - ]
Reply by Tim WescottMay 26, 2021

Could you show your results?  You keep posting code -- for Matlab, which I dislike -- and talking about your results, but you're not taking the trouble to show your results even though you're talking about them.

So show your results, then say what you think is wrong with them.

[ - ]
Reply by matlab2015May 26, 2021

@ Tim Wescort: My resuls are as below:


RxxUn =

  24.0000 + 0.0000i  -0.0000 - 0.0000i  -0.0000 + 0.0000i

  -0.0000 + 0.0000i  24.0000 + 0.0000i  -0.0000 + 0.0000i

  -0.0000 - 0.0000i  -0.0000 - 0.0000i  24.0000 + 0.0000i

RxxUn should be diagoanl matrix

RxxCo =

    24    24    24

    24    24    24

    24    24    24

Singular =

     0

RxxCo should be Non-diagoanl and singular

RxxPar =

  24.0000 + 0.0000i  24.0000 + 0.0000i  -0.0000 + 0.0000i

  24.0000 + 0.0000i  24.0000 + 0.0000i  -0.0000 + 0.0000i

  -0.0000 - 0.0000i  -0.0000 - 0.0000i  24.0000 + 0.0000i

NonSingular =

     0

RxxPar should be  Non-diagoanl and Non-singular

>> 

[ - ]
Reply by Tim WescottMay 27, 2021

Your results are correct.  Your statement that RxxPar should be non-singular is incorrect -- it should be

$$\begin{bmatrix}

24 & 24 & 0 //

24 & 24 & 0 //

0 & 0 & 24

\end{bmatrix}$$


The first two rows are not only linearly dependent, but equal, so the matrix has rank 2, and is, therefor, singular.

[ - ]
Reply by omersayliMay 26, 2021

Your results seem to be true. In the calculation of RxxPar, you should divide by N-1 (23 here, not by 4). You may compare RxxPar with the results of cov(xx.') 

[ - ]
Reply by matlab2015May 26, 2021

@ omersayli: Why do you say to divide by N-1 in case of RxxPar?

[ - ]
Reply by omersayliMay 28, 2021

Think about calculating standard variance, you divide by the number of samples - 1 (for unbiased sample variance). 

Here N is your data sample number in the covariance matrix calculation. 

[ - ]
Reply by TreefarmerMay 25, 2021

Correlation matrices show properties in rows and observations in the columns (or vice versa). For example you could have rows with names like "male" "female" "tall" "wealthy". Then you have columns like "bald" "smiling" "pleasant" "well-dressed". You would probably have a higher number (correlation) in the intersection of "male" and "bald" and a lower number in "male" and "smiling" indicating that it might be that females smile a little more than males according to your data.

I have come to think of a covariant matrix as similar to those road maps that have a table of distances between cities. You have all the cities listed in rows; and the same cities are listed in columns. So where St. Louis and Kansas City intersect, that value is the number of miles between the cities. But since a city is zero miles from itself, the table has blanks in a linear diagonal from top left to bottom right. But we have to remember that covariance is the distance from the mean (positive or negative). So we have to take the absolute value of the distance to determine the relevance.
So your correlation matrix will help you determine how much this property is related to that observation. For example, how does this frequency correlate with that kind of distortion.

Jerry Cogswell
Kalispell, MT

[ - ]
Reply by fharrisMay 25, 2021

The problem is you have not defined the interval over which you expect the sample values to be collected to form the correlation estimates.  Repeat your experiment with N=50 and then with N=500, and finally with 5000.

 Number of measurements is an important consideration... that's why people ask about size of data set 


fred h