The literature says that when three signals are uncorrelated i.e., they have different frequencies, then their COVARIANCE MATRIX Rxx is diagoanl. I wanted to implement it in Matlab like the following but I don't get what I should expect. So where am I not doing correct?

% Case for Un-correlated signals

w=[pi/4 pi/3 pi/2]'; % Un-correlated signals of different frequencies

 N=5;

 xx=2*exp(1j*(w*[1:N])); 

 RxxUn=xx*xx'

% Case for Fully correlated signals of same frequencies

 w=[pi/4 pi/4 pi/4]'; % Correlated signals

 N=5;

 xx=2*exp(1j*(w*[1:N])); 

 RxxCo=xx*xx'

% Case for Partially correlated signals

 w=[pi/4 pi/4 pi/2]'; % Partially Correlated signals

 N=5;

 xx=2*exp(1j*(w*[1:N])); 

 RxxPar=xx*xx'

You should show us your correlation matrices.

I would expect, looking at that code, that none of the correlation matrices are diagonal, and that is because the signals *as you have defined them* are, in fact, correlated.

Sinusoidal signals of different frequencies, like the ones you're working with, are, in general, uncorrelated *as the duration of the signal goes to infinity*.  Slightly more specifically, if the frequencies are related by rational ratios, then two signals of different frequencies will be uncorrelated *over a time period that captures an integer number of cycles of all the signals involved*.

A signal with a radian frequency of pi/2 repeats every four steps; a signal with a radian frequency of pi/3 repeats every 6 steps.  A signal with a radian frequency of pi/4 repeats every 8 steps.

So N needs to be equal to the least common multiple of 4, 6, and 8 -- i.e., 24.  Try that.

(and show us the correlation matrices, please -- it'll be interesting).

Still I don't get what is expected even after changes as below:

clear all

close all

clc

% UnCorrelated

w=[pi/4 pi/3 pi/2]'; % Un-correlated signals

 N=24; 

 xx=2*exp(1j*(w*[1:N])); 

%  xxMean=mean(xx(1,:));

 RxxUn=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

 %RxxUn=xx*xx'/N

  disp('RxxUn should be diagoanl matrix')

 [VxxUn,DxxUn]=eig(RxxUn);

 % Fully correlated

  w=[pi/4 pi/4 pi/4]'; % Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxCo=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

% RxxCo=xx*xx'/N

 Singular=det(RxxCo)

 disp('RxxCo should be Non-diagoanl and singular')

 [VxxCo,DxxCo]=eig(RxxCo);

 % Partially correlated

  w=[pi/4 pi/4 pi/2]'; % Partially Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxPar=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

%  RxxPar=xx*xx'/N

 NonSingular=det(RxxPar)

 disp('RxxPar should be  Non-diagoanl and Non-singular')

[VxxPar,DxxPar]=eig(RxxPar);

For this example: w=[pi/4 pi/3 pi/2]' try N=24 (or some multiple of 24) and check if you get the expected result.

Complex exponentials decorrelate over full cycles. 

I tried it for N=24 also but the last result is still wrong.

I should have clarified. The N=24 suggestion was for your first case only. For the last case, try N=8.

Are you starting to see a pattern here? You need to pick an N such that N*f is a multiple of 2*pi (i.e. full cycle) for all frequencies (f) of interest.

I strongly suggest you try different values of N and f and make some observations on your own. Just posting MATLAB code and claiming it doesn't work won't lead to a very productive discussion (as @Tim Wescott also pointed out).

No I am not interested in any pattern. I just want to verify the literature.

One example is not going to verify the literature. And calling a result "wrong" doesn't make sense if the example you have constructed is itself wrong. 

Anyway, enough said, because this is not a particularly meaningful discussion if you are unwilling or unable to explore the concept you are trying to understand.

Hi,

Since you use 5 samples for the 3 frequencies in complex exponential functions, you should use below for calculating covariance function (Resembling Matlab's covariance fnc definition) 

Rxx=conj(xx - mean(xx,2)) * (xx-mean(xx,2)).' / 4

or you may use 

cov(xx.') (Matlab assumes samples are in each row, you have them in columns). 

Omer 

The sequences still won't be orthogonal.

Omer, I believe you would have to take the absolute values of xx - mean(xx). We are interested in the distance either way from the mean. In my explanation above, the distance is the same from St. Louis to Kansas City as from Kansas City to St. Louis. Covariance does not care about which direction we go.

Hi, 

No, there is no absolute value operation in covariance definition, but transposition and conjugation. 

Even after the following changes didn't give me the correct result:

clear all

close all

clc

% UnCorrelated

w=[pi/4 pi/3 pi/2]'; % Un-correlated signals

 N=24; 

 xx=2*exp(1j*(w*[1:N])); 

%  xxMean=mean(xx(1,:));

 RxxUn=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

 %RxxUn=xx*xx'/N

  disp('RxxUn should be diagoanl matrix')

 [VxxUn,DxxUn]=eig(RxxUn);

 % Fully correlated

  w=[pi/4 pi/4 pi/4]'; % Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxCo=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

% RxxCo=xx*xx'/N

 Singular=det(RxxCo)

 disp('RxxCo should be Non-diagoanl and singular')

 [VxxCo,DxxCo]=eig(RxxCo);

 % Partially correlated

  w=[pi/4 pi/4 pi/2]'; % Partially Correlated signals

 N=24;

 xx=2*exp(1j*(w*[1:N])); 

 RxxPar=( (xx'-mean(xx')))' * (xx'-mean(xx'))/4 % Help from DSPsite

%  RxxPar=xx*xx'/N

 NonSingular=det(RxxPar)

 disp('RxxPar should be  Non-diagoanl and Non-singular')

[VxxPar,DxxPar]=eig(RxxPar);

Could you show your results?  You keep posting code -- for Matlab, which I dislike -- and talking about your results, but you're not taking the trouble to show your results even though you're talking about them.

So show your results, then say what you think is wrong with them.

@ Tim Wescort: My resuls are as below:

RxxUn =

  24.0000 + 0.0000i  -0.0000 - 0.0000i  -0.0000 + 0.0000i

  -0.0000 + 0.0000i  24.0000 + 0.0000i  -0.0000 + 0.0000i

  -0.0000 - 0.0000i  -0.0000 - 0.0000i  24.0000 + 0.0000i

RxxUn should be diagoanl matrix

RxxCo =

    24    24    24

    24    24    24

    24    24    24

Singular =

     0

RxxCo should be Non-diagoanl and singular

RxxPar =

  24.0000 + 0.0000i  24.0000 + 0.0000i  -0.0000 + 0.0000i

  24.0000 + 0.0000i  24.0000 + 0.0000i  -0.0000 + 0.0000i

  -0.0000 - 0.0000i  -0.0000 - 0.0000i  24.0000 + 0.0000i

NonSingular =

     0

RxxPar should be  Non-diagoanl and Non-singular

>> 

Your results are correct.  Your statement that RxxPar should be non-singular is incorrect -- it should be

$$\begin{bmatrix}

24 & 24 & 0 //

24 & 24 & 0 //

0 & 0 & 24

\end{bmatrix}$$

The first two rows are not only linearly dependent, but equal, so the matrix has rank 2, and is, therefor, singular.

Your results seem to be true. In the calculation of RxxPar, you should divide by N-1 (23 here, not by 4). You may compare RxxPar with the results of cov(xx.') 

@ omersayli: Why do you say to divide by N-1 in case of RxxPar?

Think about calculating standard variance, you divide by the number of samples - 1 (for unbiased sample variance). 

Here N is your data sample number in the covariance matrix calculation. 

Correlation matrices show properties in rows and observations in the columns (or vice versa). For example you could have rows with names like "male" "female" "tall" "wealthy". Then you have columns like "bald" "smiling" "pleasant" "well-dressed". You would probably have a higher number (correlation) in the intersection of "male" and "bald" and a lower number in "male" and "smiling" indicating that it might be that females smile a little more than males according to your data.

I have come to think of a covariant matrix as similar to those road maps that have a table of distances between cities. You have all the cities listed in rows; and the same cities are listed in columns. So where St. Louis and Kansas City intersect, that value is the number of miles between the cities. But since a city is zero miles from itself, the table has blanks in a linear diagonal from top left to bottom right. But we have to remember that covariance is the distance from the mean (positive or negative). So we have to take the absolute value of the distance to determine the relevance.
So your correlation matrix will help you determine how much this property is related to that observation. For example, how does this frequency correlate with that kind of distortion.

Jerry Cogswell
Kalispell, MT

The problem is you have not defined the interval over which you expect the sample values to be collected to form the correlation estimates.  Repeat your experiment with N=50 and then with N=500, and finally with 5000.

 Number of measurements is an important consideration... that's why people ask about size of data set 

fred h