Forums

Correlation Function/Decorrelation Time Estimate Discrepancies

Started by Greg Heath July 31, 2004
When I was analyzing some time-series data, I obtained the autocorrelation
function and corresponding estimate of the 1/e (=0.3679) decorrelation time
by ifft-ing the power spectral density. As a check, I used the MATLAB
 function xcorr. As can be seen below and in the plot for the test function
z = (x-mean(x))/std(x), x = t.*exp(-t), The functions and decorrelation
time estimates were significantly different .

                         Autocorrelation Value
Lag(sec)          Temporal         Spectral         Difference
---------------------------------------------------------
  1.2                  0.5694             0.4383
*1.3                  0.5273           *0.3790          0.1483
  1.4                  0.4862             0.3212
  1.5                  0.4461             0.2650
  1.6                  0.4072             0.2106
*1.7                *0.3695             0.1584          0.2111
  1.8                  0.3331             0.1082


On the other hand, if the statement, z = x;, is uncommented, the spectral
and temporal functions and decorrelation time estimates are almost
identical.

Can anyone explain this?

TIA,

Greg
************************************************************
clear, close all, clc
display('***** Standardized Time Function **********')
dt =  0.1, N =  100, T =  N*dt, t =  dt*(0:N-1)';
x  =  t.*exp(-t); meanx =  mean(x), stdx =  std(x)
z  =  (x-meanx)/stdx;
%z =  x;
display('* Autocorrelation Function (Temporal Estimate) *')
Az  =  xcorr(z); M =  length(Az), em1 =  exp(-1)
tAz =  [-t(end:-1:2); t];
[Azmax iAzmax] =  max(Az),  Az =  Az/Azmax;
Azcheck =  [M - (2*N-1);Azmax - sumsqr(z);iAzmax - N]
display('* 1/e Decorrelation Time (Temporal Estimate) *')
i         =  1:M-1; i1 =  find( Az(i) >= em1 & Az(i+1) < em1)
t1       =  tAz(i1), t2 = t1+dt, A1 = Az(i1),  A2=Az(i1+1)
tauAz =  t1 + [(A1-em1)/(A1-A2)]*dt
I         =  (i1-5:i1+2)';
tauAzcheck = [ I tAz(I) Az(I) ]
display('********** Power Spectral Density **********')
Z =  fft(z); PSDz =  abs(Z/N).^2; N2 = N^2
PSDzcheck =  [ PSDz(1) - 0 ; sum(PSDz) - (N-1)/N ]
display('* Autocorrrelation Function (Spectral Estimate) *')
Azf    =  ifftshift(real(ifft(PSDz))); Q =  fix((N+2)/2)
tAzf   =  [-t(Q:-1:2); t(1:Q-1)];
[Azfmax iAzfmax] =  max(Azf),  Azf =  Azf/Azfmax;
Azfcheck    = [length(Azf) - N; Azfmax - sumsqr(z)/N2;...
               iAzfmax - Q]
display('* 1/e Decorrelation Time (Spectral Estimate) *')
j      =  1:N-1;j1 = find( Azf(j) >= em1 & Azf(j+1) < em1)
t1     =  tAzf(j1),t2 = t1 + dt,A1 = Azf(j1),A2 = Azf(j1+1)
tauAzf =  t1 + [(A1-em1)/(A1-A2)]*dt
J      =  (j1-1:j1+5)';
tauAzfcheck = [ J tAzf(J) Azf(J) ]
display('******* Autocorrelation Comparison Plots ********')
figure, plot(tAz, Az), hold on, plot(tAzf, Azf,'r')
hold on, plot(tauAz, em1,'b*'), hold on
plot(tauAzf, em1,'r*'), hold on
plot([-T T],[em1 em1],'r--'), hold on
plot([-T T],[0 0],'k')
legend('Temporal Estimate','Spectral Estimate')
xlabel('Time (sec)')
title('Comparison of Autocorrelation Estimates')


"Greg Heath" <heath@alumni.brown.edu> wrote in message
news:2n0hjlFr99v0U1@uni-berlin.de...
> When I was analyzing some time-series data, I obtained the >autocorrelation function and corresponding estimate of the >1/e (=0.3679) decorrelation time by ifft-ing the power >spectral density. As a check, I used the MATLAB function >xcorr. As can be seen below and in the plot for the test >function z = (x-mean(x))/std(x), x = t.*exp(-t), The >functions and decorrelation time estimates were >significantly different .
-----SNIP
> On the other hand, if the statement, z = x;, is > uncommented, the spectral and temporal functions > and decorrelation time estimates are almost identical. > > Can anyone explain this?
Hey guys, it's not fair for me to have to answer my own questions! The basic reason is that xcorr is obtained by zero-padding. If the temporal estimate is given by Azt = xcorr(z);, % then M = length(Azt) % = 2*N-1, where N = length(z) The corresponding spectral estimate can be obtained from Z1 = fft(z,M); PSD1 = abs(Z1).^2; Azf1 = fftshift(real(ifft(PSD1))); . Notice that fftshift ( and not ifftshift) is used to get a completely symmetric function about the middle index N. In general, these results will be different from those obtained from the nonzero-padded estimate Z2 = fft(z); PSD2 = abs(Z2).^2; Azf2 = fftshift(real(ifft(PSD1))); . unless zero-padding doesn't change the shape of the function. For example, the plot of x and [x;zeros(M,1)] are sufficiently similar whereas the plots of z and [z;zeros(M,1)] are significantly different. I guess the lesson here is that you must be aware of the zero-padding whenever you use xcorr! Hope this helps. Greg -----SNIP
"Greg Heath" <heath@alumni.brown.edu> wrote in message
news:2n9vmiFupk20U1@uni-berlin.de...
> "Greg Heath" <heath@alumni.brown.edu> wrote in message > news:2n0hjlFr99v0U1@uni-berlin.de... > > When I was analyzing some time-series data, I obtained the > >autocorrelation function and corresponding estimate of the > >1/e (=0.3679) decorrelation time by ifft-ing the power > >spectral density. As a check, I used the MATLAB function > >xcorr. As can be seen below and in the plot for the test > >function z = (x-mean(x))/std(x), x = t.*exp(-t), The > >functions and decorrelation time estimates were > >significantly different . > -----SNIP > > On the other hand, if the statement, z = x;, is > > uncommented, the spectral and temporal functions > > and decorrelation time estimates are almost identical. > > > > Can anyone explain this? > > Hey guys, it's not fair for me to have to answer my own > questions! > > The basic reason is that xcorr is obtained by zero-padding. > > If the temporal estimate is given by > > Azt = xcorr(z);, % then > M = length(Azt) % = 2*N-1, where > N = length(z) > > The corresponding spectral estimate can be obtained from > > Z1 = fft(z,M); > PSD1 = abs(Z1).^2; > Azf1 = fftshift(real(ifft(PSD1))); . > > Notice that fftshift ( and not ifftshift) is used to get a > completely symmetric function about the middle index > N. > > In general, these results will be different from those > obtained from the nonzero-padded estimate > > Z2 = fft(z); > PSD2 = abs(Z2).^2; > Azf2 = fftshift(real(ifft(PSD1))); , > > unless zero-padding doesn't change the shape of the > function. > > For example, the plot of x and [x;zeros(M,1)] are > sufficiently similar whereas the plots of z and > [z;zeros(M,1)] are significantly different. > > I guess the lesson here is that you must be aware of > the zero-padding whenever you use xcorr!
Actually, there is one more point to make. First note the decorrelation time estimate in the following table for N = 100: length x z -------- -------- ------- N 2.16 1.32 M 2.14 1.70 Now, notice what happens when the hanning window is applied: length x z -------- -------- ------- N 2.62 1.82 M 2.62 1.83 Finally, the consistency I was looking for! However,... since MATLAB contains 16 different window functions, a precise definition of decorrelation time for a general sampled data time series may not exist. Hope this helps, Greg