When I was analyzing some time-series data, I obtained the autocorrelation function and corresponding estimate of the 1/e (=0.3679) decorrelation time by ifft-ing the power spectral density. As a check, I used the MATLAB function xcorr. As can be seen below and in the plot for the test function z = (x-mean(x))/std(x), x = t.*exp(-t), The functions and decorrelation time estimates were significantly different . Autocorrelation Value Lag(sec) Temporal Spectral Difference --------------------------------------------------------- 1.2 0.5694 0.4383 *1.3 0.5273 *0.3790 0.1483 1.4 0.4862 0.3212 1.5 0.4461 0.2650 1.6 0.4072 0.2106 *1.7 *0.3695 0.1584 0.2111 1.8 0.3331 0.1082 On the other hand, if the statement, z = x;, is uncommented, the spectral and temporal functions and decorrelation time estimates are almost identical. Can anyone explain this? TIA, Greg ************************************************************ clear, close all, clc display('***** Standardized Time Function **********') dt = 0.1, N = 100, T = N*dt, t = dt*(0:N-1)'; x = t.*exp(-t); meanx = mean(x), stdx = std(x) z = (x-meanx)/stdx; %z = x; display('* Autocorrelation Function (Temporal Estimate) *') Az = xcorr(z); M = length(Az), em1 = exp(-1) tAz = [-t(end:-1:2); t]; [Azmax iAzmax] = max(Az), Az = Az/Azmax; Azcheck = [M - (2*N-1);Azmax - sumsqr(z);iAzmax - N] display('* 1/e Decorrelation Time (Temporal Estimate) *') i = 1:M-1; i1 = find( Az(i) >= em1 & Az(i+1) < em1) t1 = tAz(i1), t2 = t1+dt, A1 = Az(i1), A2=Az(i1+1) tauAz = t1 + [(A1-em1)/(A1-A2)]*dt I = (i1-5:i1+2)'; tauAzcheck = [ I tAz(I) Az(I) ] display('********** Power Spectral Density **********') Z = fft(z); PSDz = abs(Z/N).^2; N2 = N^2 PSDzcheck = [ PSDz(1) - 0 ; sum(PSDz) - (N-1)/N ] display('* Autocorrrelation Function (Spectral Estimate) *') Azf = ifftshift(real(ifft(PSDz))); Q = fix((N+2)/2) tAzf = [-t(Q:-1:2); t(1:Q-1)]; [Azfmax iAzfmax] = max(Azf), Azf = Azf/Azfmax; Azfcheck = [length(Azf) - N; Azfmax - sumsqr(z)/N2;... iAzfmax - Q] display('* 1/e Decorrelation Time (Spectral Estimate) *') j = 1:N-1;j1 = find( Azf(j) >= em1 & Azf(j+1) < em1) t1 = tAzf(j1),t2 = t1 + dt,A1 = Azf(j1),A2 = Azf(j1+1) tauAzf = t1 + [(A1-em1)/(A1-A2)]*dt J = (j1-1:j1+5)'; tauAzfcheck = [ J tAzf(J) Azf(J) ] display('******* Autocorrelation Comparison Plots ********') figure, plot(tAz, Az), hold on, plot(tAzf, Azf,'r') hold on, plot(tauAz, em1,'b*'), hold on plot(tauAzf, em1,'r*'), hold on plot([-T T],[em1 em1],'r--'), hold on plot([-T T],[0 0],'k') legend('Temporal Estimate','Spectral Estimate') xlabel('Time (sec)') title('Comparison of Autocorrelation Estimates')

# Correlation Function/Decorrelation Time Estimate Discrepancies

Started by ●July 31, 2004

Reply by ●August 3, 20042004-08-03

"Greg Heath" <heath@alumni.brown.edu> wrote in message news:2n0hjlFr99v0U1@uni-berlin.de...> When I was analyzing some time-series data, I obtained the >autocorrelation function and corresponding estimate of the >1/e (=0.3679) decorrelation time by ifft-ing the power >spectral density. As a check, I used the MATLAB function >xcorr. As can be seen below and in the plot for the test >function z = (x-mean(x))/std(x), x = t.*exp(-t), The >functions and decorrelation time estimates were >significantly different .-----SNIP> On the other hand, if the statement, z = x;, is > uncommented, the spectral and temporal functions > and decorrelation time estimates are almost identical. > > Can anyone explain this?Hey guys, it's not fair for me to have to answer my own questions! The basic reason is that xcorr is obtained by zero-padding. If the temporal estimate is given by Azt = xcorr(z);, % then M = length(Azt) % = 2*N-1, where N = length(z) The corresponding spectral estimate can be obtained from Z1 = fft(z,M); PSD1 = abs(Z1).^2; Azf1 = fftshift(real(ifft(PSD1))); . Notice that fftshift ( and not ifftshift) is used to get a completely symmetric function about the middle index N. In general, these results will be different from those obtained from the nonzero-padded estimate Z2 = fft(z); PSD2 = abs(Z2).^2; Azf2 = fftshift(real(ifft(PSD1))); . unless zero-padding doesn't change the shape of the function. For example, the plot of x and [x;zeros(M,1)] are sufficiently similar whereas the plots of z and [z;zeros(M,1)] are significantly different. I guess the lesson here is that you must be aware of the zero-padding whenever you use xcorr! Hope this helps. Greg -----SNIP

Reply by ●August 5, 20042004-08-05

"Greg Heath" <heath@alumni.brown.edu> wrote in message news:2n9vmiFupk20U1@uni-berlin.de...> "Greg Heath" <heath@alumni.brown.edu> wrote in message > news:2n0hjlFr99v0U1@uni-berlin.de... > > When I was analyzing some time-series data, I obtained the > >autocorrelation function and corresponding estimate of the > >1/e (=0.3679) decorrelation time by ifft-ing the power > >spectral density. As a check, I used the MATLAB function > >xcorr. As can be seen below and in the plot for the test > >function z = (x-mean(x))/std(x), x = t.*exp(-t), The > >functions and decorrelation time estimates were > >significantly different . > -----SNIP > > On the other hand, if the statement, z = x;, is > > uncommented, the spectral and temporal functions > > and decorrelation time estimates are almost identical. > > > > Can anyone explain this? > > Hey guys, it's not fair for me to have to answer my own > questions! > > The basic reason is that xcorr is obtained by zero-padding. > > If the temporal estimate is given by > > Azt = xcorr(z);, % then > M = length(Azt) % = 2*N-1, where > N = length(z) > > The corresponding spectral estimate can be obtained from > > Z1 = fft(z,M); > PSD1 = abs(Z1).^2; > Azf1 = fftshift(real(ifft(PSD1))); . > > Notice that fftshift ( and not ifftshift) is used to get a > completely symmetric function about the middle index > N. > > In general, these results will be different from those > obtained from the nonzero-padded estimate > > Z2 = fft(z); > PSD2 = abs(Z2).^2; > Azf2 = fftshift(real(ifft(PSD1))); , > > unless zero-padding doesn't change the shape of the > function. > > For example, the plot of x and [x;zeros(M,1)] are > sufficiently similar whereas the plots of z and > [z;zeros(M,1)] are significantly different. > > I guess the lesson here is that you must be aware of > the zero-padding whenever you use xcorr!Actually, there is one more point to make. First note the decorrelation time estimate in the following table for N = 100: length x z -------- -------- ------- N 2.16 1.32 M 2.14 1.70 Now, notice what happens when the hanning window is applied: length x z -------- -------- ------- N 2.62 1.82 M 2.62 1.83 Finally, the consistency I was looking for! However,... since MATLAB contains 16 different window functions, a precise definition of decorrelation time for a general sampled data time series may not exist. Hope this helps, Greg