Finding windows of high correlation?

Hi everyone,

I know this is a pretty classic problem, however I couldn't find any
useful links or introductions into solving this problem.

I basically have two simple signals which have low correlation mostly,
however given certain conditions will maintain high levels of
correlation.   Is there an optimal way to detect those periods of high
correlation?   I can use a sliding window approach, but that seems
somewhat naive.     I am looking at using DFT, but I'm unclear exactly
on what the alg for that would look like or how much in the way of
processing that will save me.

Any pointers, links, suggestions would be very much appreciated.

Cheers,

Kate.

On Feb 21, 5:01 am, ksheph...@gmail.com wrote:
> Hi everyone,
>
> I know this is a pretty classic problem, however I couldn't find any
> useful links or introductions into solving this problem.
>
> I basically have two simple signals which have low correlation mostly,
> however given certain conditions will maintain high levels of
> correlation.   Is there an optimal way to detect those periods of high
> correlation?   I can use a sliding window approach, but that seems
> somewhat naive.     I am looking at using DFT, but I'm unclear exactly
> on what the alg for that would look like or how much in the way of
> processing that will save me.
>
> Any pointers, links, suggestions would be very much appreciated.
>
> Cheers,
>
> Kate.

As you thought, you could convolve the signals together to determine
where their maximum cross-correlation is at a maximum. This is the
exact operation performed by matched filters in communications
systems. As always, you can use an FFT-based fast convolution approach
to speed the process up. Try this link:

http://www.dspguide.com/ch18/1.htm

Jason

On Feb 21, 4:01 am, ksheph...@gmail.com wrote:
> Hi everyone,
>
> I know this is a pretty classic problem, however I couldn't find any
> useful links or introductions into solving this problem.
>
> I basically have two simple signals which have low correlation mostly,
> however given certain conditions will maintain high levels of
> correlation.

I think it will help significantly if you re-state
the above more precisely.

What do you mean that your signals are "simple"?

And what are these "certain conditions" that will
maintain high levels of correlation?  Is it because
the signal levels change?

Are you correlating an incoming stream with a
template signal?

Julius

On Feb 21, 7:15 am, "julius" <juli...@gmail.com> wrote:
> On Feb 21, 4:01 am, ksheph...@gmail.com wrote:
>
> > Hi everyone,
>
> > I know this is a pretty classic problem, however I couldn't find any
> > useful links or introductions into solving this problem.
>
> > I basically have two simple signals which have low correlation mostly,
> > however given certain conditions will maintain high levels of
> > correlation.
>
> I think it will help significantly if you re-state
> the above more precisely.
>
> What do you mean that your signals are "simple"?
>
> And what are these "certain conditions" that will
> maintain high levels of correlation?  Is it because
> the signal levels change?
>
> Are you correlating an incoming stream with a
> template signal?
>
> Julius


Certainly, they are two seperate audio signals.   I'm not sure what
the conditions are just yet, however I am hoping to determine them by
locating periods of high correlation between the two signals.

Thanks for you the link to DSP guide, it was pure gold.

Cheers,

Kate.

kshephard@gmail.com wrote:

   ...

> Certainly, they are two seperate audio signals.   I'm not sure what
> the conditions are just yet, however I am hoping to determine them by
> locating periods of high correlation between the two signals.

   ...

Correlation involves extended time.* How far separated might your 
"periods of high correlation" be?

Jerry
______________________________________
The great pianist Vladimir Horowitz held the non-mystical view that the 
sound of a piano is determined by the velocity and timing of the hammers 
and dampers on the strings, nothing more. He said, "There is such a 
thing as pianistic touch, but it can't be demonstrated with one note."
-- 
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

On 21 Feb, 11:01, ksheph...@gmail.com wrote:
> Hi everyone,
>
> I know this is a pretty classic problem, however I couldn't find any
> useful links or introductions into solving this problem.
>
> I basically have two simple signals which have low correlation mostly,
> however given certain conditions will maintain high levels of
> correlation.   Is there an optimal way to detect those periods of high
> correlation?   I can use a sliding window approach, but that seems
> somewhat naive.     I am looking at using DFT, but I'm unclear exactly
> on what the alg for that would look like or how much in the way of
> processing that will save me.
>
> Any pointers, links, suggestions would be very much appreciated.

It is a bit unclear exactly what you want to measure. Do you expect
the high correlaon to occur with reative time shifts between the
data,
or does it suffice to monitor the zero-lag correlation coefficient?

If you can get away with monitoring the zero-lag coefficient, you can
do it very efficiently as follows:

--- Initialization

1) Decide on a window length N
2) Compute the zero-lag correlation coefficient for the
   N first samples in the two data streams
3) Store the current coefficient as r0(0)

--- Loop
for n = 1 to ... do
   4) Subtract the contribution from the two earliest data points
       x(n-1)*conj(y(n-1)), from r0(n-1)
   5) Add the contribution from the next data points, x(n+N)*conj(y(n
+N)),
       to the correlation and store the result as r0(n)
end

and that's about it.

Rune