Seperating Multiple Signals in the Same Band

Overview of the problem:

I have a quite complicated filtering application here that I could
really use some help and insight on. I'm trying to extract a
repetitive signal (a pulse) that is not cyclical but appears on
certain modulus. (Meaning a pulse could come 2 to 5 modulus after
another.) This places the signal in a certain band, we'll call it the
I band, as in the signal of interest. Now there's a bit of background
noise, repetitive, cyclical, but varies in phase and frequency
slightly over time. (This is caused by a mechanical device!) The
device creates one signal with a very strong amplitude, that has it's
principle energy in about 4 different bands. The first is slightly
lower than I, the next is I itself, and the next two are slightly
higher than I. We'll call these principle frequencies A,B,C, with I
appearing after A and before B in order of increasing frequency from
left to right. Now while A,B, and C are particular and fairly constant
to this device, that is certainly not true of similar devices that
will be encountered in the future, so any method of filtering must be
adaptive. My goal is to eliminate A,B,C (if need be) and subtract that
portion of signal that is caused by any other signal except for the
signal of interest in the band I. The presence of the second signal
overall gives me a signal to noise ration of about -12dB.

What I've tried so far:

Traditional linear filters, even if they're adaptive, appear to be
totally out of the question. Their attenuation of the band I, also
results in the attenuation of I itself.

I can quite easily lock on to the cycle of I, create an average of
it's cycles, and subtract an approximate guess from the current cycle.
This produces mixed results.

I've tried using independent component analysis, but this technique
seems to rely on the fact that the two signals are in separate bands.

Using a short time fourier transform, I can generally see the change
in the I band due to my signal. I have also achieved similar results
with the continuous wavelet transform.

Where am at now:

At this point I could really use some community input as to how to
handle this. I'd love to hear if anyone has had the same problem, has
any suggestions, knows of any literature that could help me, or
anything at all.

Thank you very much for your time.

Regards,
Ken

xkenneth wrote:
> Overview of the problem:
> 
> I have a quite complicated filtering application here that I could
> really use some help and insight on. I'm trying to extract a
> repetitive signal (a pulse) that is not cyclical but appears on
> certain modulus. (Meaning a pulse could come 2 to 5 modulus after
> another.) This places the signal in a certain band, we'll call it the
> I band, as in the signal of interest. Now there's a bit of background
> noise, repetitive, cyclical, but varies in phase and frequency
> slightly over time. (This is caused by a mechanical device!) The
> device creates one signal with a very strong amplitude, that has it's
> principle energy in about 4 different bands. The first is slightly
> lower than I, the next is I itself, and the next two are slightly
> higher than I. We'll call these principle frequencies A,B,C, with I
> appearing after A and before B in order of increasing frequency from
> left to right. Now while A,B, and C are particular and fairly constant
> to this device, that is certainly not true of similar devices that
> will be encountered in the future, so any method of filtering must be
> adaptive. My goal is to eliminate A,B,C (if need be) and subtract that
> portion of signal that is caused by any other signal except for the
> signal of interest in the band I. The presence of the second signal
> overall gives me a signal to noise ration of about -12dB.
> 
> What I've tried so far:
> 
> Traditional linear filters, even if they're adaptive, appear to be
> totally out of the question. Their attenuation of the band I, also
> results in the attenuation of I itself.
> 
> I can quite easily lock on to the cycle of I, create an average of
> it's cycles, and subtract an approximate guess from the current cycle.
> This produces mixed results.
> 
> I've tried using independent component analysis, but this technique
> seems to rely on the fact that the two signals are in separate bands.
> 
> Using a short time fourier transform, I can generally see the change
> in the I band due to my signal. I have also achieved similar results
> with the continuous wavelet transform.
> 
> Where am at now:
> 
> At this point I could really use some community input as to how to
> handle this. I'd love to hear if anyone has had the same problem, has
> any suggestions, knows of any literature that could help me, or
> anything at all.
> 
> Thank you very much for your time.
> 
> Regards,
> Ken

This sounds like a profoundly fun problem to solve; I wish a client 
would call me up with something like this!

Here are no answers, just some questions to clarify your problem:

Is I also synchronized to the same prime mover as A, B and C?

If you know the frequency of A, do you know the frequency of B and C?

Are A, B and C predictable enough that you can subtract them out?  What 
about a plain old set of tunable notches?

Are you trying to find some average value of I, or are you trying to 
find out at each possible iteration if it has fired or not?  More 
broadly, what is the information you're really trying to extract?

If the intended signal is really a pulse, and if it really is close in 
frequency to the interference, you won't be able to eliminate the other 
signals without at least cutting into the intended signal.  I'm not sure 
where your optimum lies, but that's a fact that I think you'll have to 
resign yourself to.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

As Tim mentioned, this sounds like an interesting problem. Any chance
you can post a recording on some website for us to play with?

Are you trying to extract the signal I or just some feature of it
(i.e. the timing of the pulse, etc.).

Maybe something model based would work (i.e. create a model for the
physical cause of A,B,C; estimate the relevant parameters of the
model, then synthesize a signal based on that estimate and subtract as
needed)

Chris

Tim Wescott wrote:
> xkenneth wrote:
>> Overview of the problem:
>>
>> I have a quite complicated filtering application here that I could
>> really use some help and insight on. I'm trying to extract a
>> repetitive signal (a pulse) that is not cyclical but appears on
>> certain modulus. (Meaning a pulse could come 2 to 5 modulus after
>> another.) This places the signal in a certain band, we'll call it the
>> I band, as in the signal of interest. Now there's a bit of background
>> noise, repetitive, cyclical, but varies in phase and frequency
>> slightly over time. (This is caused by a mechanical device!) The
>> device creates one signal with a very strong amplitude, that has it's
>> principle energy in about 4 different bands. The first is slightly
>> lower than I, the next is I itself, and the next two are slightly
>> higher than I. We'll call these principle frequencies A,B,C, with I
>> appearing after A and before B in order of increasing frequency from
>> left to right. Now while A,B, and C are particular and fairly constant
>> to this device, that is certainly not true of similar devices that
>> will be encountered in the future, so any method of filtering must be
>> adaptive. My goal is to eliminate A,B,C (if need be) and subtract that
>> portion of signal that is caused by any other signal except for the
>> signal of interest in the band I. The presence of the second signal
>> overall gives me a signal to noise ration of about -12dB.
>>
>> What I've tried so far:
>>
>> Traditional linear filters, even if they're adaptive, appear to be
>> totally out of the question. Their attenuation of the band I, also
>> results in the attenuation of I itself.
>>
>> I can quite easily lock on to the cycle of I, create an average of
>> it's cycles, and subtract an approximate guess from the current cycle.
>> This produces mixed results.
>>
>> I've tried using independent component analysis, but this technique
>> seems to rely on the fact that the two signals are in separate bands.
>>
>> Using a short time fourier transform, I can generally see the change
>> in the I band due to my signal. I have also achieved similar results
>> with the continuous wavelet transform.
>>
>> Where am at now:
>>
>> At this point I could really use some community input as to how to
>> handle this. I'd love to hear if anyone has had the same problem, has
>> any suggestions, knows of any literature that could help me, or
>> anything at all.
>>
>> Thank you very much for your time.
>>
>> Regards,
>> Ken
> 
> This sounds like a profoundly fun problem to solve; I wish a client 
> would call me up with something like this!
> 
> Here are no answers, just some questions to clarify your problem:
> 
> Is I also synchronized to the same prime mover as A, B and C?
> 
> If you know the frequency of A, do you know the frequency of B and C?
> 
> Are A, B and C predictable enough that you can subtract them out?  What 
> about a plain old set of tunable notches?
> 
> Are you trying to find some average value of I, or are you trying to 
> find out at each possible iteration if it has fired or not?  More 
> broadly, what is the information you're really trying to extract?
> 
> If the intended signal is really a pulse, and if it really is close in 
> frequency to the interference, you won't be able to eliminate the other 
> signals without at least cutting into the intended signal.  I'm not sure 
> where your optimum lies, but that's a fact that I think you'll have to 
> resign yourself to.
> 
I left out the most important question, of which the above questions are 
just corollaries: in what way is your intended signal different from the 
interference?  Some of your description hinted at differences beyond the 
ones you mentioned, but to really sort things out with a filter you need 
to know what to filter out, and what to leave in.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

All,

   Thanks for the speedy replies! I'll answer everybody's questions in
one email.

Here are no answers, just some questions to clarify your problem:
Is I also synchronized to the same prime mover as A, B and C?

I'm not sure what you mean by this...

But there's probably something I forgot to mention. The device that's
causing this noise is a mechanical pump (the signal is pressure over
time of a fluid.) It's got a view cycles that can be represent as
several summed sinusoids of harmonic frequencies. And as in theory
A,I (the part that does not contain my signal), B, C, are all loose
harmonics.

If you know the frequency of A, do you know the frequency of B and C?
Yes, I just need to count on them changing from device to device, so I
can't have a hard coded solution.

Are A, B and C predictable enough that you can subtract them out?
They vary in amplitude and frequency slightly over time, so
subtraction is a bit shaky.

What
about a plain old set of tunable notches?

I've have no experience with these,(this is my first signal processing
project!) I'll do some research. Do you have any good links?

Are you trying to find some average value of I, or are you trying to
find out at each possible iteration if it has fired or not?

I assumed the easiest way was to find out the average magnitude of the
band I, and then use that as a baseline to look for my signal.

More broadly, what is the information you're really trying to
extract?

The peak of the pulse.

If the intended signal is really a pulse, and if it really is close
in
frequency to the interference, you won't be able to eliminate the
other
signals without at least cutting into the intended signal.

This seems to be the way it's happening.

As Tim mentioned, this sounds like an interesting problem. Any chance
you can post a recording on some website for us to play with?
I'll see if this is possible.


Are you trying to extract the signal I or just some feature of it

Timing/Peak of the pulse.

Maybe something model based would work (i.e. create a model for the
physical cause of A,B,C; estimate the relevant parameters of the
model, then synthesize a signal based on that estimate and subtract
as
needed)

I've gone down this path, with mixed results, the fact that the
signals are not "stable" (they are for a generally long period of
time) over frequency and time make it a pain to recreate the signal.

Iin what way is your intended signal different from the
interference?
The interference is sinusoidal, even though it's composed of multiple
sinusoids. My signal is a pulse that "could" be though of a half-cycle
sin wave. I hope there's enough difference between those two. The main
point here is that the pulse causes the energy in it's band to
increase when it occurs.

Some of your description hinted at differences beyond the
ones you mentioned, but to really sort things out with a filter you
need
to know what to filter out, and what to leave in.
Could you elaborate on this?

Regards,
Ken

All,
	I don't think my first reply made it through, so I'll try this again.
I'll answer all of the questions in one email.

Is I also synchronized to the same prime mover as A, B and C?

Not sure what you mean by this..?

If you know the frequency of A, do you know the frequency of B and C?

Yes, in fact they are all loosely harmonics. I say loosely because
it's a mechanical system and they're slightly off harmonic. +/-5%

Are A, B and C predictable enough that you can subtract them out?

I've tried this, but they vary slightly in frequency and amplitude
over time, so it's a bit hard to do.



 What about a plain old set of tunable notches?

I have no experience with this, I'll do some research. Do you have any
good links?

Are you trying to find some average value of I, or are you trying to
find out at each possible iteration if it has fired or not?

I assumed the easiest method was to find the average magnitude of the
band I and remove that, essentially finding the baseline for my
signal.

More
broadly, what is the information you're really trying to extract?

The peak of the pulse.

If the intended signal is really a pulse,

yes

and if it really is close in
frequency to the interference, you won't be able to eliminate the
other
signals without at least cutting into the intended signal.

I believe so...

 I'm not sure
where your optimum lies, but that's a fact that I think you'll have
to
resign yourself to.

Ok

As Tim mentioned, this sounds like an interesting problem. Any chance
you can post a recording on some website for us to play with?

I'll try..
Are you trying to extract the signal I or just some feature of it
(i.e. the timing of the pulse, etc.).

Exactly, the timing of the pulse.

Maybe something model based would work (i.e. create a model for the
physical cause of A,B,C; estimate the relevant parameters of the
model, then synthesize a signal based on that estimate and subtract
as
needed)

I've tried this as well as subtraction with mixed results. The device
is electromechanical, the it can vary quite a bit from cycle to cycle.

Chris



I left out the most important question, of which the above questions
are
just corollaries: in what way is your intended signal different from
the
interference?

I'm not sure that it is, the interference is a sinusoid, as well as
the pulse can be viewed as a half-cycle sinusoid.

Some of your description hinted at differences beyond the
ones you mentioned, but to really sort things out with a filter you
need
to know what to filter out, and what to leave in.

Right.

Regards,

Ken

xkenneth wrote:
> All,
> 	I don't think my first reply made it through, so I'll try this again.
> I'll answer all of the questions in one email.
> 
> Is I also synchronized to the same prime mover as A, B and C?
> 
> Not sure what you mean by this..?

Are the signal sources all turned from the same motor, and connected 
rigidly (or at least by belts, as opposed to hydraulics).  I.e., if you 
know how A is going, do you automatically know B and C or do you have to 
find them individually.
> 
> If you know the frequency of A, do you know the frequency of B and C?
> 
> Yes, in fact they are all loosely harmonics. I say loosely because
> it's a mechanical system and they're slightly off harmonic. +/-5%
> 
> Are A, B and C predictable enough that you can subtract them out?
> 
> I've tried this, but they vary slightly in frequency and amplitude
> over time, so it's a bit hard to do.
> 
> 
> 
>  What about a plain old set of tunable notches?
> 
> I have no experience with this, I'll do some research. Do you have any
> good links?

(a), no, and (b), a notch filter only works as well as the 
predictability of the signal that it's notching out.

> Are you trying to find some average value of I, or are you trying to
> find out at each possible iteration if it has fired or not?
> 
> I assumed the easiest method was to find the average magnitude of the
> band I and remove that, essentially finding the baseline for my
> signal.
> 
> More
> broadly, what is the information you're really trying to extract?
> 
> The peak of the pulse.
> 
> If the intended signal is really a pulse,
> 
> yes
> 
> and if it really is close in
> frequency to the interference, you won't be able to eliminate the
> other
> signals without at least cutting into the intended signal.
> 
> I believe so...
> 
>  I'm not sure
> where your optimum lies, but that's a fact that I think you'll have
> to
> resign yourself to.
> 
> Ok
> 
> As Tim mentioned, this sounds like an interesting problem. Any chance
> you can post a recording on some website for us to play with?
> 
> I'll try..
> Are you trying to extract the signal I or just some feature of it
> (i.e. the timing of the pulse, etc.).
> 
> Exactly, the timing of the pulse.
> 
> Maybe something model based would work (i.e. create a model for the
> physical cause of A,B,C; estimate the relevant parameters of the
> model, then synthesize a signal based on that estimate and subtract
> as
> needed)
> 
> I've tried this as well as subtraction with mixed results. The device
> is electromechanical, the it can vary quite a bit from cycle to cycle.
> 
> Chris
> 
> 
> 
> I left out the most important question, of which the above questions
> are
> just corollaries: in what way is your intended signal different from
> the
> interference?
> 
> I'm not sure that it is, the interference is a sinusoid, as well as
> the pulse can be viewed as a half-cycle sinusoid.
> 
> Some of your description hinted at differences beyond the
> ones you mentioned, but to really sort things out with a filter you
> need
> to know what to filter out, and what to leave in.
> 
> Right.
> 
> Regards,
> 
> Ken
> 
You really have two problems: first you have to identify when a pulse 
happened, then you have to pull out it's peak energy.  You are correct 
in thinking that if you could eliminate the 'A, B and C' signals that 
you'd be left with the intended signal.

This sounds like a problem where once you know what questions to ask 
they answer themselves; unfortunately I feel like I'm asking the wrong 
questions...

So just what differences in frequency _are_ you looking at between A, I 
and B?  That has a lot of bearing on what you can do with this.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

Ok a little more info here:

http://xkenneth.wordpress.com/signal-processing-problem/

On  the above page, you can see an FFT of the device's signature with
and without the signal, as well as the STFT of the raw signal over
time. Re-evaluating the FFT's with and without the signal, I can see
that the signal is not in one distinct band as I had thought, although
I'm not sure how I overlooked this. The spectrum of the pulse seems to
be spread out with energy in one of the pumps bands, but also in
several others. Looking at the STFT, the major ridges are the overall
change in frequency due to the pulse. (It's coming in at regular
intervals for this data) The window size for the STFT is small, and
therefore reflects generally overall change of all of the bands over
time.

Regards,
Ken

Lateral thoughts :

Can you use other sensors, to extract more data ?

- an angle sensor on the motor shaft ?
This would perhaps allow a better cancellation of the noise...

- a second pressure sensor located somewhere else ?
If the signal of interest comes from point B and the noise comes from  
point A, both propagate in opposite directions in your pipe so if you use  
2 sensors you can use the phase information in your processing (like  
diversity).

What physical event causes the signal of interest ?

On Tue, 10 Jun 2008 16:10:37 -0700, xkenneth wrote:

> Ok a little more info here:
> 
> http://xkenneth.wordpress.com/signal-processing-problem/
> 
> On  the above page, you can see an FFT of the device's signature with
> and without the signal, as well as the STFT of the raw signal over time.
> Re-evaluating the FFT's with and without the signal, I can see that the
> signal is not in one distinct band as I had thought, although I'm not
> sure how I overlooked this. The spectrum of the pulse seems to be spread
> out with energy in one of the pumps bands, but also in several others.
> Looking at the STFT, the major ridges are the overall change in
> frequency due to the pulse. (It's coming in at regular intervals for
> this data) The window size for the STFT is small, and therefore reflects
> generally overall change of all of the bands over time.
> 
> Regards,
> Ken

It would appear that a good sharp low-pass filter at a frequency of 0.5 
whatevers/whatever (no axis labels!) would certainly distinguish your 
signal from your noise.  Whether it would leave enough of your signal to 
do the job you want to do is open to conjecture, but have you tried it?

-- 
Tim Wescott
Control systems and communications consulting
http://www.wescottdesign.com

Need to learn how to apply control theory in your embedded system?
"Applied Control Theory for Embedded Systems" by Tim Wescott
Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html