Hi all,
I am not a DSP expert so please go easy on me. I have a VLF radio signal
centred at 3KHz. The signal is corrupted with the mains harmonics, which
seem to be over 20dB higher than the signal!
I am trying to implement an algorithm that isolates an out-of-band harmonic
using a band-pass filter, resamples it several times to obtain the in-band
harmonics, and subtracts them from the corrupted signal.
Let's say I have a reference harmonic x with N samples and I do
y = resample(x,I,Q);
the length of y will be N*I/Q samples.
I need then to take the the fft of y and z (the corrupted signal), subtract
them and reconstruct z using the ifft.
My understanding is that I need to use the same Nfft for both ffts in order
to be able to subtract one from the other.
What value should that be? N or N*I/Q?
And what should the ifft factor be?
I would be very grateful if someone could give me some pointers. I am doing
this in Matlab and any pieces of code will be very much appreciated.
Kind regards,
Samir.
Resampling for interference cancellation
Started by ●October 13, 2010
Reply by ●October 13, 20102010-10-13
On 10/13/2010 09:37 AM, bendous wrote:> Hi all, > > I am not a DSP expert so please go easy on me. I have a VLF radio signal > centred at 3KHz. The signal is corrupted with the mains harmonics, which > seem to be over 20dB higher than the signal! > > I am trying to implement an algorithm that isolates an out-of-band harmonic > using a band-pass filter, resamples it several times to obtain the in-band > harmonics, and subtracts them from the corrupted signal. > > Let's say I have a reference harmonic x with N samples and I do > > y = resample(x,I,Q); > > the length of y will be N*I/Q samples. > > I need then to take the the fft of y and z (the corrupted signal), subtract > them and reconstruct z using the ifft. > > My understanding is that I need to use the same Nfft for both ffts in order > to be able to subtract one from the other. > > What value should that be? N or N*I/Q? > And what should the ifft factor be? > > I would be very grateful if someone could give me some pointers. I am doing > this in Matlab and any pieces of code will be very much appreciated.I think a comb filter would be easier to implement, and likely would work better. Better yet, see if you can come up with a means of reception that naturally rejects harmonics -- have you tried a loop antenna instead of a whip, or visa-versa? Have you experimented with antenna polarization? Have you tried running the receiver off of battery power (which would at least let you know that it's power supply noise)? -- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" was written for you. See details at http://www.wescottdesign.com/actfes/actfes.html
Reply by ●October 13, 20102010-10-13
On Oct 13, 12:37=A0pm, "bendous" <samir@n_o_s_p_a_m.eee.strath.ac.uk> wrote:> Hi all, > > I am not a DSP expert so please go easy on me. I have a VLF radio signal > centred at 3KHz. The signal is corrupted with the mains harmonics, which > seem to be over 20dB higher than the signal! > > I am trying to implement an algorithm that isolates an out-of-band harmon=ic> using a band-pass filter, resamples it several times to obtain the in-ban=d> harmonics, and subtracts them from the corrupted signal. > > Let's say I have a reference harmonic x with N samples and I do > > =A0 =A0 =A0 =A0 =A0y =3D resample(x,I,Q); > > the length of y will be N*I/Q samples. > > I need then to take the the fft of y and z (the corrupted signal), subtra=ct> them and reconstruct z using the ifft. > > My understanding is that I need to use the same Nfft for both ffts in ord=er> to be able to subtract one from the other. > > What value should that be? N or N*I/Q? > And what should the ifft factor be? > > I would be very grateful if someone could give me some pointers. I am doi=ng> this in Matlab and any pieces of code will be very much appreciated. > > Kind regards, > Samir.Samir, Keep in mind that the phase of the subsampled signals may not be correct; if you were going to actually subtract the complex FFTs, that would effect the amount of cancellation. Since the harmonics would probably (almost guaranteed) fall in more than a single FFT bin there are multiple FFT bin phases per harmonic to deal with. Is the signal bandwidth sufficiently low that significant low frequency harmonics can be removed by a lowpass filter? The best solution if it is possible, as Tim says below, is don't let the power line harmonics get in the signal in the first place. Dirk
Reply by ●October 13, 20102010-10-13
On 10/13/2010 03:01 PM, Dirk Bell wrote:> On Oct 13, 12:37 pm, "bendous"<samir@n_o_s_p_a_m.eee.strath.ac.uk> > wrote: >> Hi all, >> >> I am not a DSP expert so please go easy on me. I have a VLF radio signal >> centred at 3KHz. The signal is corrupted with the mains harmonics, which >> seem to be over 20dB higher than the signal! >> >> I am trying to implement an algorithm that isolates an out-of-band harmonic >> using a band-pass filter, resamples it several times to obtain the in-band >> harmonics, and subtracts them from the corrupted signal. >> >> Let's say I have a reference harmonic x with N samples and I do >> >> y = resample(x,I,Q); >> >> the length of y will be N*I/Q samples. >> >> I need then to take the the fft of y and z (the corrupted signal), subtract >> them and reconstruct z using the ifft. >> >> My understanding is that I need to use the same Nfft for both ffts in order >> to be able to subtract one from the other. >> >> What value should that be? N or N*I/Q? >> And what should the ifft factor be? >> >> I would be very grateful if someone could give me some pointers. I am doing >> this in Matlab and any pieces of code will be very much appreciated. >> >> Kind regards, >> Samir. > > Samir, > > Keep in mind that the phase of the subsampled signals may not be > correct; if you were going to actually subtract the complex FFTs, that > would effect the amount of cancellation. Since the harmonics would > probably (almost guaranteed) fall in more than a single FFT bin there > are multiple FFT bin phases per harmonic to deal with.Which is why I suggested a comb filter (inverse comb filter? multiple-notch filter?). By the time you figure out the phase and amplitude of each harmonic and inject them back into the signal "in reverse" to cancel them, you've essentially built a comb filter but in a really wacky and hard-to-understand way.> Is the signal bandwidth sufficiently low that significant low > frequency harmonics can be removed by a lowpass filter? > > The best solution if it is possible, as Tim says below, is don't let > the power line harmonics get in the signal in the first place. > > Dirk-- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" was written for you. See details at http://www.wescottdesign.com/actfes/actfes.html
Reply by ●October 13, 20102010-10-13
On 10/13/2010 4:44 PM, Tim Wescott wrote:> On 10/13/2010 03:01 PM, Dirk Bell wrote: >> On Oct 13, 12:37 pm, "bendous"<samir@n_o_s_p_a_m.eee.strath.ac.uk> >> wrote: >>> Hi all, >>> >>> I am not a DSP expert so please go easy on me. I have a VLF radio signal >>> centred at 3KHz. The signal is corrupted with the mains harmonics, which >>> seem to be over 20dB higher than the signal! >>> >>> I am trying to implement an algorithm that isolates an out-of-band >>> harmonic >>> using a band-pass filter, resamples it several times to obtain the >>> in-band >>> harmonics, and subtracts them from the corrupted signal. >>> >>> Let's say I have a reference harmonic x with N samples and I do >>> >>> y = resample(x,I,Q); >>> >>> the length of y will be N*I/Q samples. >>> >>> I need then to take the the fft of y and z (the corrupted signal), >>> subtract >>> them and reconstruct z using the ifft. >>> >>> My understanding is that I need to use the same Nfft for both ffts in >>> order >>> to be able to subtract one from the other. >>> >>> What value should that be? N or N*I/Q? >>> And what should the ifft factor be? >>> >>> I would be very grateful if someone could give me some pointers. I am >>> doing >>> this in Matlab and any pieces of code will be very much appreciated. >>> >>> Kind regards, >>> Samir. >> >> Samir, >> >> Keep in mind that the phase of the subsampled signals may not be >> correct; if you were going to actually subtract the complex FFTs, that >> would effect the amount of cancellation. Since the harmonics would >> probably (almost guaranteed) fall in more than a single FFT bin there >> are multiple FFT bin phases per harmonic to deal with. > > Which is why I suggested a comb filter (inverse comb filter? > multiple-notch filter?). By the time you figure out the phase and > amplitude of each harmonic and inject them back into the signal "in > reverse" to cancel them, you've essentially built a comb filter but in a > really wacky and hard-to-understand way. > >> Is the signal bandwidth sufficiently low that significant low >> frequency harmonics can be removed by a lowpass filter? >> >> The best solution if it is possible, as Tim says below, is don't let >> the power line harmonics get in the signal in the first place. >> >> Dirk > >I agree with Tim and Dirk. Depending on how much data you have, this seems like relatively natural application for an automatic line canceller ALC. That is if the mains harmonics are relatively stable and the VLF signal has some dynamics that would not cause it to be cancelled. Effectively this is close to what you were trying to do with subtraction but without the FFT / IFFT. Comments anyone? If the harmonics are of 50/60Hz then there will be a lot of them around 3kHz, right? Fred
Reply by ●October 14, 20102010-10-14
On Oct 13, 8:07�pm, Fred Marshall <fmarshall_xremove_the...@xacm.org> wrote:> On 10/13/2010 4:44 PM, Tim Wescott wrote: > > > > > On 10/13/2010 03:01 PM, Dirk Bell wrote: > >> On Oct 13, 12:37 pm, "bendous"<samir@n_o_s_p_a_m.eee.strath.ac.uk> > >> wrote: > >>> Hi all, > > >>> I am not a DSP expert so please go easy on me. I have a VLF radio signal > >>> centred at 3KHz. The signal is corrupted with the mains harmonics, which > >>> seem to be over 20dB higher than the signal! > > >>> I am trying to implement an algorithm that isolates an out-of-band > >>> harmonic > >>> using a band-pass filter, resamples it several times to obtain the > >>> in-band > >>> harmonics, and subtracts them from the corrupted signal. > > >>> Let's say I have a reference harmonic x with N samples and I do > > >>> y = resample(x,I,Q); > > >>> the length of y will be N*I/Q samples. > > >>> I need then to take the the fft of y and z (the corrupted signal), > >>> subtract > >>> them and reconstruct z using the ifft. > > >>> My understanding is that I need to use the same Nfft for both ffts in > >>> order > >>> to be able to subtract one from the other. > > >>> What value should that be? N or N*I/Q? > >>> And what should the ifft factor be? > > >>> I would be very grateful if someone could give me some pointers. I am > >>> doing > >>> this in Matlab and any pieces of code will be very much appreciated. > > >>> Kind regards, > >>> Samir. > > >> Samir, > > >> Keep in mind that the phase of the subsampled signals may not be > >> correct; if you were going to actually subtract the complex FFTs, that > >> would effect the amount of cancellation. Since the harmonics would > >> probably (almost guaranteed) fall in more than a single FFT bin there > >> are multiple FFT bin phases per harmonic to deal with. > > > Which is why I suggested a comb filter (inverse comb filter? > > multiple-notch filter?). By the time you figure out the phase and > > amplitude of each harmonic and inject them back into the signal "in > > reverse" to cancel them, you've essentially built a comb filter but in a > > really wacky and hard-to-understand way. > > >> Is the signal bandwidth sufficiently low that significant low > >> frequency harmonics can be removed by a lowpass filter? > > >> The best solution if it is possible, as Tim says below, is don't let > >> the power line harmonics get in the signal in the first place. > > >> Dirk > > I agree with Tim and Dirk. �Depending on how much data you have, this > seems like relatively natural application for an automatic line > canceller ALC. �That is if the mains harmonics are relatively stable and > the VLF signal has some dynamics that would not cause it to be > cancelled. �Effectively this is close to what you were trying to do with > subtraction but without the FFT / IFFT. > > Comments anyone? > > If the harmonics are of 50/60Hz then there will be a lot of them around > 3kHz, right? > > Fred- Hide quoted text - > > - Show quoted text -You would think the level of the harmonics would be pretty low at 3kHz (20th/30th harmonic at 50/60 Hz). That is why I was asking about the bandwidth and a highpass filter (originally typed lowpass while thinking highpass) since the signal is centered at 3 KHz. A simple comb filter is potentially going to remove a lot of the signal and the high attenuation part of the all of the notches is so narrow that the first notch being off a little from the signal means that the attenuation isn't nearly as good. It also means the attenuation decreases as the harmonics go up because the mistuning increases, so harmonics can be missed altogether. If the notch frequencies are static, then power line frequency may wander into and out of the deep part of the notches over time. With audio (not the target signal here) I have seen improvements in descreasing the power line frequency and harmonics signals, but the problem isn't solved as far as I know. Dirk
Reply by ●October 14, 20102010-10-14
On Oct 13, 6:01�pm, Dirk Bell <bellda2...@cox.net> wrote:> On Oct 13, 12:37�pm, "bendous" <samir@n_o_s_p_a_m.eee.strath.ac.uk> > wrote: > > > > > > > Hi all, > > > I am not a DSP expert so please go easy on me. I have a VLF radio signal > > centred at 3KHz. The signal is corrupted with the mains harmonics, which > > seem to be over 20dB higher than the signal! > > > I am trying to implement an algorithm that isolates an out-of-band harmonic > > using a band-pass filter, resamples it several times to obtain the in-band > > harmonics, and subtracts them from the corrupted signal. > > > Let's say I have a reference harmonic x with N samples and I do > > > � � � � �y = resample(x,I,Q); > > > the length of y will be N*I/Q samples. > > > I need then to take the the fft of y and z (the corrupted signal), subtract > > them and reconstruct z using the ifft. > > > My understanding is that I need to use the same Nfft for both ffts in order > > to be able to subtract one from the other. > > > What value should that be? N or N*I/Q? > > And what should the ifft factor be? > > > I would be very grateful if someone could give me some pointers. I am doing > > this in Matlab and any pieces of code will be very much appreciated. > > > Kind regards, > > Samir. > > Samir, > > Keep in mind that the phase of the subsampled signals may not be > correct; if you were going to actually subtract the complex FFTs, that > would effect the amount of cancellation. �Since the harmonics would > probably (almost guaranteed) fall in more than a single FFT bin there > are multiple FFT bin phases per harmonic to deal with. > > Is the signal bandwidth sufficiently low that significant low > frequency harmonics can be removed by a lowpass filter? > > The best solution if it is possible, as Tim says below, is don't let > the power line harmonics get in the signal in the first place. > > Dirk- Hide quoted text - > > - Show quoted text -'lowpass' should be 'highpass'. Fingers and brain weren't working together. Dirk
Reply by ●October 14, 20102010-10-14
On Oct 13, 8:07�pm, Fred Marshall <fmarshall_xremove_the...@xacm.org> wrote:> On 10/13/2010 4:44 PM, Tim Wescott wrote: > > > > > On 10/13/2010 03:01 PM, Dirk Bell wrote: > >> On Oct 13, 12:37 pm, "bendous"<samir@n_o_s_p_a_m.eee.strath.ac.uk> > >> wrote: > >>> Hi all, > > >>> I am not a DSP expert so please go easy on me. I have a VLF radio signal > >>> centred at 3KHz. The signal is corrupted with the mains harmonics, which > >>> seem to be over 20dB higher than the signal! > > >>> I am trying to implement an algorithm that isolates an out-of-band > >>> harmonic > >>> using a band-pass filter, resamples it several times to obtain the > >>> in-band > >>> harmonics, and subtracts them from the corrupted signal. > > >>> Let's say I have a reference harmonic x with N samples and I do > > >>> y = resample(x,I,Q); > > >>> the length of y will be N*I/Q samples. > > >>> I need then to take the the fft of y and z (the corrupted signal), > >>> subtract > >>> them and reconstruct z using the ifft. > > >>> My understanding is that I need to use the same Nfft for both ffts in > >>> order > >>> to be able to subtract one from the other. > > >>> What value should that be? N or N*I/Q? > >>> And what should the ifft factor be? > > >>> I would be very grateful if someone could give me some pointers. I am > >>> doing > >>> this in Matlab and any pieces of code will be very much appreciated. > > >>> Kind regards, > >>> Samir. > > >> Samir, > > >> Keep in mind that the phase of the subsampled signals may not be > >> correct; if you were going to actually subtract the complex FFTs, that > >> would effect the amount of cancellation. Since the harmonics would > >> probably (almost guaranteed) fall in more than a single FFT bin there > >> are multiple FFT bin phases per harmonic to deal with. > > > Which is why I suggested a comb filter (inverse comb filter? > > multiple-notch filter?). By the time you figure out the phase and > > amplitude of each harmonic and inject them back into the signal "in > > reverse" to cancel them, you've essentially built a comb filter but in a > > really wacky and hard-to-understand way. > > >> Is the signal bandwidth sufficiently low that significant low > >> frequency harmonics can be removed by a lowpass filter? > > >> The best solution if it is possible, as Tim says below, is don't let > >> the power line harmonics get in the signal in the first place. > > >> Dirk > > I agree with Tim and Dirk. �Depending on how much data you have, this > seems like relatively natural application for an automatic line > canceller ALC. �That is if the mains harmonics are relatively stable and > the VLF signal has some dynamics that would not cause it to be > cancelled. �Effectively this is close to what you were trying to do with > subtraction but without the FFT / IFFT. > > Comments anyone? > > If the harmonics are of 50/60Hz then there will be a lot of them around > 3kHz, right? > > Fred- Hide quoted text - > > - Show quoted text -You would think the level of the harmonics would be pretty low at 3kHz (60th/50th harmonic at 50/60 Hz). That is why I was asking about the bandwidth and a highpass filter (originally typed lowpass while thinking highpass) since the signal is centered at 3 KHz. A simple comb filter is potentially going to remove a lot of the signal and the high attenuation part of the all of the notches is so narrow that the first notch being off a little from the signal means that the attenuation isn't nearly as good. It also means the attenuation decreases as the harmonics go up because the mistuning increases, so harmonics can be missed altogether. If the notch frequencies are static, then power line frequency may wander into and out of the deep part of the notches over time. With audio (not the target signal here) I have seen improvements in descreasing the power line frequency and harmonics signals, but the problem isn't solved as far as I know. Dirk
Reply by ●October 14, 20102010-10-14
Dirk Bell wrote:> You would think the level of the harmonics would be pretty low at 3kHz > (20th/30th harmonic at 50/60 Hz).In fact, the power harmonics can go pretty high; they can dominate over other noise up to the frequencies of 50kHz or even higher.> That is why I was asking about the > bandwidth and a highpass filter (originally typed lowpass while > thinking highpass) since the signal is centered at 3 KHz.The vicinity of 3kHz is the one of the worst possible choices as it is heavily hit by atmospheric and man-made interferrence.> A simple comb filter is potentially going to remove a lot of the > signal and the high attenuation part of the all of the notches is so > narrow that the first notch being off a little from the signal means > that the attenuation isn't nearly as good. It also means the > attenuation decreases as the harmonics go up because the mistuning > increases, so harmonics can be missed altogether. If the notch > frequencies are static, then power line frequency may wander into and > out of the deep part of the notches over time. > With audio (not the target signal here) I have seen improvements in > descreasing the power line frequency and harmonics signals, but the > problem isn't solved as far as I know.The power interferrence is rather non-stationary; the non-stationarity sets the limit of how much improvement the comb filter could do. Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
Reply by ●October 14, 20102010-10-14
On 10/14/2010 7:47 AM, Dirk Bell wrote:> On Oct 13, 8:07 pm, Fred Marshall<fmarshall_xremove_the...@xacm.org> > wrote: >> On 10/13/2010 4:44 PM, Tim Wescott wrote: >> >> >> >>> On 10/13/2010 03:01 PM, Dirk Bell wrote: >>>> On Oct 13, 12:37 pm, "bendous"<samir@n_o_s_p_a_m.eee.strath.ac.uk> >>>> wrote: >>>>> Hi all, >> >>>>> I am not a DSP expert so please go easy on me. I have a VLF radio signal >>>>> centred at 3KHz. The signal is corrupted with the mains harmonics, which >>>>> seem to be over 20dB higher than the signal! >> >>>>> I am trying to implement an algorithm that isolates an out-of-band >>>>> harmonic >>>>> using a band-pass filter, resamples it several times to obtain the >>>>> in-band >>>>> harmonics, and subtracts them from the corrupted signal. >> >>>>> Let's say I have a reference harmonic x with N samples and I do >> >>>>> y = resample(x,I,Q); >> >>>>> the length of y will be N*I/Q samples. >> >>>>> I need then to take the the fft of y and z (the corrupted signal), >>>>> subtract >>>>> them and reconstruct z using the ifft. >> >>>>> My understanding is that I need to use the same Nfft for both ffts in >>>>> order >>>>> to be able to subtract one from the other. >> >>>>> What value should that be? N or N*I/Q? >>>>> And what should the ifft factor be? >> >>>>> I would be very grateful if someone could give me some pointers. I am >>>>> doing >>>>> this in Matlab and any pieces of code will be very much appreciated. >> >>>>> Kind regards, >>>>> Samir. >> >>>> Samir, >> >>>> Keep in mind that the phase of the subsampled signals may not be >>>> correct; if you were going to actually subtract the complex FFTs, that >>>> would effect the amount of cancellation. Since the harmonics would >>>> probably (almost guaranteed) fall in more than a single FFT bin there >>>> are multiple FFT bin phases per harmonic to deal with. >> >>> Which is why I suggested a comb filter (inverse comb filter? >>> multiple-notch filter?). By the time you figure out the phase and >>> amplitude of each harmonic and inject them back into the signal "in >>> reverse" to cancel them, you've essentially built a comb filter but in a >>> really wacky and hard-to-understand way. >> >>>> Is the signal bandwidth sufficiently low that significant low >>>> frequency harmonics can be removed by a lowpass filter? >> >>>> The best solution if it is possible, as Tim says below, is don't let >>>> the power line harmonics get in the signal in the first place. >> >>>> Dirk >> >> I agree with Tim and Dirk. Depending on how much data you have, this >> seems like relatively natural application for an automatic line >> canceller ALC. That is if the mains harmonics are relatively stable and >> the VLF signal has some dynamics that would not cause it to be >> cancelled. Effectively this is close to what you were trying to do with >> subtraction but without the FFT / IFFT. >> >> Comments anyone? >> >> If the harmonics are of 50/60Hz then there will be a lot of them around >> 3kHz, right? >> >> Fred- Hide quoted text - >> >> - Show quoted text - > > You would think the level of the harmonics would be pretty low at 3kHz > (20th/30th harmonic at 50/60 Hz). That is why I was asking about the > bandwidth and a highpass filter (originally typed lowpass while > thinking highpass) since the signal is centered at 3 KHz. > > A simple comb filter is potentially going to remove a lot of the > signal and the high attenuation part of the all of the notches is so > narrow that the first notch being off a little from the signal means > that the attenuation isn't nearly as good. It also means the > attenuation decreases as the harmonics go up because the mistuning > increases, so harmonics can be missed altogether. If the notch > frequencies are static, then power line frequency may wander into and > out of the deep part of the notches over time. > > With audio (not the target signal here) I have seen improvements in > descreasing the power line frequency and harmonics signals, but the > problem isn't solved as far as I know. > > DirkThat's why I was suggesting a line canceller. The objective is to get the adaptive FIR filter in the block diagram to become a comb passband filter for the interference only - for subsequent subtraction. To the extent that this block isn't too selective (i.e. long transient response) it will track noise phase because that's based on the input. Then, if the signal dynamics are OK, the signal won't be subtracted at the output. Depending on signal and noise dynamics (e.g. if the signal has a short correlation time relative to the noise) then one might delay one of the paths so that the signal isn't cancelled (because the two paths are uncorrelated) but the noise is. I'm sure I'm being a bit cryptic here but it's something that would have to be worked out with real signals. Fred
