Filtering out 60Hz + harmonics

Hello all,

I am quite new to the world of DSP and am required to perform some DSP for
my student job using Matlab. I need to remove the noise ofthe utility
frequencies (60 Hz + harmonics) from a number of signals. I have been
reading many forums on this topic and tried many possible solutions but was
unable to get it to work properly.

For instance, I tried using a notch filter and also tried a comb filter
which worked for the first few harmonics but then started to overcompensate
and would create spikes in the opposite direction. 

I also tried modifying the magnitude of the fft (I made a new variable of
the absolute value of the fft, averaged over the whole time span
intervals). I modified it so that the spikes at the harmonics is removed
and interpolated a value in between them. However, all I am left now is the
magnitude and no phase info. so I am unable to transform that into the time
domain.

Is there a standard way of filtering out these frequencies which I am not
seeing? 

Please let me know. 

Thanks,

Vic

On Tue, 23 Apr 2013 20:21:37 -0500, Tic Tac wrote:

> Hello all,
> 
> I am quite new to the world of DSP and am required to perform some DSP
> for my student job using Matlab. I need to remove the noise ofthe
> utility frequencies (60 Hz + harmonics) from a number of signals. I have
> been reading many forums on this topic and tried many possible solutions
> but was unable to get it to work properly.
> 
> For instance, I tried using a notch filter and also tried a comb filter
> which worked for the first few harmonics but then started to
> overcompensate and would create spikes in the opposite direction.

Describe what you mean by "overcompensate".  A comb filter should fix all 
the harmonics, assuming that it's frequency is dead on the line 
frequency, but it'll also remove those frequency components from the 
signal itself.  If you can't have this, then a comb going from DC to 
light won't work.

> I also tried modifying the magnitude of the fft (I made a new variable
> of the absolute value of the fft, averaged over the whole time span
> intervals). I modified it so that the spikes at the harmonics is removed
> and interpolated a value in between them. However, all I am left now is
> the magnitude and no phase info. so I am unable to transform that into
> the time domain.

That's an interesting approach.  Possibly wrong, but interesting.  Try 
doing all that, then putting the phase back in.  I think you'll find that 
it has the right amplitude but the wrong phase at all the harmonics of 
the line frequency.

> Is there a standard way of filtering out these frequencies which I am
> not seeing?

The best thing to do is to rework your data acquisition so the noise 
isn't there in the first place.

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com

On 4/23/2013 8:21 PM, Tic Tac wrote:
> Hello all,
>

> I also tried modifying the magnitude of the fft (I made a new variable of
> the absolute value of the fft, averaged over the whole time span
> intervals). I modified it so that the spikes at the harmonics is removed
> and interpolated a value in between them.

How did you decide that missing data could be interpolated?
Does your signal have some special properties so the interpolation in 
the spectral domain would be valid ?

Vladimir Vassilevsky
DSP and Mixed Signal Designs
www.abvolt.com

If you are talking about an audio application, you will find that once your notches are narrow enough to not affect the perceptual audio quality in terms of frequency response,  the impulse response is very long, and you will hear obvious artifacts on transient inputs.  

One common technique is to design a high pass filter and then substitute Z^-n for Z. This gives you a wider notch than a simple sinc so the frequency alignment is less critical.  

Bob


Bob

Does anyone have much to say about modeling main noise as 60 Hz plus
harmonics?  Wouldn't other factors affect the mains signal?

On Tue, 23 Apr 2013 20:21:37 -0500, Tic Tac wrote:

> Hello all,
> 
> I am quite new to the world of DSP and am required to perform some DSP
> for my student job using Matlab. I need to remove the noise ofthe
> utility frequencies (60 Hz + harmonics) from a number of signals. I have
> been reading many forums on this topic and tried many possible solutions
> but was unable to get it to work properly.
> 
> For instance, I tried using a notch filter and also tried a comb filter
> which worked for the first few harmonics but then started to
> overcompensate and would create spikes in the opposite direction.
> 
> I also tried modifying the magnitude of the fft (I made a new variable
> of the absolute value of the fft, averaged over the whole time span
> intervals). I modified it so that the spikes at the harmonics is removed
> and interpolated a value in between them. However, all I am left now is
> the magnitude and no phase info. so I am unable to transform that into
> the time domain.
> 
> Is there a standard way of filtering out these frequencies which I am
> not seeing?
> 
> Please let me know.
> 
> Thanks,
> 
> Vic

Since this noise is repetitive, you can (in principle) perform a time-
synced averaging of your signal.  Assuming that your signal is 
asynchronous with the power line noise, eventually your averaging will 
only contain the average power-related noise signal.  This can be 
subtracted from the incoming signal.  This is the method used by the 
commercially sold "humbug" noise filter, and has been described in some 
papers (which unfortunately I can't supply the references at the moment).

HTH...

>> I also tried modifying the magnitude of the fft (I made a new variable
>> of the absolute value of the fft, averaged over the whole time span
>> intervals). I modified it so that the spikes at the harmonics is
removed
>> and interpolated a value in between them. However, all I am left now is
>> the magnitude and no phase info. so I am unable to transform that into
>> the time domain.
>
>That's an interesting approach.  Possibly wrong, but interesting.  Try 
>doing all that, then putting the phase back in.  I think you'll find that

>it has the right amplitude but the wrong phase at all the harmonics of 
>the line frequency.

If you are filtering in the frequency domain, don't you have to zero pad
the data coming in and use an overlap add method for reconstruction?

I have made a few versions of these in years gone by. Biggest problem is that if you make the notches narrow enough so that you don't hear notes disappearing into the notches, you end up with long impulse responses that can be heard on transients. 
The time averaging method mentioned above is another way to make a series of notch filters, but it does require that the mains fundamental period needs to be an exact integer multiple of the sampling period, so in practice you might need a pll locked to the mains. 

Bob

>Since this noise is repetitive, you can (in principle) perform a time-
>synced averaging of your signal.  Assuming that your signal is 
>asynchronous with the power line noise, eventually your averaging will 
>only contain the average power-related noise signal.  This can be 
>subtracted from the incoming signal.  This is the method used by the 
>commercially sold "humbug" noise filter, and has been described in some 
>papers (which unfortunately I can't supply the references at the moment).
>
>HTH...
>

So a tranfer function of:

   H(z) = (1/K) * SUM(z^(-kM)) for k = 0 to K-1

Where M is the delay.

The response looks like:

  |H(w)| = ABS( (1/K)*sin(KMw/2)/sin(Mw/2) )

Its a pseudo-sinc looking thing that repeats M times from 0 to 2*PI and the
width is controlled with K.  It can even be implemented recursively like a
CIC filter.  Thats so dope!

On 4/24/2013 3:16 AM, radams2000@gmail.com wrote:

> One common technique is to design a high pass filter and then
> substitute Z^-n for Z.

The same substitution could be done with basic lowpass as well. That 
produces regular comb type response with adjustable pass/notch.

It gets interesting if you insert frequency dependent delay into the 
basic lopass/hipass filter. The result is irregular comb.

Vladimir Vassilevsky
DSP and Mixed Signal Designs
www.abvolt.com