How well can discrete wavelet packet transform reduce noises that are similar to the input signal in the same frequency band?Started by 1 year ago●13 replies●latest reply 1 year ago●87 views
If I had 50Hz noise coming from power line, and signals in the same frequency range (EEG for example 0.1Hz to 100Hz). If my sampling frequency is 30kHz but I downsample my signal to 937Hz and use the discrete wavelet packet transform (with Daubechies wavelet) for denoising purposes.
My frequency content now is 0-937Hz in level 0 and divided to the power of 2 with respect to the level (as in the binary tree).
I have two questions:
- If my 50 Hz noise is high in amplitude comparison to my signal of interest, looking at node of "43.94Hz-59.59Hz" frequency band at level 6, If I perform thresholdolding the noise would still be present in the signal as it would have high amplitude, correct? While to remove it completely I would need to zero out all coefficients in that band.
- If my 50Hz is smaller in amplitude than my signal of interest, performing thresholding could theoretically remove the noise while keeping my signal of interest, correct? (theoretically because it depends on the threshold level and method)
Please correct me if I am wrong along the process. I am still trying to understand the benefits and drawbacks of wavelets.
I noticed that you use extremely high sampling frequency of 937kHz, while the EEG signal frequency content is below 100 Hz. Is it 937 Hz, is there a typo?
Practically, what you have said is true, you eliminate the coefficients in the "narrowest band" possible containing noise frequency spectrum, i.e. 50 Hz band.
For both of your questions, using narrowest band is essential. But you have to observe the magnitudes for the harmonics also. (100 Hz is not a deal for EEG)
Definitely a typo and I just fixed it, I apologize for the confusion.
And got it, harmonics are always a problem.
Thank you for the answer.
There exists no means to perfectly reduce in-band noise -- ever. If the noise is 1 Hz away, it will still affect the measurement of the signal due to the filter's pass-band shape that has side lobes and is not a brick wall. It's no different than trying to determine if a composite number is singular or based on the sum of multiple integers (i.e. is 5 really "5", or is it the sum of say 1 + 2 + 3 + -1). That's the "down" side of superposition.
I think that your best approach is in fact an adaptive filter, which I now see you posted a question about in a different posting.
It is not that hard to remove a single tone if the tone remains stable.
Also, the damage done to the remaining signal can be carefully controlled.
Since the interference is 50 Hz mains noise, there may also be 2nd, 3rd and other harmonics to deal with as well.
Is you application real time, or are you trying to cleanup a recording with post-processing?
Obviously the latter is easier, while real time is harder.
Anyway, another approach is to try using an oscillator as the reference for your 50 Hz.
You can do this with a bi-quad with poles on the unit circle at 50 Hz.
Instead of the bi-quad just being implemented as a stand alone oscillator, you can provide an input to it that will receive the remaining error signal after you subtract the oscillator output from the recording signal.
With some carefully controlled gains, the oscillator will eventually "sing" in sympathy with the power line noise to minimize the evidence of it in the recording.
The training time for the oscillator may be short, and the gain of the error input into the oscillator can be reduced to minimize the damage to the residual signal, effectively narrowing its bandwidth as a notch filter.
One additional trick is to have the oscillator be implemented as a complex device, so that the phase can roll internally if the power line signal is not exactly lined up with the accuracy of your digital implementation and sampling rate.
Thank you dgshaw6 for taking the time and answering both of my posts.
My application is intended for real-time.
The oscillator approach sounds interesting, is it like dither in a way or PLL? as in it oscillates to match the power line noise then subtracts it somehow?
I suffer tremendously from power line noise and its harmonic. I tried the DWT approach first but saw that the bigger the magnitude of the PLI the more I would lose from my signal of interest. My options were to use the DWT as a "notch filter" to zero out the subband PLI is in but it is not my intension as I would lose signals or threshold and noise is left(lost with the signal).
I think the "driven" oscillator approach will work well.
Do you have access to a copy of the power-line noise independently?
That could be used as a reference for the noise and be the second input to your adaptive filter.
Do you have an example of the signal with the interfering noise?
I would be happy to tinker with the oscillator technique and return a cleaned up copy as an example of what can be done.
Yes I do, actually all I have are power-line noises so far and nothing with meaningful signals. I will get in touch to send the data.
1) if power line noise is high, this might indicate a wiring/connection problem, quite usual it has to do with ground issues.
The closer the sensor setup comes to a clean differential signal propagation, the less you will see the power line noise.
2) And: reduce the incoming noise is always the best approach.
Especially, because suppressing 50 Hz will become a problem when pulse is in that area. This is not so unusual: my pulse at night is around 55..57, my daughter's quite often around 50Hz. You'll see this frequently with people practising sports.
3) Therefore I like your idea of wavelets. But you'll not get to a very good result if you think of a wavelet filter bank with "standard" wavelets.
Heart pulse waveform is very typical. In a very simplified (may be too much :-) view you can take it as a wavelet. If you'd take the challenge to do the maths to generate a "similarly" looking wavelet with all the necessary orthogonality requirements, this could lead you to a wavelet "processor" with much better response as you could expect with Daubechy or other standard wavelets.
Or by reducing the 50Hz line.
Thank you Bernhard, funny enough I was just saying I should just make my own wavelet and give it my name.
I quite liked the wavelet denosing approach since I am not only aiming to remove the powerline noise but other noises as well and I thought DWT would enable me to do so.
For now I think I will experiment with Daubechies and see how I can work around the shortcomings of it.
FFT is most effective, if you are able to evaluate results in the frequency domain. So is it with DWT.
If you can evaluate in DWT domain, it's most effective;
better than using it just as a filter bank.
However, after having read a lot of the mathematic backgrounds, I'd say it's challenging to find the mathematical formulas for a good wavelet (Matlab might have helpful modules).
Yes, I have been using matlab and they have an awesome toolbox for wavelets for some simulations before digging into implementation.
Thank you again!
I thought I will repeat what others have already stated:
1. Since you state that you are seeing only the line signal, most likely you have grounding or shielding problems. It is important to recognise that the EEG signal is of the order of microvolts, while the line voltage is tens of million times higher.
2. If the signal and noise occupy the same time band and same frequency band, and do not have some other distinguishing property, it is impossible to separate by any software processing.
3. Because the line voltage presence in recorded data can be very high, it is absolutely desirable to filter out the line voltage signal before recording. This is usually done by analog filters prior to recording. So the solution to your problem is most likely in checking the grounding/ shielding of your recording setup.
Noted, thank you!