DSPRelated.com
Forums

Spectral subtraction

Started by jlavrenz October 21, 2008
We have data that is amplitude modulated on a specific frequency.  The
data/amplitude of the sin wave at 50Hz is slow varying in time.  There is
stationary background noise that is white to pink in nature.  Our matlab
simulation only shows improvement in SNR for a SNR value below about 15 and
more noticeably below 10 but after 15 it offers basically nothing.  We are
doing magnitude spectral subtraction where S=D-k*N.  S is our
corrected/cleaned signal, D is our noisy signal and N is our noise estimate
and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR.  We estimate
SNR by averaging D/N but I've even used my actual SNR from the signal I
generate in my simulation and see little improvement?  k for large SNR is
basically 1/(2*SNR) but we use the full k shown above.  We thought we would
see better results up to about an SNR of even 50.  Even if in the
simulation I make perfect ideal white noise with the same magnitude (with
random phase) across the complete spectrum and try and subtract it off
(i.e. our noise estimate is no longer an estimate) we still see no
improvement above about an SNR of 15?  at an SNR of 15 k should be about
0.03 so I suppose we should really expect only a 3% reduction in noise at
an SNR of 15 which isn't much and this is about what we see?  Are there
other algorithms that would be more effective at higher SNR (up to say an
SNR of 50)?  Are we estimating SNR incorrectly, should it be (D-N)/N?  We
have a very good estimate of N as the noise is very stationary.  Our N
isn't specifically the noise at 50Hz though it is the noise near 50Hz
(45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are
almost the same so it seems like a good way to estimate the noise at 50Hz. 
We are also wondering if our k is correct?  We've seen many version of k in
the literature.  I've tried the wiener freq domain implementation of a bin
gain of 1-1/SNR but again no improvement above 15 or so.



On 21 Okt, 21:56, "jlavrenz" <jim.lavr...@emerson.com> wrote:
> We have data that is amplitude modulated on a specific frequency. &#4294967295;The > data/amplitude of the sin wave at 50Hz is slow varying in time. &#4294967295;There is > stationary background noise that is white to pink in nature. &#4294967295;Our matlab > simulation only shows improvement in SNR for a SNR value below about 15 and > more noticeably below 10 but after 15 it offers basically nothing. &#4294967295;We are > doing magnitude spectral subtraction where S=D-k*N. &#4294967295;S is our > corrected/cleaned signal, D is our noisy signal and N is our noise estimate > and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR. &#4294967295;We estimate > SNR by averaging D/N but I've even used my actual SNR from the signal I > generate in my simulation and see little improvement? &#4294967295;k for large SNR is > basically 1/(2*SNR) but we use the full k shown above. &#4294967295;We thought we would > see better results up to about an SNR of even 50. &#4294967295;Even if in the > simulation I make perfect ideal white noise with the same magnitude (with > random phase) across the complete spectrum and try and subtract it off > (i.e. our noise estimate is no longer an estimate) we still see no > improvement above about an SNR of 15? &#4294967295;at an SNR of 15 k should be about > 0.03 so I suppose we should really expect only a 3% reduction in noise at > an SNR of 15 which isn't much and this is about what we see? &#4294967295;Are there > other algorithms that would be more effective at higher SNR (up to say an > SNR of 50)? &#4294967295;Are we estimating SNR incorrectly, should it be (D-N)/N? &#4294967295;We > have a very good estimate of N as the noise is very stationary. &#4294967295;Our N > isn't specifically the noise at 50Hz though it is the noise near 50Hz > (45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are > almost the same so it seems like a good way to estimate the noise at 50Hz. > We are also wondering if our k is correct? &#4294967295;We've seen many version of k in > the literature. &#4294967295;I've tried the wiener freq domain implementation of a bin > gain of 1-1/SNR but again no improvement above 15 or so.
I don't understand why spectral subtraction would work at all. Consider the case of noise only, viewed in spectrum domain: W[k] = DFT{w[n]} [1] where w[n] is noise. The coefficient W[k] has magnitude |W[k]| and random unknown phase p: W[k] = |W[k]| exp(j*2*pi*p) [2] Then try to subtract a number A = |W[k]| from [2]: V[k] = |W[k]| exp(j*2*pi*p) - |W[k]| [3] = |W[k]| (exp(j*2*pi*p) - 1 ) [4] = |W[k]| (exp(j*2*pi*p) - exp(j*pi)) [5] So the net effect of all this is random, depending on the true phase p. One can study the phase terms in eq. [5] (I wouldn't be surprised if I've made some mistakes in the derivations, but you get the big picture) and see that for some true phases p one sees a net reduction of the noise term, while for others one sees increases. So basically, spectrum subtraction is a random process which will give random results. Which more or less explains why you don't see much consistent improvement. Apart from a whitening filter (make the pink noise white) there seems to be little room for improvment unless you can bring in signal characteristics. The Wiener filter is an obvious candidate to try if you can find the signal characteristics. Rune
On Oct 21, 4:27&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 21 Okt, 21:56, "jlavrenz" <jim.lavr...@emerson.com> wrote: > > > > > > > We have data that is amplitude modulated on a specific frequency. &#4294967295;The > > data/amplitude of the sin wave at 50Hz is slow varying in time. &#4294967295;There is > > stationary background noise that is white to pink in nature. &#4294967295;Our matlab > > simulation only shows improvement in SNR for a SNR value below about 15 and > > more noticeably below 10 but after 15 it offers basically nothing. &#4294967295;We are > > doing magnitude spectral subtraction where S=D-k*N. &#4294967295;S is our > > corrected/cleaned signal, D is our noisy signal and N is our noise estimate > > and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR. &#4294967295;We estimate > > SNR by averaging D/N but I've even used my actual SNR from the signal I > > generate in my simulation and see little improvement? &#4294967295;k for large SNR is > > basically 1/(2*SNR) but we use the full k shown above. &#4294967295;We thought we would > > see better results up to about an SNR of even 50. &#4294967295;Even if in the > > simulation I make perfect ideal white noise with the same magnitude (with > > random phase) across the complete spectrum and try and subtract it off > > (i.e. our noise estimate is no longer an estimate) we still see no > > improvement above about an SNR of 15? &#4294967295;at an SNR of 15 k should be about > > 0.03 so I suppose we should really expect only a 3% reduction in noise at > > an SNR of 15 which isn't much and this is about what we see? &#4294967295;Are there > > other algorithms that would be more effective at higher SNR (up to say an > > SNR of 50)? &#4294967295;Are we estimating SNR incorrectly, should it be (D-N)/N? &#4294967295;We > > have a very good estimate of N as the noise is very stationary. &#4294967295;Our N > > isn't specifically the noise at 50Hz though it is the noise near 50Hz > > (45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are > > almost the same so it seems like a good way to estimate the noise at 50Hz. > > We are also wondering if our k is correct? &#4294967295;We've seen many version of k in > > the literature. &#4294967295;I've tried the wiener freq domain implementation of a bin > > gain of 1-1/SNR but again no improvement above 15 or so. > > I don't understand why spectral subtraction would work at all. > > Consider the case of noise only, viewed in spectrum domain: > > W[k] = DFT{w[n]} &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [1] > > where w[n] is noise. The coefficient W[k] has magnitude |W[k]| and > random unknown phase p: > > W[k] = |W[k]| exp(j*2*pi*p) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[2] > > Then try to subtract a number A = |W[k]| from [2]: > > V[k] = |W[k]| exp(j*2*pi*p) - |W[k]| &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [3] > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - 1 ) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [4] > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - exp(j*pi)) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[5] > > So the net effect of all this is random, depending on the true > phase p. One can study the phase terms in eq. [5] (I wouldn't be > surprised if I've made some mistakes in the derivations, but > you get the big picture) and see that for some true phases p > one sees a net reduction of the noise term, while for others > one sees increases. > > So basically, spectrum subtraction is a random process which > will give random results. Which more or less explains why > you don't see much consistent improvement. > > Apart from a whitening filter (make the pink noise white) > there seems to be little room for improvment unless you > can bring in signal characteristics. The Wiener filter > is an obvious candidate to try if you can find the > signal characteristics. > > Rune- Hide quoted text - > > - Show quoted text -
Rune, For magnutude spectral subtraction, you are supposed to be subtracting magnitude from magnitude (potentially with negative results reset to 0.0) and recombining with the original phase term. Dirk
On 21 Okt, 23:33, dbell <bellda2...@cox.net> wrote:
> On Oct 21, 4:27&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote: > > > > > > > On 21 Okt, 21:56, "jlavrenz" <jim.lavr...@emerson.com> wrote: > > > > We have data that is amplitude modulated on a specific frequency. &#4294967295;The > > > data/amplitude of the sin wave at 50Hz is slow varying in time. &#4294967295;There is > > > stationary background noise that is white to pink in nature. &#4294967295;Our matlab > > > simulation only shows improvement in SNR for a SNR value below about 15 and > > > more noticeably below 10 but after 15 it offers basically nothing. &#4294967295;We are > > > doing magnitude spectral subtraction where S=D-k*N. &#4294967295;S is our > > > corrected/cleaned signal, D is our noisy signal and N is our noise estimate > > > and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR. &#4294967295;We estimate > > > SNR by averaging D/N but I've even used my actual SNR from the signal I > > > generate in my simulation and see little improvement? &#4294967295;k for large SNR is > > > basically 1/(2*SNR) but we use the full k shown above. &#4294967295;We thought we would > > > see better results up to about an SNR of even 50. &#4294967295;Even if in the > > > simulation I make perfect ideal white noise with the same magnitude (with > > > random phase) across the complete spectrum and try and subtract it off > > > (i.e. our noise estimate is no longer an estimate) we still see no > > > improvement above about an SNR of 15? &#4294967295;at an SNR of 15 k should be about > > > 0.03 so I suppose we should really expect only a 3% reduction in noise at > > > an SNR of 15 which isn't much and this is about what we see? &#4294967295;Are there > > > other algorithms that would be more effective at higher SNR (up to say an > > > SNR of 50)? &#4294967295;Are we estimating SNR incorrectly, should it be (D-N)/N? &#4294967295;We > > > have a very good estimate of N as the noise is very stationary. &#4294967295;Our N > > > isn't specifically the noise at 50Hz though it is the noise near 50Hz > > > (45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are > > > almost the same so it seems like a good way to estimate the noise at 50Hz. > > > We are also wondering if our k is correct? &#4294967295;We've seen many version of k in > > > the literature. &#4294967295;I've tried the wiener freq domain implementation of a bin > > > gain of 1-1/SNR but again no improvement above 15 or so. > > > I don't understand why spectral subtraction would work at all. > > > Consider the case of noise only, viewed in spectrum domain: > > > W[k] = DFT{w[n]} &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [1] > > > where w[n] is noise. The coefficient W[k] has magnitude |W[k]| and > > random unknown phase p: > > > W[k] = |W[k]| exp(j*2*pi*p) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[2] > > > Then try to subtract a number A = |W[k]| from [2]: > > > V[k] = |W[k]| exp(j*2*pi*p) - |W[k]| &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [3] > > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - 1 ) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [4] > > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - exp(j*pi)) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[5] > > > So the net effect of all this is random, depending on the true > > phase p. One can study the phase terms in eq. [5] (I wouldn't be > > surprised if I've made some mistakes in the derivations, but > > you get the big picture) and see that for some true phases p > > one sees a net reduction of the noise term, while for others > > one sees increases. > > > So basically, spectrum subtraction is a random process which > > will give random results. Which more or less explains why > > you don't see much consistent improvement. > > > Apart from a whitening filter (make the pink noise white) > > there seems to be little room for improvment unless you > > can bring in signal characteristics. The Wiener filter > > is an obvious candidate to try if you can find the > > signal characteristics. > > > Rune- Hide quoted text - > > > - Show quoted text - > > Rune, > > For magnutude spectral subtraction, you are supposed to be subtracting > magnitude from magnitude (potentially with negative results reset to > 0.0) and recombining with the original phase term.
I'll take your word for it; I've never seen a definition, let alone an analysis, of spectral subtraction. Assuming you are right - what would be the point? What can be gained by spectral subtraction that couldn't be gained a lot easier by filtering? To succeed with noise reduction by spectral subtraction you would need quite a few steps and get them right: - Transform to polar form W[k] = |W[k]| exp(j pi phi) - Extract the magnitude term - Store the phase term - Find the term A to be subtracted, presumably A ~ |W[k]| - Perform the subtraction - Recombine result with the phase term - Transform to carthesian form Elaborate, and lots of opportunities to screw things up. With the usual filter one just multiplies with a factor |H[k]| < 1 and there you are. Simple to implement, works on either polar or carthesian form, robust. If somebody really has done all that work to come up with a good estimate for the noise statistics (lab work and measurements are usually way more time-consuming and expensive than data analysis) one ought to make sure those efforts pay off. Depending on the context of the analysis I would squeeze the Wiener filter for all it's worth, turning it inside-out and upside-down to see if I could make it utilize those noise measurements and statistics. Rune
On Oct 22, 3:27&#4294967295;am, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 21 Okt, 23:33, dbell <bellda2...@cox.net> wrote: > > > > > > > On Oct 21, 4:27&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote: > > > > On 21 Okt, 21:56, "jlavrenz" <jim.lavr...@emerson.com> wrote: > > > > > We have data that is amplitude modulated on a specific frequency. &#4294967295;The > > > > data/amplitude of the sin wave at 50Hz is slow varying in time. &#4294967295;There is > > > > stationary background noise that is white to pink in nature. &#4294967295;Our matlab > > > > simulation only shows improvement in SNR for a SNR value below about 15 and > > > > more noticeably below 10 but after 15 it offers basically nothing. &#4294967295;We are > > > > doing magnitude spectral subtraction where S=D-k*N. &#4294967295;S is our > > > > corrected/cleaned signal, D is our noisy signal and N is our noise estimate > > > > and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR. &#4294967295;We estimate > > > > SNR by averaging D/N but I've even used my actual SNR from the signal I > > > > generate in my simulation and see little improvement? &#4294967295;k for large SNR is > > > > basically 1/(2*SNR) but we use the full k shown above. &#4294967295;We thought we would > > > > see better results up to about an SNR of even 50. &#4294967295;Even if in the > > > > simulation I make perfect ideal white noise with the same magnitude (with > > > > random phase) across the complete spectrum and try and subtract it off > > > > (i.e. our noise estimate is no longer an estimate) we still see no > > > > improvement above about an SNR of 15? &#4294967295;at an SNR of 15 k should be about > > > > 0.03 so I suppose we should really expect only a 3% reduction in noise at > > > > an SNR of 15 which isn't much and this is about what we see? &#4294967295;Are there > > > > other algorithms that would be more effective at higher SNR (up to say an > > > > SNR of 50)? &#4294967295;Are we estimating SNR incorrectly, should it be (D-N)/N? &#4294967295;We > > > > have a very good estimate of N as the noise is very stationary. &#4294967295;Our N > > > > isn't specifically the noise at 50Hz though it is the noise near 50Hz > > > > (45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are > > > > almost the same so it seems like a good way to estimate the noise at 50Hz. > > > > We are also wondering if our k is correct? &#4294967295;We've seen many version of k in > > > > the literature. &#4294967295;I've tried the wiener freq domain implementation of a bin > > > > gain of 1-1/SNR but again no improvement above 15 or so. > > > > I don't understand why spectral subtraction would work at all. > > > > Consider the case of noise only, viewed in spectrum domain: > > > > W[k] = DFT{w[n]} &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [1] > > > > where w[n] is noise. The coefficient W[k] has magnitude |W[k]| and > > > random unknown phase p: > > > > W[k] = |W[k]| exp(j*2*pi*p) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[2] > > > > Then try to subtract a number A = |W[k]| from [2]: > > > > V[k] = |W[k]| exp(j*2*pi*p) - |W[k]| &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [3] > > > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - 1 ) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [4] > > > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - exp(j*pi)) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[5] > > > > So the net effect of all this is random, depending on the true > > > phase p. One can study the phase terms in eq. [5] (I wouldn't be > > > surprised if I've made some mistakes in the derivations, but > > > you get the big picture) and see that for some true phases p > > > one sees a net reduction of the noise term, while for others > > > one sees increases. > > > > So basically, spectrum subtraction is a random process which > > > will give random results. Which more or less explains why > > > you don't see much consistent improvement. > > > > Apart from a whitening filter (make the pink noise white) > > > there seems to be little room for improvment unless you > > > can bring in signal characteristics. The Wiener filter > > > is an obvious candidate to try if you can find the > > > signal characteristics. > > > > Rune- Hide quoted text - > > > > - Show quoted text - > > > Rune, > > > For magnutude spectral subtraction, you are supposed to be subtracting > > magnitude from magnitude (potentially with negative results reset to > > 0.0) and recombining with the original phase term. > > I'll &#4294967295;take your word for it; I've never seen a definition, let alone > an > analysis, of spectral subtraction. > > Assuming you are right - what would be the point? What can be gained > by > spectral subtraction that couldn't be gained a lot easier by > filtering? > > To succeed with noise reduction by spectral subtraction you would need > quite a few steps and get them right: > > - Transform to polar form W[k] = |W[k]| exp(j pi phi) > - Extract the magnitude term > - Store the phase term > - Find the term A to be subtracted, presumably A ~ |W[k]| > - Perform the subtraction > - Recombine result with the phase term > - Transform to carthesian form > > Elaborate, and lots of opportunities to screw things up. > With the usual filter one just multiplies with a factor > |H[k]| < 1 and there you are. Simple to implement, works > on either polar or carthesian form, robust. > > If somebody really has done all that work to come up with > a good estimate for the noise statistics (lab work and > measurements are usually way more time-consuming and > expensive than data analysis) one ought to make sure > those efforts pay off. > > Depending on the context of the analysis I would squeeze > the Wiener filter for all it's worth, turning it inside-out > and upside-down to see if I could make it utilize those > noise measurements and statistics. > > Rune- Hide quoted text - > > - Show quoted text -
The method was published by Steven Boll in the late 1970's IIRC. Lots of references to original and modifications of it on Yahoo! search "spectral subtraction". It has it's problems, which you can find in the references. It is really not that hard to implement. Has some use for wideband noise reduction. Dirk
On Oct 21, 3:27&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 21 Okt, 21:56, "jlavrenz" <jim.lavr...@emerson.com> wrote: > > > > > > > We have data that is amplitude modulated on a specific frequency. &#4294967295;The > > data/amplitude of the sin wave at 50Hz is slow varying in time. &#4294967295;There is > > stationary background noise that is white to pink in nature. &#4294967295;Our matlab > > simulation only shows improvement in SNR for a SNR value below about 15 and > > more noticeably below 10 but after 15 it offers basically nothing. &#4294967295;We are > > doing magnitude spectral subtraction where S=D-k*N. &#4294967295;S is our > > corrected/cleaned signal, D is our noisy signal and N is our noise estimate > > and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR. &#4294967295;We estimate > > SNR by averaging D/N but I've even used my actual SNR from the signal I > > generate in my simulation and see little improvement? &#4294967295;k for large SNR is > > basically 1/(2*SNR) but we use the full k shown above. &#4294967295;We thought we would > > see better results up to about an SNR of even 50. &#4294967295;Even if in the > > simulation I make perfect ideal white noise with the same magnitude (with > > random phase) across the complete spectrum and try and subtract it off > > (i.e. our noise estimate is no longer an estimate) we still see no > > improvement above about an SNR of 15? &#4294967295;at an SNR of 15 k should be about > > 0.03 so I suppose we should really expect only a 3% reduction in noise at > > an SNR of 15 which isn't much and this is about what we see? &#4294967295;Are there > > other algorithms that would be more effective at higher SNR (up to say an > > SNR of 50)? &#4294967295;Are we estimating SNR incorrectly, should it be (D-N)/N? &#4294967295;We > > have a very good estimate of N as the noise is very stationary. &#4294967295;Our N > > isn't specifically the noise at 50Hz though it is the noise near 50Hz > > (45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are > > almost the same so it seems like a good way to estimate the noise at 50Hz. > > We are also wondering if our k is correct? &#4294967295;We've seen many version of k in > > the literature. &#4294967295;I've tried the wiener freq domain implementation of a bin > > gain of 1-1/SNR but again no improvement above 15 or so. > > I don't understand why spectral subtraction would work at all. > > Consider the case of noise only, viewed in spectrum domain: > > W[k] = DFT{w[n]} &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [1] > > where w[n] is noise. The coefficient W[k] has magnitude |W[k]| and > random unknown phase p: > > W[k] = |W[k]| exp(j*2*pi*p) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[2] > > Then try to subtract a number A = |W[k]| from [2]: > > V[k] = |W[k]| exp(j*2*pi*p) - |W[k]| &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [3] > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - 1 ) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [4] > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - exp(j*pi)) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[5] > > So the net effect of all this is random, depending on the true > phase p. One can study the phase terms in eq. [5] (I wouldn't be > surprised if I've made some mistakes in the derivations, but > you get the big picture) and see that for some true phases p > one sees a net reduction of the noise term, while for others > one sees increases. > > So basically, spectrum subtraction is a random process which > will give random results. Which more or less explains why > you don't see much consistent improvement. > > Apart from a whitening filter (make the pink noise white) > there seems to be little room for improvment unless you > can bring in signal characteristics. The Wiener filter > is an obvious candidate to try if you can find the > signal characteristics. > > Rune- Hide quoted text - > > - Show quoted text -
Rune, Hvordan ha det du i dag? I have something to send you but do not have your email address. Maurice Givens
On 22 Okt, 20:39, maury <maury...@core.com> wrote:
> On Oct 21, 3:27&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote: > > > > > > > On 21 Okt, 21:56, "jlavrenz" <jim.lavr...@emerson.com> wrote: > > > > We have data that is amplitude modulated on a specific frequency. &#4294967295;The > > > data/amplitude of the sin wave at 50Hz is slow varying in time. &#4294967295;There is > > > stationary background noise that is white to pink in nature. &#4294967295;Our matlab > > > simulation only shows improvement in SNR for a SNR value below about 15 and > > > more noticeably below 10 but after 15 it offers basically nothing. &#4294967295;We are > > > doing magnitude spectral subtraction where S=D-k*N. &#4294967295;S is our > > > corrected/cleaned signal, D is our noisy signal and N is our noise estimate > > > and k we calculated to be k = (sqrt(1 + 1 /(SNR)^2)-1)*SNR. &#4294967295;We estimate > > > SNR by averaging D/N but I've even used my actual SNR from the signal I > > > generate in my simulation and see little improvement? &#4294967295;k for large SNR is > > > basically 1/(2*SNR) but we use the full k shown above. &#4294967295;We thought we would > > > see better results up to about an SNR of even 50. &#4294967295;Even if in the > > > simulation I make perfect ideal white noise with the same magnitude (with > > > random phase) across the complete spectrum and try and subtract it off > > > (i.e. our noise estimate is no longer an estimate) we still see no > > > improvement above about an SNR of 15? &#4294967295;at an SNR of 15 k should be about > > > 0.03 so I suppose we should really expect only a 3% reduction in noise at > > > an SNR of 15 which isn't much and this is about what we see? &#4294967295;Are there > > > other algorithms that would be more effective at higher SNR (up to say an > > > SNR of 50)? &#4294967295;Are we estimating SNR incorrectly, should it be (D-N)/N? &#4294967295;We > > > have a very good estimate of N as the noise is very stationary. &#4294967295;Our N > > > isn't specifically the noise at 50Hz though it is the noise near 50Hz > > > (45-49 and 51-55 Hz) but when we do an FFT of the data the magnitudes are > > > almost the same so it seems like a good way to estimate the noise at 50Hz. > > > We are also wondering if our k is correct? &#4294967295;We've seen many version of k in > > > the literature. &#4294967295;I've tried the wiener freq domain implementation of a bin > > > gain of 1-1/SNR but again no improvement above 15 or so. > > > I don't understand why spectral subtraction would work at all. > > > Consider the case of noise only, viewed in spectrum domain: > > > W[k] = DFT{w[n]} &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [1] > > > where w[n] is noise. The coefficient W[k] has magnitude |W[k]| and > > random unknown phase p: > > > W[k] = |W[k]| exp(j*2*pi*p) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[2] > > > Then try to subtract a number A = |W[k]| from [2]: > > > V[k] = |W[k]| exp(j*2*pi*p) - |W[k]| &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [3] > > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - 1 ) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; [4] > > &#4294967295; &#4294967295; &#4294967295;= |W[k]| (exp(j*2*pi*p) - exp(j*pi)) &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;[5] > > > So the net effect of all this is random, depending on the true > > phase p. One can study the phase terms in eq. [5] (I wouldn't be > > surprised if I've made some mistakes in the derivations, but > > you get the big picture) and see that for some true phases p > > one sees a net reduction of the noise term, while for others > > one sees increases. > > > So basically, spectrum subtraction is a random process which > > will give random results. Which more or less explains why > > you don't see much consistent improvement. > > > Apart from a whitening filter (make the pink noise white) > > there seems to be little room for improvment unless you > > can bring in signal characteristics. The Wiener filter > > is an obvious candidate to try if you can find the > > signal characteristics. > > > Rune- Hide quoted text - > > > - Show quoted text - > > Rune, > Hvordan ha det du i dag?
S&#4294967295; som s&#4294967295;, som du kanskje har skj&#4294967295;nt fra andre tr&#4294967295;der... ;)
> I have something to send you but do not have your email address.
I'll send an email to the address in your post. Rune