Question regarding the validity of a SNR calculation

> could get an arbitrarily high "SNR" 
To put it more bluntly: there is nothing wrong about defining
a new figure of merit. But giving it the name of a established 
one would be confusing.

> Mean Opinion Scores
  Low distortion systems like PCM-codecs use S/N-plots to 
judge performance.
  For simple waveformcoders like CVSD S/N already failed: 
slope-overload would give poor S/N, but be listeners wouldn�t 
object. This was already known in the 60ies when work on 
vocoders started and a need for measuring their quality 
was emerging.
All the more complex coders since then are in the end judged 
by MOS. Which done properly is tedious, expensive, not too 
reliable either. And obviously impractial for daily work. 
  No cheap and simple way to judge how noise & distortion
affects the quality of speech has emerged in all these years. 

>> where var() stands for variance.
  The usual "signal" for testing S/N in a phone-channel
would be a sine 1KHz. A filter limiting the noise like C-message 
would be used. This S/N calculation is based on power.

  Very simple ( but murky ) statistical models of speech have been 
used in the past. For defining the PCM-codecs the longtime-average pdf
of speech which is supposed to be a gamma-distribution 
( similar: Laplacian ). But the short-term is Gauss.
  In the 60ies some people have used whitenoise-generators with
formfilters that give the shape of the longtime-average spectrum
of speech. For some applications that may be a more usefull
representation of speech then a sine. 

MfG  JRD

"Rafael Deliano" <Rafael_Deliano@t-online.de> skrev i en meddelelse 
news:45178F7A.E09422A@t-online.de...
>> could get an arbitrarily high "SNR"
> To put it more bluntly: there is nothing wrong about defining
> a new figure of merit. But giving it the name of a established
> one would be confusing.



I don't quite understand what you mean by a "new figure of merit".
SNR is defined as 10 multiplied by the 10-base logarithm of the ratio 
between signal
power and noise power. However, in the papers I have read the authors
always calculate the ratio as s/e where e is defined as e=s-s_hat where s is 
the original
clean speech and s_hat is the estimated clean speech. In my mind, it makes 
no sense to first calculate
the SNR of the input signal as s/n and then postulate that the "noise" 
component
in the output signal is the difference between the original speech signal 
and
the estimated speech. An algorithm, A, which operates more or less in a 
linear
way on an input signal s+n will output A(s)+A(n). Hence, SNR of output 
should
be measured as 10log10(var(A(s))/var(A(n))) and not 
10log10(var(A(s))/var(s-s_hat)).
In the latter case, the SNR ratio is not even comparable with the SNR of the 
input.

Hi John,

interesting question. I'm with you in your way of evaluating the
suppression of noise by an algorithm.

> However, in the papers I have read the authors
> always calculate the ratio as s/e where e is defined as e=s-s_hat where s is
> the original
> clean speech and s_hat is the estimated clean speech. In my mind, it makes
> no sense to first calculate
> the SNR of the input signal as s/n and then postulate that the "noise"
> component
> in the output signal is the difference between the original speech signal
> and
> the estimated speech.

Imagine that your algorithm does not only change the noise but also the
speech signal. I think, with your calculation you get the amount of
noise suppression by the algorithm. If you calculate it by the
difference between s and s_hat you schould obtain the noise suppression
plus any distortion of the speech. Your approach on the other hand does
only get the noise suppression, doesn't it *?*

Another thing I just thought about: How about noise which is not
gaussian, e.g. impulsive noise or deterministic interferences? I think
just calculating the variance doesn't work anymore for such a signal.

Regards

stereo

> Imagine that your algorithm does not only change the noise but also the
> speech signal. I think, with your calculation you get the amount of
> noise suppression by the algorithm.

The algorithm changes both the unknown noise component n and the unknown 
speech component s in
the known noisy input speech sequence x. And yes, the amount of noise 
suppression is _exactly_ what I want to measure
since I want to measure the performance of my _noise suppression_ algorithm.

>If you calculate it by the
> difference between s and s_hat you schould obtain the noise suppression
> plus any distortion of the speech.

How do you deduct that?


>Your approach on the other hand does
> only get the noise suppression, doesn't it *?*

Yes.


> Another thing I just thought about: How about noise which is not
> gaussian, e.g. impulsive noise or deterministic interferences?

My algorithm is designed for quasi-stationary noise sources. A deterministic
source like a sine would be suppressed no problem. An impulsive noise
source would probably be harder to suppress unless it has some cyclic
nature to it.


>I think
> just calculating the variance doesn't work anymore for such a signal.


I disagree. It depends on the context.



I am really grateful that you took time out to post a reply. It is very
inspiring to discuss ideas and problems in this forum.

Cheers...

Hi again,

> >If you calculate it by the
> > difference between s and s_hat you schould obtain the noise suppression
> > plus any distortion of the speech.
>
> How do you deduct that?

No math here, just intuitively: s_hat is the estimated clean speech
(and exactly what you output to a listener). As the output of the
algorithm it is built from the distorted version of the speech (since
the speech is not estimated error-free) and from the remaining noise,
i.e. the part of the noise that is not removed by the algorithm.
Therefore the difference between s and s_hat should include distortion
and noise.

I'm not postulating this as an expert or so, it is just what comes into
mind when analysing the problem. Corrections are very welcome.

> > Another thing I just thought about: How about noise which is not
> > gaussian, e.g. impulsive noise or deterministic interferences?
> My algorithm is designed for quasi-stationary noise sources. A deterministic
> source like a sine would be suppressed no problem. An impulsive noise
> source would probably be harder to suppress unless it has some cyclic
> nature to it.
> >I think just calculating the variance doesn't work anymore for such a signal.
> I disagree. It depends on the context.

I don't get this. If my noise is not gaussian-like, the variance of the
signal will not equal to the signal's power anymore, does it? Imagine a
(nearly) rectangular signal, the histogram will just have two
spikes...mean and variance are not applicable here in the way you do it
above.

Regards

stereo

stereo said the following on 25/09/2006 13:00:

>>> I think just calculating the variance doesn't work anymore for such a signal.
>> I disagree. It depends on the context.
> 
> I don't get this. If my noise is not gaussian-like, the variance of the
> signal will not equal to the signal's power anymore, does it? Imagine a
> (nearly) rectangular signal, the histogram will just have two
> spikes...mean and variance are not applicable here in the way you do it
> above.

Variance is defined as the second central moment; assuming zero mean 
this is:

     sigma^2 = int x^2 f(x) dx

integrated across all x.

However, this is also the definition of power.  So variance and power 
are always equal in the zero-mean case, I think.


-- 
Oli

Simple example: i want to measure
the distortion of a uLAW-PCM-Coder. I can do that
digital as a simulation on a sample by sample basis with a
sine generator:


sine --+----------------------------------+------- "signal" 
       |                                  |
       +--ulaw-coder--ulaw-decoder-----subtract--- "noise"
     
The ulaw-decoder outputs the distorted signal: signal plus noise.

To get the RMS-value of "signal" and "noise" i square and 
sum all samples and do a SQRT at the end.
How would you do the variance ? Sum of squared samples ?
If you say: standard deviation, which is SQRT(deviation)
we have done the same thing.

MfG  JRD

Rafael Deliano wrote:

> Simple example: i want to measure
> the distortion of a uLAW-PCM-Coder.

[...]

The biggest problem is that the difference between the input and the 
output is generally not the adequate measure for the distortion for the 
audio signal. It is valid only for the memoryless distortion at the 
small levels of -30dB or less.
The weighted difference in the spectral domain should be used.

  Vladimir Vassilevsky

DSP and Mixed Signal Design Consultant

http://www.abvolt.com

> It is valid only for the memoryless distortion ...
You are right, thats why i used the uLAW-coder/decoder ( no filters
in it assumed ) as an example. 
The "Denoiser" would look like that:

speech --+----------------+------- "signal" 
         |                |
 noise -add--Denoiser--subtract--- "noise"

One could assume speech and noise are fed in with 8kHz linear
PCM. But the delay in the "Denoiser" would make that
setup unusable.

MfG  JRD

> How would you do the variance ? Sum of squared samples ?
> If you say: standard deviation, which is SQRT(deviation)
> we have done the same thing.
  Slowly i am too getting at it: you square the RMS-"voltage" 
to get it to a "power", because you want the S/N expressed
as power 10log() not voltage 20log().
And on that way the variance creeps in.

MfG  JRD