Standard Deviation

I have a doubt with respect to the standard deviation of an N point
signal.

I have a book on statistics that says average deviation of an N point
signal
is the sum of the (difference between sample value and mean) divided by
N.
and that sum of the squares of the difference between sample value and
mean
divided by N gives the variance of the signal and the root of this
gives standard
deviation.

Why is this squaring required?

Sum of the squares of the deviation from  mean gives the power of the
current
sample deviation and when we add all these powers up, and divide by
total number
of samples, we get the variance, root of which gives standard
deviation, in
other words, we're taking the root of the average deviation "power" of
the signal
to get the deviation or fluctuation of the signal from the mean value,

How is this different from "adding all the (sample value - mean) values
and dividing
by the total number of samples?

Why can't we just add all the sample deviation amplitudes and divide by
N ?
Why should we find the sample deviation power and then divide by N and
take root ?
Isn't it the same thing?

Thanks a lot,

With Regards,

Joel

"Joel" <joelagnel@gmail.com> wrote in message 
news:1161671304.854671.253000@m7g2000cwm.googlegroups.com...
>I have a doubt with respect to the standard deviation of an N point
> signal.
>
> I have a book on statistics that says average deviation of an N point
> signal
> is the sum of the (difference between sample value and mean) divided by
> N.
> and that sum of the squares of the difference between sample value and
> mean
> divided by N gives the variance of the signal and the root of this
> gives standard
> deviation.
>
> Why is this squaring required?
>
> Sum of the squares of the deviation from  mean gives the power of the
> current
> sample deviation and when we add all these powers up, and divide by
> total number
> of samples, we get the variance, root of which gives standard
> deviation, in
> other words, we're taking the root of the average deviation "power" of
> the signal
> to get the deviation or fluctuation of the signal from the mean value,
>
> How is this different from "adding all the (sample value - mean) values
> and dividing
> by the total number of samples?
>
> Why can't we just add all the sample deviation amplitudes and divide by
> N ?
> Why should we find the sample deviation power and then divide by N and
> take root ?
> Isn't it the same thing?

No, it isn't.

One is the sum of the deviations which will be both positive and negative.
I'm not sure why that isn't equal to the mean.  And, if the mean is zero 
then it would be zero.

On the other hand, squaring the values gives all positive results and there 
will be a nonzero average.

Fred

Joel wrote:

> I have a doubt with respect to the standard deviation of an N point
> signal.

> I have a book on statistics that says average deviation of
 >  an N point signal is the sum of the (difference between sample
 > value and mean) divided by N.  and that sum of the squares of
 >  the difference between sample value and mean
> divided by N gives the variance of the signal and the root of this
> gives standard deviation.

> Why is this squaring required?

The sum of the deviations from the mean will be zero.

There are some statistics that use the average absolute value
of the deviation from the mean, but that is rare.
One reason for squaring is that the result is an analytical
function, where absolute value is not.

Median is much more common than absolute deviation, though
somewhat harder to calculate.

-- glen

Fred Marshall wrote:
> "Joel" <joelagnel@gmail.com> wrote in message 
> news:1161671304.854671.253000@m7g2000cwm.googlegroups.com...
> 
>>I have a doubt with respect to the standard deviation of an N point
>>signal.
>>
>>I have a book on statistics that says average deviation of an N point
>>signal
>>is the sum of the (difference between sample value and mean) divided by
>>N.
>>and that sum of the squares of the difference between sample value and
>>mean
>>divided by N gives the variance of the signal and the root of this
>>gives standard
>>deviation.
>>
>>Why is this squaring required?
>>
>>Sum of the squares of the deviation from  mean gives the power of the
>>current
>>sample deviation and when we add all these powers up, and divide by
>>total number
>>of samples, we get the variance, root of which gives standard
>>deviation, in
>>other words, we're taking the root of the average deviation "power" of
>>the signal
>>to get the deviation or fluctuation of the signal from the mean value,
>>
>>How is this different from "adding all the (sample value - mean) values
>>and dividing
>>by the total number of samples?
>>
>>Why can't we just add all the sample deviation amplitudes and divide by
>>N ?
>>Why should we find the sample deviation power and then divide by N and
>>take root ?
>>Isn't it the same thing?
> 
> 
> No, it isn't.
> 
> One is the sum of the deviations which will be both positive and negative.
> I'm not sure why that isn't equal to the mean.  And, if the mean is zero 
> then it would be zero.
> 
> On the other hand, squaring the values gives all positive results and there 
> will be a nonzero average.
> 
> Fred
> 
> 
> 
> 
And even when the 'average deviation' is defined to be the average of 
the absolute values of the differences, this does not give the same 
numerical result as taking the square root of the average of the squares 
of the differences.

Joel can check this with a short numerical example.


Regards,
John

Thanks a lot, that really helped.

One thing, the book im reading says when 2 noise signals enter an
electronic circuit, to get the resultant signal, we calculate the power
of the 2 signals, add them and then take root to get the amplitudes.

Why can't we just add the voltages of the 2 signals at any point of
time to get the resultant signal.

Regards,

Joel

and to find the standard deviation, as you, said, a simple addition
of the deviations from the mean will cancel them out so we go for
squaring, why can't we just take the absolute value of all the
deviation
and add them without squaring.

Thank you

Joel

Joel wrote:
> Thanks a lot, that really helped.
> 
> One thing, the book im reading says when 2 noise signals enter an
> electronic circuit, to get the resultant signal, we calculate the power
> of the 2 signals, add them and then take root to get the amplitudes.
> 
> Why can't we just add the voltages of the 2 signals at any point of
> time to get the resultant signal.
> 
> Regards,
> 
> Joel
> 
Joel,
if the noise sources are independent, there will be no particular phase 
relationship between the various frequency components.

Doing an addition would be useful only if all the components were in 
phase.  However, we can add the two signal POWERS and then convert the 
total to a voltage.

Joel wrote:
> and to find the standard deviation, as you, said, a simple addition
> of the deviations from the mean will cancel them out so we go for
> squaring, why can't we just take the absolute value of all the
> deviation and add them without squaring.

Hello Joel,

your question is a good one, and is not asked often enough. Take a step
back and look at what you are trying to do: you have some data, which
you think is centered around a mean value, with some additional errors.
One mathematical model for this situation is

x_k = m + e_k,   1 <= k <= N,

where x_k is your data, m is the mean value and e_k are random
variables that follow some distribution. Usually you assume the e_k
independent and identically distributed (the acronym is i.i.d.). From
the data x_k you want to estimate m (called the "location" of the
data), and you want to quantify how good the estimate is by determining
how wide the data is spread about the location (called the "scale" or
"dispersion" of the data).

The assumption that the e_k are i.i.d. Gaussian distributed,

e_k ~ N(0, s^2), and therefore
x_k ~ N(m, s^2),

with deviation s, is very common. In this case, there are optimal
procedures to estimate m and s from the data x_k. They are the
arithmetic mean and the (biased) sample deviation (SD), respectively.
They are optimal in the sense that they yield the most accurate
estimate given the size of the data set. Sometimes, instead of the
biased SD, the unbiased SD is computed. The two differ only by a
constant factor of sqrt(N/(N-1)).

However, what happens if the errors are _not_ iid Gaussian?

For example, what happens if the errors have larger tail probabilites?
One very important special case is the possibility of gross outliers.
If you consider a data set which is an instance of the Gaussian
process, except for one error (perhaps a measurement error), which on a
plot lies "far away" from the main bulk of the data - something *very*
unlikely if the data really were Gaussian. It is immediate from the
definition of the arithmetic mean and the SD that such an outlier has
unbounded influence: if the outlier goes towards +/-  infinity, so do
the estimates. For this reason, so called "robust" statistical methods
have been studied, which perform well in the Gaussian case, but where
the influence of outliers is bounded. Examples are the median as an
estimate for the mean, and the median absolute deviation from the
median (abbreviated MAD) for the deviation. They tolerate a large
percentage of outliers at the cost of efficiency.

Another case is where the distribution of the errors is known, and not
Gaussian. For example, if the errors are iid uniform. Again, one aims
to find the optimal (as above) estimator. It turns out that the
(scaled) standard absolute deviation (SAD) is the optimal estimator for
two-sided exponentially distributed errors (I think, I might be wrong
about the exact distribution). Such estimators are called M-estimators
for Maximum likelihood: they aim to be optimal (with respect to
maximizing the likelihood function). The SAD also has the property that
it is more robust against slight deviation from Gaussian errors than
the SD. This fact was known already known to some astronomer (whose
name I always forget), who pushed the SAD instead of the SD for
astronomical observations, where the error distributions are known to
deviate from Gaussian.

So, you see that your question is justified, and that knowing the
answer is fundamental to knowing how to correctly apply these
statistical methods.

Regards,
Andor

Joel wrote:

> I have a doubt with respect to the standard deviation of an N point
> signal.
> 
> I have a book on statistics that says average deviation of an N point
> signal
> is the sum of the (difference between sample value and mean) divided by
> N.
> and that sum of the squares of the difference between sample value and
> mean
> divided by N gives the variance of the signal and the root of this
> gives standard
> deviation.
> 
> Why is this squaring required?

Because it makes the math easy.  This is a good thing, because if you 
step away from the standard deviation as a measure of 'goodness' then 
you step away from solving problems analytically, and are left groping 
in the dark with simulations.
> 
> Sum of the squares of the deviation from  mean gives the power of the
> current
> sample deviation and when we add all these powers up, and divide by
> total number
> of samples, we get the variance, root of which gives standard
> deviation, in
> other words, we're taking the root of the average deviation "power" of
> the signal
> to get the deviation or fluctuation of the signal from the mean value,
> 
> How is this different from "adding all the (sample value - mean) values
> and dividing
> by the total number of samples?

1/N * sum over N of (sample value - mean) = (1/N sum over N of sample 
value) - mean = mean - mean = 0.  Always.  Identically.  Very boring.
> 
> Why can't we just add all the sample deviation amplitudes and divide by
> N ?
> Why should we find the sample deviation power and then divide by N and
> take root ?
> Isn't it the same thing?

I suggest you take some of your "why nots" and work them out on paper, 
to see what results you get.


-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

Joel wrote:
> Thanks a lot, that really helped.
> 
> One thing, the book im reading says when 2 noise signals enter an
> electronic circuit, to get the resultant signal, we calculate the power
> of the 2 signals, add them and then take root to get the amplitudes.

Not quite. To get the power of the composite, you add the powers of the 
components only if they are "orthogonal". Random noise signals, by their 
random nature, are orthogonal. So by the way, are sin(x) and cos(x), and 
sin(x) and sin(kx) where k ~= 1.

> Why can't we just add the voltages of the 2 signals at any point of
> time to get the resultant signal.

The sum of the voltages *is* the the voltage signal. We don't normally 
know much about noise beyond its statistical properties, and we deal 
with noise *power*. Power is proportional to the square of voltage. If 
you think about it, you'll see that that makes those puzzling squaring 
operations agree with what's real.

Let's take some examples. In both, we add two known (not just 
statistically characterized) signals and examine the result.

1. v1 = Vsin(wt); v2 = Vcos(wt).
    p1 = V^2*sin^2(wt)/R; p2 = V^2*cos^2(wt)/R
    p1 + p2 = V^2*[sin^2(wt) + cos^2(wt)]/R = V^2/R

Those are instantaneous powers, and we need to integrate over a complete 
cycle then divide by the period to get the average, but since our sum is 
constant, we needn't bother in this case.

Now consider v3 = v1 + v2 = sin(wt) + cos(wt) = sqrt(2)*sin(wt + pi/4). 
The instantaneous power is v3^2/R, and the average power is

  1        2*pi              1
--- * integral (v3^2 dt) * ---. Work it out to get -- tada! -- V^2/R
2*pi         0              R

The signals are orthogonal, so the sum of the powers is the power of the 
sum of the voltages.

2. v1 = Vsin(wt); v2 = Vsin(wt).
    p1 = V^2*sin^2(wt)/R; p2 = V^2*sin^2(wt)/R
    p1 + p2 = V^2*[sin^2(wt) + sin^2(wt)]/R = V^2/R, as before.

Now consider v3 = v1 + v2 = sin(wt) + sin(wt) = 2*sin(wt)

I leave you to show that the power of the combined signal is 2*v^2/R.

The signals are not orthogonal, so the sum of the powers is not the 
power of the sum of the voltages.

To summarize, the square of a composite signal is always proportional to 
its power. The sum of its component signal's powers is not always the 
composite signal's power. In a very direct way, that is the basis for 
the Fourier transform.

Jerry
-- 
        "The rights of the best of men are secured only as the
        rights of the vilest and most abhorrent are protected."
            - Chief Justice Charles Evans Hughes, 1927
���������������������������������������������������������������������