Forums

Is White Noise Necessarily Zero-Mean?

Started by Randy Yates February 15, 2006
On Wed, 15 Feb 2006 14:35:28 -0800, "Fred Marshall"
<fmarshallx@remove_the_x.acm.org> wrote:
> >"Randy Yates" <yates@ieee.org> wrote in message >news:m3bqx82t1e.fsf@ieee.org... >> Just Cocky <just@cocky.com> writes: >>> [...] >>> Huh? Mean of a sample? What's the point of speaking of the mean of one >>> known value? >> >> I think he meant a section of time from a realization of a random process. >> >> I've seen the term "sample waveform" used in Proakis and I don't like it >> either. > >Right and point taken. I suppose it leaks over from statistics where a >"sample" is what we'd here call a "sequence" or "vector" or ..... a set of >samples taken over a temporal epoch. >
Ok, got it. For some reason, sample has a different "feel" when thinking digital electronics...
Just Cocky wrote:
> On Wed, 15 Feb 2006 12:41:58 -0800, "Fred Marshall" > <fmarshallx@remove_the_x.acm.org> wrote: > >>"Jani Huhtanen" <jani.huhtanen@kolumbus.fi> wrote in message >>news:dsvpsq$l2f$1@phys-news4.kolumbus.fi... >> >>>Randy Yates wrote: >>> >>> >>>>Let Z(t) be a white-noise (stationary) random process. >>>>Can we conclude that Z(t) has zero mean? >>>> >>>>My thought is: yes. The intuitive reason that comes to >>>>mind (and it may be wrong!) is this: If Z(t) has a >>>>non-zero mean, then there would be some amount of >>>>correlation between samples due to the means. Thus >>>>the autocorrelation would be a delta function. >>> >>>Yes. Whit noise is necessarily zero-mean. This follows from the definition >>>of white (i.e. covariance matrix is identity matrix). >>> >>>Let x be a random vector. Covariance matrix of a random vector is Cx = >>>E{x*x'}. Mean of the random vector is m = E{x}. Let y be a white random >>>vector with zero mean so that y = x + m; >>> >>>E{x*x'} = E{(y+m)*(y'+m')} = E{y*y'}+E{y*m'}+E{m*y'}+E{m*m'} = I + E{y}*m' >>>+ >>>m*E{y'} + m*m' = I + m*m' != I => random vector is not white if the mean >>>is >>>not zero. >>> >> >>Just be careful to note that a *sample* of such white noise doesn't >>necessarily have zero mean. >> > > > Huh? Mean of a sample? What's the point of speaking of the mean of one > known value?
Sample in the statistical sense. Read "subset". Jerry -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;
Fred Marshall wrote:

> > "Jani Huhtanen" <jani.huhtanen@kolumbus.fi> wrote in message > news:dsvpsq$l2f$1@phys-news4.kolumbus.fi... >> Randy Yates wrote: >> >>> Let Z(t) be a white-noise (stationary) random process. >>> Can we conclude that Z(t) has zero mean? >>> >>> My thought is: yes. The intuitive reason that comes to >>> mind (and it may be wrong!) is this: If Z(t) has a >>> non-zero mean, then there would be some amount of >>> correlation between samples due to the means. Thus >>> the autocorrelation would be a delta function. >> >> Yes. Whit noise is necessarily zero-mean. This follows from the >> definition of white (i.e. covariance matrix is identity matrix). >> >> Let x be a random vector. Covariance matrix of a random vector is Cx = >> E{x*x'}. Mean of the random vector is m = E{x}. Let y be a white random >> vector with zero mean so that y = x + m; >> >> E{x*x'} = E{(y+m)*(y'+m')} = E{y*y'}+E{y*m'}+E{m*y'}+E{m*m'} = I + >> E{y}*m' + >> m*E{y'} + m*m' = I + m*m' != I => random vector is not white if the mean >> is >> not zero. >> >> >> -- >> Jani Huhtanen >> Tampere University of Technology, Pori > > Just be careful to note that a *sample* of such white noise doesn't > necessarily have zero mean. > > Start with a white noise generator (with zero mean implied / assured). > Grab a sample over some finite temporal epoch such that the mean *is* > zero. It should be easy enough to do this by generating the noise and > computing the mean from the beginning, for some time, and then continuing > to compute the mean of the entire record until the mean is zero - then > stop grabbing data. > Now, cut the data into two equal halves in time. > There is some large probability that each half will not each have zero > mean. Of course the sum of the means of the two halves will be zero > because that's what you constructed in the grab. > > Fred
There is a difference between mean and average. If you calculate something from the data then you're just estimating some parameter. In this case average is used as an estimate for the mean. Secondly, if you generate white noise (x) and calculate the average (avg_n) repeatedly after every sample generated, you should notice that the chance of the average being larger than 0 is equal to of it being smaller than 0 (i.e. P(avg_n>0) = 0.5). (avg_n = sum_{i=1}^n x[n], where x is a sequence of random samples.) This follows from the central limit theorem. (On a second thought, this was probably just what you were trying to say) -- Jani Huhtanen Tampere University of Technology, Pori
Jani Huhtanen wrote:

> Fred Marshall wrote: > >> >> "Jani Huhtanen" <jani.huhtanen@kolumbus.fi> wrote in message >> news:dsvpsq$l2f$1@phys-news4.kolumbus.fi... >>> Randy Yates wrote: >>> >>>> Let Z(t) be a white-noise (stationary) random process. >>>> Can we conclude that Z(t) has zero mean? >>>> >>>> My thought is: yes. The intuitive reason that comes to >>>> mind (and it may be wrong!) is this: If Z(t) has a >>>> non-zero mean, then there would be some amount of >>>> correlation between samples due to the means. Thus >>>> the autocorrelation would be a delta function. >>> >>> Yes. Whit noise is necessarily zero-mean. This follows from the >>> definition of white (i.e. covariance matrix is identity matrix). >>> >>> Let x be a random vector. Covariance matrix of a random vector is Cx = >>> E{x*x'}. Mean of the random vector is m = E{x}. Let y be a white random >>> vector with zero mean so that y = x + m; >>> >>> E{x*x'} = E{(y+m)*(y'+m')} = E{y*y'}+E{y*m'}+E{m*y'}+E{m*m'} = I + >>> E{y}*m' + >>> m*E{y'} + m*m' = I + m*m' != I => random vector is not white if the mean >>> is >>> not zero. >>> >>> >>> -- >>> Jani Huhtanen >>> Tampere University of Technology, Pori >> >> Just be careful to note that a *sample* of such white noise doesn't >> necessarily have zero mean. >> >> Start with a white noise generator (with zero mean implied / assured). >> Grab a sample over some finite temporal epoch such that the mean *is* >> zero. It should be easy enough to do this by generating the noise and >> computing the mean from the beginning, for some time, and then continuing >> to compute the mean of the entire record until the mean is zero - then >> stop grabbing data. >> Now, cut the data into two equal halves in time. >> There is some large probability that each half will not each have zero >> mean. Of course the sum of the means of the two halves will be zero >> because that's what you constructed in the grab. >> >> Fred >
/*snip*/
> Secondly, if you generate white noise (x) and calculate the average > (avg_n) repeatedly after every sample generated, you should notice that > the chance of the average being larger than 0 is equal to of it being > smaller than 0 (i.e. P(avg_n>0) = 0.5). (avg_n = sum_{i=1}^n x[n], where x > is a sequence of random samples.) This follows from the central limit > theorem.
On a third thought, the avg_n isn't i.i.d anymore. So what I said is probably incorrect (i.e. the conditional probability P(avg_n avg_n-1,...,avg_0) probably doesn't have to be symmetric around 0). Not sure about this though (shouldn't post in haste :)) -- Jani Huhtanen Tampere University of Technology, Pori
I am going to discuss continuous-time random
processes and white noise only.  So, please do
not muddy the waters by bringing in discrete-time
white noise processes as obvious counterexamples
for what I say below.

For a second-order (i.e. finite variance) stationary
random process, the autocorrelation function
Rxx(t) = E[X(s)X(s+t)] is either a periodic function
or decays asymptotically as t approaches infinity
(or -infinity) to m^2, the square of the mean of the
process.  In other words, two random variables
corresponding to time instants that are far apart on
the time axis are *uncorrelated* in the statistical
sense of the word; their covariance is 0; the
autocovariance function Cxx(t) approaches 0
asymptotically, and Rxx(t) = Cxx(t) + m^2
approaches m^2.

If we apply these notions to white noise which is
*not* a second-order process but for many purposes
can be treated as a second-order processs, then
Rxx(t) = N0 delta(t) *does* imply that the mean m is
zero.

The process described by Tim Westcott (white noise
riding on a DC term) is not really a *white* noise process
in the sense that the autocorrelation function of this
other process is *not* of the form N0 delta(t).  So, the
real question is "what do we mean when we say that
X(t) is a white noise process?"  If "white" means that
the autocorrelation function is of the form N0 delta(t),
then the answer to Randy's question is Yes, white noise
has zero mean.  Notice also that the power spectral
density phixx(f) has value N0 for all f for this "version"
of white noise.  For the white noise + DC term, the
power spectral density includes an impulse at f = 0.
Do we still want to call it white noise?  I wouldn't
but then, I am well known on this group for being a
stuck-up theoretician and pedant....

Ultimately, the question of white noise is a philosophical
and mathematical question.  For many linear filters, it
is an experimentally observed fact that the noise
process at the output has an autocorrelation function
Ryy(t) that is proportional to the autocorrelation function
Rhh(t) of the impulse response h(t) of the filter.
Equivalently, phiyy(f) the power spectral density of the filter
output is proportional to |H(f)|^2, the power transfer
function of the filter.  These results can be "justified"
very easily by PRETENDING that there exists a process
with autocorrelation function N0 delta(t), and that we can
treat this process as a second order process even though
it has infinite variance.  The standard second-order theory
for linear systems says that
Ryyy(t) = Rxx(t) * Rhh(t) which equals N0 Rhh(t) if
Rxx(t) happens to be N0 delta(t).  Similarly,
phiyy(f) = phixx(f)|H(f)|^2 which equals N0 |H(f))^2 if
phixx(f) happens to have value N0 for all f.  Thus,
the theory gives results that are in line with experimental
observations.  Does white noise actually exist?
What is the meaning of life, the universe, and everything?

I think it is best (as Randy says in a later note in
this thread) that we use the term white noise
exclusively to denote a (zero-mean) process with
autocorrelation function N0 delta(t), and power
spectral density phixx(f) = N0 for all f.

Finally, I want to re-emphasize that this discussion
is about continuous-time processes.  So, let's
keep discrete-time white noise processes out of it.

Hope this helps

dvsarwate@ieee.org wrote:

   ...

> Hope this helps
Of course it helps. Some of us would rather think than do math; that's like walking on thin ice. Others would rather do math than think; that's like going around blindfolded. Dilip does both wonderfully well, and so usually has the last word, even when we don't realize and keep talking. Jerry -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;
Dilip, Tim, et al.,

Thank you for that clarification, and thanks for your help. 

I claim that white noise, as defined by at least two authors, does not
require a zero mean. However, in each case I feel the definitions from
these authors is missing a necessary ingredient. I describe
this in detail in the new paper "Mean of a White-Noise Process,"

  http://www.digitalsignallabs.com/white.pdf

Please have a look and critique - if you see an error or
an oversight, I want to know about it.

--Randy Yates


dvsarwate@ieee.org writes:

> I am going to discuss continuous-time random > processes and white noise only. So, please do > not muddy the waters by bringing in discrete-time > white noise processes as obvious counterexamples > for what I say below. > > For a second-order (i.e. finite variance) stationary > random process, the autocorrelation function > Rxx(t) = E[X(s)X(s+t)] is either a periodic function > or decays asymptotically as t approaches infinity > (or -infinity) to m^2, the square of the mean of the > process. In other words, two random variables > corresponding to time instants that are far apart on > the time axis are *uncorrelated* in the statistical > sense of the word; their covariance is 0; the > autocovariance function Cxx(t) approaches 0 > asymptotically, and Rxx(t) = Cxx(t) + m^2 > approaches m^2. > > If we apply these notions to white noise which is > *not* a second-order process but for many purposes > can be treated as a second-order processs, then > Rxx(t) = N0 delta(t) *does* imply that the mean m is > zero. > > The process described by Tim Westcott (white noise > riding on a DC term) is not really a *white* noise process > in the sense that the autocorrelation function of this > other process is *not* of the form N0 delta(t). So, the > real question is "what do we mean when we say that > X(t) is a white noise process?" If "white" means that > the autocorrelation function is of the form N0 delta(t), > then the answer to Randy's question is Yes, white noise > has zero mean. Notice also that the power spectral > density phixx(f) has value N0 for all f for this "version" > of white noise. For the white noise + DC term, the > power spectral density includes an impulse at f = 0. > Do we still want to call it white noise? I wouldn't > but then, I am well known on this group for being a > stuck-up theoretician and pedant.... > > Ultimately, the question of white noise is a philosophical > and mathematical question. For many linear filters, it > is an experimentally observed fact that the noise > process at the output has an autocorrelation function > Ryy(t) that is proportional to the autocorrelation function > Rhh(t) of the impulse response h(t) of the filter. > Equivalently, phiyy(f) the power spectral density of the filter > output is proportional to |H(f)|^2, the power transfer > function of the filter. These results can be "justified" > very easily by PRETENDING that there exists a process > with autocorrelation function N0 delta(t), and that we can > treat this process as a second order process even though > it has infinite variance. The standard second-order theory > for linear systems says that > Ryyy(t) = Rxx(t) * Rhh(t) which equals N0 Rhh(t) if > Rxx(t) happens to be N0 delta(t). Similarly, > phiyy(f) = phixx(f)|H(f)|^2 which equals N0 |H(f))^2 if > phixx(f) happens to have value N0 for all f. Thus, > the theory gives results that are in line with experimental > observations. Does white noise actually exist? > What is the meaning of life, the universe, and everything? > > I think it is best (as Randy says in a later note in > this thread) that we use the term white noise > exclusively to denote a (zero-mean) process with > autocorrelation function N0 delta(t), and power > spectral density phixx(f) = N0 for all f. > > Finally, I want to re-emphasize that this discussion > is about continuous-time processes. So, let's > keep discrete-time white noise processes out of it. > > Hope this helps >
-- % Randy Yates % "I met someone who looks alot like you, %% Fuquay-Varina, NC % she does the things you do, %%% 919-577-9882 % but she is an IBM." %%%% <yates@ieee.org> % 'Yours Truly, 2095', *Time*, ELO http://home.earthlink.net/~yatescr
Hi,

Equation 7 (def. for uncorrelated). You have

E[x(t+tau)x(t)]=E[x(t+tau)]E[x(t)]

That seems like the definition of independence and not uncorrelation. I
think you meant:

E[x(t+tau)x(t)]= 0

Offcourse indepedence results in uncorrelation, but uncorrelation does
not result in independence.

Is that a typo or am i missing something ?

Hi,

I think Brown's definition of WN gives zero mean for the process
(unless I did a mistake).

>From his definition a WN (stationary with constant psd) :
E[x(t)]=mx %constant R(x(t2),x(t1))= R(tau) Sx= var (constant psd) which implies -> R(tau)=delta(tau)*var So that using this equation for the mean var = E[x(t)^2] - (E[x(t)])^2 we get (E[x(t)])^2 = E[x(t)^2] -var = E[R(0)] - var = var -var = 0 -Ikaro (E[x(t)])^2 =
Randy Yates wrote:

> Let Z(t) be a white-noise (stationary) random process. > Can we conclude that Z(t) has zero mean?
Suppose it is zero mean. Therefore the DC component of the spectrum is zero. But white noise has a constant (flat, non-zero) spectrum. Discuss. :-) Ciao, Peter K.