Definition of a Stationary Process

According to Athanasios Papoulis in "Probability, random variables, and 
stochastic processes", McGraw-Hill 1984, a stationary stochastic process 
is one whose statistical properties are invariant to a shift in the 
origin.  I've had problems with this definition ever since I heard it, 
and I'm hoping that someone will be able to clarify (hopefully with 
appropriate references).

It seems to me that the term "non-stationary" is overused, i.e. that 
processes that are stationary but are not well-behavied Gaussian 
processes are classified as non-stationary.

Let's assume that you have two random processes, x(t) and y(t).  x(t) is 
white and stationary, and y(t) is stationary and bandlimited.

Now make three processes out of them:

w(t) = x(t) + y(t),
s(t) = x(t) * cos(42 * t),
z(t) = x(t) + y(t).

Now obviously w(t) is stationary, and s(t) isn't (s(t) is 
cyclostationary, but let's not touch that with a ten-foot pole).

But what about z(t)?  To my simple mind it would seem that Papoulis's 
definition results in the conclunsion that z(t) is stationary, because 
(a) it is the result of a memoryless operation on two stationary 
processes, and (b) you can't look at the clock and gain any a-priori 
knowledge about z(t).  On the other hand one could argue that z(t) is a 
white Gaussian process whose variance changes with time, according to 
the value of y(t).

So for the moment I'll call the argument that z(t) is stationary "my 
definition" and the argument that z(t) is non-stationary "the apparant 
definition".

In nonlinear system analysis it common to simplify the analysis of a 
time-invariant nonlinear system such a jet airplane by pretending that 
it is a time-varying linear system.  This is done because in this 
example the equations of motion for the aircraft are largely 
parameterized by air density and airspeed, so you can measure those two 
states, pretend that you've measured time, and do a gain-scheduled 
controller.

I'm wondering if this is not the case with my definition of 
stationarity, where you have a process that is not a simple Gaussian 
process, is the result of a memoryless operation on two or more 
stationary processes, is indeed stationary, but for which the math can 
be vastly simplified by treating the thing as non-stationary.

Which is right?  My definition, or the apparent one?  If the apparent 
one is right, what do you call a process such as z(t) vs. something like 
s(t)?

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

In article <10q9v0jjsiie267@corp.supernews.com>, Tim Wescott <tim@wescottnospamdesign.com> wrote:

> w(t) = x(t) + y(t),
> s(t) = x(t) * cos(42 * t),
> z(t) = x(t) + y(t).

and the difference between w and z is ... ???

A N Niel wrote:

> In article <10q9v0jjsiie267@corp.supernews.com>, Tim Wescott <tim@wescottnospamdesign.com> wrote:
> 
> 
>>w(t) = x(t) + y(t),
>>s(t) = x(t) * cos(42 * t),
>>z(t) = x(t) + y(t).
> 
> 
> and the difference between w and z is ... ???

Well, if you recognize the typo you'd know that z(t) is the product, and 
w(t) is the sum, but I suppose that there isn't enough supporting text 
to make that apparent.

Thanks for picking that up.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

In article <10q9v0jjsiie267@corp.supernews.com>,
Tim Wescott  <tim@wescottnospamdesign.com> wrote:
>According to Athanasios Papoulis in "Probability, random variables, and 
>stochastic processes", McGraw-Hill 1984, a stationary stochastic process 
>is one whose statistical properties are invariant to a shift in the 
>origin.

OK.  Or in somewhat more detail: for every h, n and t_1,...,t_n
(with h and all t_j >= 0 if the process is only defined for time t >= 0)
{X(t_1),...,X(t_n)} and {X(t_1+h),...,X(t_n+h)} have the same joint
distribution.

There is also "weakly stationary" or "second-order stationary", which 
means that the expected value and covariance are invariant under shifts.
In general this is a weaker property than "stationary".

>It seems to me that the term "non-stationary" is overused, i.e. that 
>processes that are stationary but are not well-behavied Gaussian 
>processes are classified as non-stationary.

Ooh, I hope not.  Why not call them non-Gaussian?

Or is this another case where the DSP people are using different
definitions than mathematicians?

>Let's assume that you have two random processes, x(t) and y(t).  x(t) is 
>white and stationary, and y(t) is stationary and bandlimited.

You want the x and y processes to be independent, I suppose.

>Now make three processes out of them:

>w(t) = x(t) + y(t),
>s(t) = x(t) * cos(42 * t),
>z(t) = x(t) + y(t).

I think (judging from what you say below) you mean z(t) = x(t) * y(t).

>Now obviously w(t) is stationary, and s(t) isn't (s(t) is 
>cyclostationary, but let's not touch that with a ten-foot pole).

>But what about z(t)?  To my simple mind it would seem that Papoulis's 
>definition results in the conclunsion that z(t) is stationary, because 
>(a) it is the result of a memoryless operation on two stationary 
>processes, and (b) you can't look at the clock and gain any a-priori 
>knowledge about z(t).  On the other hand one could argue that z(t) is a 
>white Gaussian process whose variance changes with time, according to 
>the value of y(t).

It is stationary, and its variance does not change with time.  What does 
change is the conditional variance of z(t) given y(t).

>In nonlinear system analysis it common to simplify the analysis of a 
>time-invariant nonlinear system such a jet airplane by pretending that 
>it is a time-varying linear system.  This is done because in this 
>example the equations of motion for the aircraft are largely 
>parameterized by air density and airspeed, so you can measure those two 
>states, pretend that you've measured time, and do a gain-scheduled 
>controller.

>I'm wondering if this is not the case with my definition of 
>stationarity, where you have a process that is not a simple Gaussian 
>process, is the result of a memoryless operation on two or more 
>stationary processes, is indeed stationary, but for which the math can 
>be vastly simplified by treating the thing as non-stationary.

There is no advantage in treating a stationary thing as non-stationary
per se, i.e. there's no technique that works only if you forget the
fact that the process is stationary.  If you're talking about a 
technique that works whether or not the process is stationary: sure, 
why not?  In the case of z(t) above, maybe for some purposes it's
helpful to look at the conditional process (z(t) given y) which
is Gaussian but not stationary, rather than the actual z(t) which
is stationary but not Gaussian.

Robert Israel                                israel@math.ubc.ca
Department of Mathematics        http://www.math.ubc.ca/~israel 
University of British Columbia            Vancouver, BC, Canada

Robert Israel wrote:

> In article <10q9v0jjsiie267@corp.supernews.com>,
> Tim Wescott  <tim@wescottnospamdesign.com> wrote:
> 
>>According to Athanasios Papoulis in "Probability, random variables, and 
>>stochastic processes", McGraw-Hill 1984, a stationary stochastic process 
>>is one whose statistical properties are invariant to a shift in the 
>>origin.
> 
> 
> OK.  Or in somewhat more detail: for every h, n and t_1,...,t_n
> (with h and all t_j >= 0 if the process is only defined for time t >= 0)
> {X(t_1),...,X(t_n)} and {X(t_1+h),...,X(t_n+h)} have the same joint
> distribution.

Well, Papoulis's definition is for a continuous process, but I don't see 
anything below that depends on continuous vs. discrete time.
> 
> There is also "weakly stationary" or "second-order stationary", which 
> means that the expected value and covariance are invariant under shifts.
> In general this is a weaker property than "stationary".
> 
> 
>>It seems to me that the term "non-stationary" is overused, i.e. that 
>>processes that are stationary but are not well-behavied Gaussian 
>>processes are classified as non-stationary.
> 
> 
> Ooh, I hope not.  Why not call them non-Gaussian?

Ooh, it is done.  Specifically I have a copy of a PhD thesis ("An 
Atmospheric Noise Model with Application to Low Frequency Navigation 
Systems", D.A. Feldman, MIT, 1972), where the atmospheric noise in the 
LF and MF (300-ish kHz) radio bands is called "non-stationary".  The 
noise is akin to my (corrected) z(t) below, except that it can be 
modeled as a high-pass of a Poisson process (a discharge can happen any 
time, and has a short duration) times a white process with a Cauer-like 
density (a discharge can be anything from infinitesimal to a 
thunderbolt).  If you take the parameters of each process as fixed then 
the result should be stationary, yes?

There is a little bit of confusion in his definition because the 
parameters _do_ vary with the season and the time of day, which would 
certainly make the resulting process non-stationary, but his text seems 
to indicate that it is the non-Gaussian characteristics of the process 
which leads him to call it non-stationary.
> 
> Or is this another case where the DSP people are using different
> definitions than mathematicians?
> 
I think it's a case of engineers misusing the term, actually.  We see 
the apparently varying variance of our data and say "gee, must be a 
non-stationary process", when what we're really seeing is a non-Gaussian 
process.
> 
>>Let's assume that you have two random processes, x(t) and y(t).  x(t) is 
>>white and stationary, and y(t) is stationary and bandlimited.
> 
> 
> You want the x and y processes to be independent, I suppose.
> 
> 
>>Now make three processes out of them:
> 
> 
>>w(t) = x(t) + y(t),
>>s(t) = x(t) * cos(42 * t),
>>z(t) = x(t) + y(t).
> 
> 
> I think (judging from what you say below) you mean z(t) = x(t) * y(t).
> 
Yes, thank you.
> 
>>Now obviously w(t) is stationary, and s(t) isn't (s(t) is 
>>cyclostationary, but let's not touch that with a ten-foot pole).
> 
> 
>>But what about z(t)?  To my simple mind it would seem that Papoulis's 
>>definition results in the conclunsion that z(t) is stationary, because 
>>(a) it is the result of a memoryless operation on two stationary 
>>processes, and (b) you can't look at the clock and gain any a-priori 
>>knowledge about z(t).  On the other hand one could argue that z(t) is a 
>>white Gaussian process whose variance changes with time, according to 
>>the value of y(t).
> 
> 
> It is stationary, and its variance does not change with time.  What does 
> change is the conditional variance of z(t) given y(t).
> 
Ah HA!  OK.  This is what I have thought, but I have seen numerous 
examples of this type of thing being called "non-stationary".  Now I'll 
have to start collecting references.
> 
>>In nonlinear system analysis it common to simplify the analysis of a 
>>time-invariant nonlinear system such a jet airplane by pretending that 
>>it is a time-varying linear system.  This is done because in this 
>>example the equations of motion for the aircraft are largely 
>>parameterized by air density and airspeed, so you can measure those two 
>>states, pretend that you've measured time, and do a gain-scheduled 
>>controller.
> 
> 
>>I'm wondering if this is not the case with my definition of 
>>stationarity, where you have a process that is not a simple Gaussian 
>>process, is the result of a memoryless operation on two or more 
>>stationary processes, is indeed stationary, but for which the math can 
>>be vastly simplified by treating the thing as non-stationary.
> 
> 
> There is no advantage in treating a stationary thing as non-stationary
> per se, i.e. there's no technique that works only if you forget the
> fact that the process is stationary.  If you're talking about a 
> technique that works whether or not the process is stationary: sure, 
> why not?  In the case of z(t) above, maybe for some purposes it's
> helpful to look at the conditional process (z(t) given y) which
> is Gaussian but not stationary, rather than the actual z(t) which
> is stationary but not Gaussian.

I think that's where I was going with my control systems example -- 
optimal processors for Gaussian signals with square-law cost functions 
tend to be linear, with lots of nice closed-form integral equations to 
derive them.  As soon as the noise becomes non-Gaussian you have to toss 
out all that nice tidy linear analysis.  If you squint a little bit and 
estimate a multiplier then you can say its stationary but Gaussian.  I 
suspect that somebody did this a long time ago and the rest of us 
forgot, or that folks have been doing this intuitively without realizing it.
> 
> Robert Israel                                israel@math.ubc.ca
> Department of Mathematics        http://www.math.ubc.ca/~israel 
> University of British Columbia            Vancouver, BC, Canada
> 
> 
This is good.  Now I can say "but I was told by a guy with a _PhD_ that 
this is _true_".  If the argument gets heated I can even point out that 
it's a PhD in Math, not Fine Arts.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

"[...] Ooh, it is done. Specifically I have a copy of a PhD thesis ("An

Atmospheric Noise Model with Application to Low Frequency Navigation
Systems", D.A. Feldman, MIT, 1972), where the atmospheric noise in the
LF and MF (300-ish kHz) radio bands is called "non-stationary". "

Feldman is right: Atmospheric noise is very non-stationary. It depende
on the time of day, seasons, sun spots activity, cosmic rays, human
activity, and such. You may reasonably assume stationarity only over a
short time scale, such as seconds. It clearly has strong periodic
components over days, months and years.

In article <10q9v0jjsiie267@corp.supernews.com>,
Tim Wescott  <tim@wescottnospamdesign.com> wrote:
>According to Athanasios Papoulis in "Probability, random variables, and 
>stochastic processes", McGraw-Hill 1984, a stationary stochastic process 
>is one whose statistical properties are invariant to a shift in the 
>origin.  I've had problems with this definition ever since I heard it, 
>and I'm hoping that someone will be able to clarify (hopefully with 
>appropriate references).

>It seems to me that the term "non-stationary" is overused, i.e. that 
>processes that are stationary but are not well-behavied Gaussian 
>processes are classified as non-stationary.

>Let's assume that you have two random processes, x(t) and y(t).  x(t) is 
>white and stationary, and y(t) is stationary and bandlimited.

There is a little difficulty with proceeding from this.
A "white" continuous time process does not exist; its
integral does.  But we can ignore this.

>Now make three processes out of them:

>w(t) = x(t) + y(t),
>s(t) = x(t) * cos(42 * t),
>z(t) = x(t) + y(t).

>Now obviously w(t) is stationary,

Assuming the processes are independent, or something else
sufficiently strong.

		and s(t) isn't (s(t) is 
>cyclostationary, but let's not touch that with a ten-foot pole).

>But what about z(t)?

From what I see on my screen, there is no difference between
the w and z processes.  Do you mean * instead of +?  Some of
your later material seems to indicate this.

 To my simple mind it would seem that Papoulis's 
>definition results in the conclunsion that z(t) is stationary, because 
>(a) it is the result of a memoryless operation on two stationary 
>processes, and (b) you can't look at the clock and gain any a-priori 
>knowledge about z(t).  On the other hand one could argue that z(t) is a 
>white Gaussian process whose variance changes with time, according to 
>the value of y(t).

This argument is false in more than one way.  For one, z is
no longer Gaussian.  For the other, you are being quite 
ambiguous in your use of "white"; as I remarked above, white
noise does not exist, and how it is "used" needs to be made
precise.  

In fact, 

	q(t) = x(t) * cos(42 * (t + U)),

where U is uniform between 0 and 1/42, is a strictly stationary
Gaussian process, except for the problem of white noise, as
long as U is independent of x.

-- 
This address is for information only.  I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@stat.purdue.edu         Phone: (765)494-6054   FAX: (765)494-0558

"Herman Rubin" <hrubin@odds.stat.purdue.edu> wrote in message 
news:co593t$4bfo@odds.stat.purdue.edu...
>>
>>Let's assume that you have two random processes, x(t) and y(t).  x(t) is
>>white and stationary, and y(t) is stationary and bandlimited.
>
> There is a little difficulty with proceeding from this.
> A "white" continuous time process does not exist; its
> integral does.  But we can ignore this.

Hi Herman,

I noticed that you said a "white" continuous time process does not exist... 
so do you mean a "white" discrete random process(random sequence) does 
exist?


> This argument is false in more than one way.  For one, z is
> no longer Gaussian.  For the other, you are being quite
> ambiguous in your use of "white"; as I remarked above, white
> noise does not exist, and how it is "used" needs to be made
> precise.

Could you please tell me why z is no longer Gaussian?

>
> In fact,
>
> q(t) = x(t) * cos(42 * (t + U)),
>
> where U is uniform between 0 and 1/42, is a strictly stationary
> Gaussian process, except for the problem of white noise, as
> long as U is independent of x.
>

By intuition I understand that cos( xxx * t + U ) where U is a uniform 
random variable is always strictly stationary... But I am not sure if you 
multiply it with another x(t), will it be SSS still? I guess it is very hard 
to show mathematically?

Thanks a lot and happy holiday!

In article <co5dpo$r47$1@news.Stanford.EDU>, kiki <lunaliu3@yahoo.com> wrote:

>"Herman Rubin" <hrubin@odds.stat.purdue.edu> wrote in message 
>news:co593t$4bfo@odds.stat.purdue.edu...

>>>Let's assume that you have two random processes, x(t) and y(t).  x(t) is
>>>white and stationary, and y(t) is stationary and bandlimited.

>> There is a little difficulty with proceeding from this.
>> A "white" continuous time process does not exist; its
>> integral does.  But we can ignore this.

>Hi Herman,

>I noticed that you said a "white" continuous time process does not exist... 
>so do you mean a "white" discrete random process(random sequence) does 
>exist?

There is no problem with that.   One can easily have a
discrete process with all the values independent and
identically distributed, and sums give no existence
problems.  However, one cannot have a continuous time
process with all the values independent and identically
distributed, and the integral being anything random.

White noise is usually considered as the "derivative"
of a continuous Gaussian process with independent
increments.  There is no problem with the existence
of such processes, but the derivatives do not exist
in any sense.

>> This argument is false in more than one way.  For one, z is
>> no longer Gaussian.  For the other, you are being quite
>> ambiguous in your use of "white"; as I remarked above, white
>> noise does not exist, and how it is "used" needs to be made
>> precise.

>Could you please tell me why z is no longer Gaussian?

Products of Gaussian random variables are not Gaussian.

>> In fact,

>> q(t) = x(t) * cos(42 * (t + U)),

>> where U is uniform between 0 and 1/42, is a strictly stationary
>> Gaussian process, except for the problem of white noise, as
>> long as U is independent of x.


>By intuition I understand that cos( xxx * t + U ) where U is a uniform 
>random variable is always strictly stationary... But I am not sure if you 
>multiply it with another x(t), will it be SSS still? I guess it is very hard 
>to show mathematically?

The product of two strictly stationary independent 
processes is always strictly stationary.  Just use
the definition.

>Thanks a lot and happy holiday!





-- 
This address is for information only.  I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@stat.purdue.edu         Phone: (765)494-6054   FAX: (765)494-0558

"Herman Rubin" <hrubin@odds.stat.purdue.edu> wrote in message 
news:co5fog$1pai@odds.stat.purdue.edu...
> In article <co5dpo$r47$1@news.Stanford.EDU>, kiki <lunaliu3@yahoo.com> 
> wrote:
>
>>"Herman Rubin" <hrubin@odds.stat.purdue.edu> wrote in message
>>news:co593t$4bfo@odds.stat.purdue.edu...
>
>>>>Let's assume that you have two random processes, x(t) and y(t).  x(t) is
>>>>white and stationary, and y(t) is stationary and bandlimited.
>
>>> There is a little difficulty with proceeding from this.
>>> A "white" continuous time process does not exist; its
>>> integral does.  But we can ignore this.
>
>>Hi Herman,
>
>>I noticed that you said a "white" continuous time process does not 
>>exist...
>>so do you mean a "white" discrete random process(random sequence) does
>>exist?
>
> There is no problem with that.   One can easily have a
> discrete process with all the values independent and
> identically distributed, and sums give no existence
> problems.  However, one cannot have a continuous time
> process with all the values independent and identically
> distributed, and the integral being anything random.
>
> White noise is usually considered as the "derivative"
> of a continuous Gaussian process with independent
> increments.  There is no problem with the existence
> of such processes, but the derivatives do not exist
> in any sense.
>
>>> This argument is false in more than one way.  For one, z is
>>> no longer Gaussian.  For the other, you are being quite
>>> ambiguous in your use of "white"; as I remarked above, white
>>> noise does not exist, and how it is "used" needs to be made
>>> precise.
>
>>Could you please tell me why z is no longer Gaussian?
>
> Products of Gaussian random variables are not Gaussian.
>
>>> In fact,
>
>>> q(t) = x(t) * cos(42 * (t + U)),
>
>>> where U is uniform between 0 and 1/42, is a strictly stationary
>>> Gaussian process, except for the problem of white noise, as
>>> long as U is independent of x.
>
>
>>By intuition I understand that cos( xxx * t + U ) where U is a uniform
>>random variable is always strictly stationary... But I am not sure if you
>>multiply it with another x(t), will it be SSS still? I guess it is very 
>>hard
>>to show mathematically?
>
> The product of two strictly stationary independent
> processes is always strictly stationary.  Just use
> the definition.
>

Hi Herman,

Since I did not figure out how to show that mathematially. I did some 
experiments and played around with the uniform range of that Random Variable 
U. I tried to understand why you said "where U is uniform between 0 and 
1/42, ..."

----------------------------------
fc=1000;  %1000 Hz
N=10000;
t=[0:1/Fs:N/Fs];
% THETA=rand(1, length(t))/(2*pi*fc);  %This works...
% r=cos(2*pi*fc*(t + THETA));

THETA=rand(1, length(t));
K=2; %This works too...
r=cos(2*pi*fc*t + K*THETA);

h = spectrum.welch;
psd(h, r, 'Fs', Fs, 'SpectrumType','twosided','CenterDC',true);
------------------------------

I found out that only when K is an integer number, so that THETA is uniform 
in [0, K], then cos(2*pi*fc*t + K*THETA) is stationary... as shown by its 
autocorrelation function, which is 0.5*cos(2*pi*fc*t) and its power spectral 
density has two peak impulses at -fc Hz and +fc Hz... which is correctly the 
Fourier Transform of cos(2*pi*fc*t)...

But if you plug in other K, for example, K = 2*pi, and then THETA uniform in 
the range of [0, 2*pi], it does not generate the correct power spectral 
density function(two impulse peaks...)

What's wrong with it?

Thanks a lot!