i.i.d. sequence

Hello,

I recently posted the question:

*Let x be a zero-mean, white stochastic process and let E{ } denote the
expectation-value operator. Then how can i prove that all the odd-moments
of x are zero, namely E{x^(2k+1)}=0 ?

Tim gave a nice answer by providing the counter-example
x[i] = -1/3 with probability 3/4 
x[i] = 1 with probability 1/4
x[i] independent of x[j] for i~=j

Indeed this is a white process with E{x}=0 and E{x^3}=2/9.

*However this is not an independent and identically distributed
process(i.i.d.), because the two values of x appear with different
probabilities. What if we impose the constraint on x of being an i.i.d.
sequence? In that case my results show that E{x^(2k+1)}=0, albeit i cannot
prove it.

Any insight?

Manolis C. Tsakiris wrote:
> Hello,
>
> I recently posted the question:
>
> *Let x be a zero-mean, white stochastic process and let E{ } denote the
> expectation-value operator. Then how can i prove that all the odd-moments
> of x are zero, namely E{x^(2k+1)}=0 ?
>
> Tim gave a nice answer by providing the counter-example
> x[i] = -1/3 with probability 3/4    (1)
> x[i] = 1 with probability 1/4       (2)
> x[i] independent of x[j] for i~=j   (3)
>
> Indeed this is a white process with E{x}=0 and E{x^3}=2/9.

Tim's counter-example wasn't proper, because a non-zero mean iid
sequence of random variables does not constitute a white noise process
(look at the autocorrelation function).

> *However this is not an independent and identically distributed
> process(i.i.d.), because the two values of x appear with different
> probabilities.

But that process is iid! The samples are independent (3) and
identically distributed (1) and (2).

However, I think you can forget about random processes altogether and
just consider a simple random variable (see blow).

> What if we impose the constraint on x of being an i.i.d.
> sequence? In that case my results show that E{x^(2k+1)}=0, albeit i cannot
> prove it.

I think you have another situation in mind: presume the random
variable X has an even symmetric distribution about 0 (thus zero
mean), then

E{X^(2k+1)}=0 for positive integers k.

This is easy to prove (look at the symmetry of the expectation
integrand).

Regards,
Andor

On 12 Mrz., 12:40, Andor <andor.bari...@gmail.com> wrote:
> Manolis C. Tsakiris wrote:
> > Hello,
>
> > I recently posted the question:
>
> > *Let x be a zero-mean, white stochastic process and let E{ } denote the
> > expectation-value operator. Then how can i prove that all the odd-moments
> > of x are zero, namely E{x^(2k+1)}=0 ?
>
> > Tim gave a nice answer by providing the counter-example
> > x[i] = -1/3 with probability 3/4 &#4294967295; &#4294967295;(1)
> > x[i] = 1 with probability 1/4 &#4294967295; &#4294967295; &#4294967295; (2)
> > x[i] independent of x[j] for i~=j &#4294967295; (3)
>
> > Indeed this is a white process with E{x}=0 and E{x^3}=2/9.
>
> Tim's counter-example wasn't proper, because a non-zero mean iid
> sequence of random variables does not constitute a white noise process
> (look at the autocorrelation function).

Oops, scratch that. It is zero mean, sorry.

>Tim's counter-example wasn't proper, because a non-zero mean iid
>sequence of random variables does not constitute a white noise process
>(look at the autocorrelation function).

You mean in the sense that although the samples are independent, the
variance of each sample can be either (-1/4)^2=1/16 or (1)^2=1. In a white
sequence all the samples are independent and have the same variance.
Right?

>> *However this is not an independent and identically distributed
>> process(i.i.d.), because the two values of x appear with different
>> probabilities.
>
>But that process is iid! The samples are independent (3) and
>identically distributed (1) and (2).

Now, i disagree here. According to my experience so far, an i.i.d.
sequence is a sequence in which different values of the samples appear
with the same probability. That's the "identically distributed" part! For
example, the samples of a 16-QAM signal can take 16 different values, each
one with probability 1/16. Differently, where does the "identically
distributed" part refer to? 

Manolis

On 12 Mrz., 13:03, "Manolis C. Tsakiris" <el01...@mail.ntua.gr> wrote:
> >Tim's counter-example wasn't proper, because a non-zero mean iid
> >sequence of random variables does not constitute a white noise process
> >(look at the autocorrelation function).
>
> You mean in the sense that although the samples are independent, the
> variance of each sample can be either (-1/4)^2=1/16 or (1)^2=1. In a white
> sequence all the samples are independent and have the same variance.
> Right?

No, my statement was wrong and I corrected it in a second post.


> >> *However this is not an independent and identically distributed
> >> process(i.i.d.), because the two values of x appear with different
> >> probabilities.
>
> >But that process is iid! The samples are independent (3) and
> >identically distributed (1) and (2).
>
> Now, i disagree here. According to my experience so far, an i.i.d.
> sequence is a sequence in which different values of the samples appear
> with the same probability. That's the "identically distributed" part! For
> example, the samples of a 16-QAM signal can take 16 different values, each
> one with probability 1/16. Differently, where does the "identically
> distributed" part refer to?

An IID process consists of a sequence of independent random variables
that all have the same (identical) distribution. IID doesn't say
anything about the distribution itself.

Regards,
Andor

>
>An IID process consists of a sequence of independent random variables
>that all have the same (identical) distribution. IID doesn't say
>anything about the distribution itself.
>

Right! Do the samples of the example that Tim gave have the same
distribution? No! The value -1/3 appears with probability 3/4 and the
value 1 appears with probability 1/4. So this sequence is not i.i.d.

It would be i.i.d. if -1/3 appeared with probability 1/2 and 1 with
probability 1/2. Dont'you agree?

Manolis

On 12 Mrz., 13:28, "Manolis C. Tsakiris" <el01...@mail.ntua.gr> wrote:
> >An IID process consists of a sequence of independent random variables
> >that all have the same (identical) distribution. IID doesn't say
> >anything about the distribution itself.
>
> Right! Do the samples of the example that Tim gave have the same
> distribution? No! The value -1/3 appears with probability 3/4 and the
> value 1 appears with probability 1/4. So this sequence is not i.i.d.
>
> It would be i.i.d. if -1/3 appeared with probability 1/2 and 1 with
> probability 1/2. Dont'you agree?

You are mixing up a random process and a random variable. The random
variable X described above has a distribution (on the discrete space
{-1/3, 1} given by

P[X=-1/3] = 3/4, P[X=1] = 1/4.

Now make a process {X_k}, k= 0, 1, ..., and each X_k is an independent
random variable with the same distribution as X above, ie.

P[X_k=-1/3] = 3/4, P[X_k=1] = 1/4.

This process {X_k} is then i.i.d.

Regards,
Andor

>You are mixing up a random process and a random variable. The random
>variable X described above has a distribution (on the discrete space
>{-1/3, 1} given by
>
>P[X=-1/3] = 3/4, P[X=1] = 1/4.
>
>Now make a process {X_k}, k= 0, 1, ..., and each X_k is an independent
>random variable with the same distribution as X above, ie.
>
>P[X_k=-1/3] = 3/4, P[X_k=1] = 1/4.
>
>This process {X_k} is then i.i.d.
>
>Regards,
>Andor
>

***********************************
mmmmmmmm...that was very enlightening!

Be well Andor.

Manolis

On Wed, 12 Mar 2008 07:59:33 -0500, Manolis C. Tsakiris wrote:

>>You are mixing up a random process and a random variable. The random
>>variable X described above has a distribution (on the discrete space
>>{-1/3, 1} given by
>>
>>P[X=-1/3] = 3/4, P[X=1] = 1/4.
>>
>>Now make a process {X_k}, k= 0, 1, ..., and each X_k is an independent
>>random variable with the same distribution as X above, ie.
>>
>>P[X_k=-1/3] = 3/4, P[X_k=1] = 1/4.
>>
>>This process {X_k} is then i.i.d.
>>
>>Regards,
>>Andor
>>
>>
> ***********************************
> mmmmmmmm...that was very enlightening!
> 
> Be well Andor.
> 
> Manolis

Besides, it's not true even by your definition.  First, because you could 
just consider the problem for x = 1/4, 1/4, 1/4 and -3/4, all with 
probability 1/4.  If you don't like that, you can consider the problem 
for x = 1/2-e, 1/2+e and -1, all with probability 1/3.  For _any_ value 
of e your supposition doesn't hold.

Curiosity is good, but you really need to try working some simple 
examples before you go and ask the wide world for the answers to your 
questions.

-- 
Tim Wescott
Control systems and communications consulting
http://www.wescottdesign.com

Need to learn how to apply control theory in your embedded system?
"Applied Control Theory for Embedded Systems" by Tim Wescott
Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html

>
>Curiosity is good, but you really need to try working some simple 
>examples before you go and ask the wide world for the answers to your 
>questions.
>
****************************
Dear Tim,

had i not asked, i would not have realized the true meaning of an i.i.d.
sequence. Andor's example was crystal clear for me and for anyone who is
going to read it. I have read the definition of an i.i.d. sequence before
but it was so abstract that it prevented me from understanding it.

So, please restrict yourself to providing good and reliable answers and do
not criticize questions. If you find a question very obvious for you, just
don't deal with it, because as a professor of mine used to say "there are
not stupid questions, only stupid answers". 

thanx,
Manolis