Independent vs uncorrelated

Could someone clarify the diff between independent and uncorrelated
random variables?

Partho

Ben wrote:

> Could someone clarify the diff between independent and uncorrelated
> random variables?
> 
> Partho

Any spherically symmetric joint distribution implies uncorrelatedness.
However, gaussian joint distribution is the _only_ distribution that may be
both spherically symmetric (i.e., uncorrelated) and can be expressed as
product:

        P(X0,X1,...,Xn)=P(X0)P(X1)...P(Xn). 

If a joint distribution can be expressed as a product, the corresponding
random variables (X0...Xn) are independent. Thus there exists a large set
of distribution that the corresponding RVs are uncorrelated but not
independent.

Note that uncorrelatedness does _not_ imply spherical symmetry, but
independence does imply uncorrelatedness. 

Hope this gives some intuition, at least.

-- 
Jani Huhtanen
Tampere University of Technology, Pori

R.V. are independent if the joint PDF of X and Y=  (PDF of X) * ( PDF
of Y). This is the direct result of the fact that if X and Y are
independent than conditioning does not change the PDF.

R.V are uncorrelated if Expectation {XY} =E{x}E{Y}. As mentioned,
independence is a stronger condition which implied uncorrelatedness
(but not vice versa).

See also: http://www.stat.yale.edu/~pollard/241.fall97/Variance.pdf

Shlomo Kashani.

Ben wrote:
> Could someone clarify the diff between independent and uncorrelated
> random variables?
> 
> Partho

To paraphrase:

spherical symmetry of joint pdf P(X,Y,Z,....) ----> uncorrelated (rxy,
ryz, rxz,..... all are dirac functions)



P(X,Y,Z,.....) = P(X)P(Y)P(Z)..... <-----> independence ------->
uncorrelated


Please note the direction of implication signs.......

Partho



Jani Huhtanen wrote:
> Ben wrote:
>
> > Could someone clarify the diff between independent and uncorrelated
> > random variables?
> >
> > Partho
>
> Any spherically symmetric joint distribution implies uncorrelatedness.
> However, gaussian joint distribution is the _only_ distribution that may be
> both spherically symmetric (i.e., uncorrelated) and can be expressed as
> product:
>
>         P(X0,X1,...,Xn)=P(X0)P(X1)...P(Xn).
>
> If a joint distribution can be expressed as a product, the corresponding
> random variables (X0...Xn) are independent. Thus there exists a large set
> of distribution that the corresponding RVs are uncorrelated but not
> independent.
>
> Note that uncorrelatedness does _not_ imply spherical symmetry, but
> independence does imply uncorrelatedness.
>
> Hope this gives some intuition, at least.
> 
> -- 
> Jani Huhtanen
> Tampere University of Technology, Pori

Ben wrote:
> Could someone clarify the diff between independent and uncorrelated
> random variables?

Try this:

http://en.wikipedia.org/wiki/Normally_distributed_and_uncorrelated_does_not_imply_independent

Jani Huhtanen wrote:

> Ben wrote:
> 
>> Could someone clarify the diff between independent and uncorrelated
>> random variables?
>> 
>> Partho
> 
> Any spherically symmetric joint distribution implies uncorrelatedness.
> However, gaussian joint distribution is the _only_ distribution that may
> be both spherically symmetric (i.e., uncorrelated) and can be expressed as
> product:
> 
>         P(X0,X1,...,Xn)=P(X0)P(X1)...P(Xn).
> 

BTW. Does anyone know a proof that the only joint distribution that can be
both spherically symmetric and such that the corresponding RVs are
independent is gaussian distribution? Or equivalently proof for that the
only separable spherically symmetric 2d (or kd) kernel is gaussian.

-- 
Jani Huhtanen
Tampere University of Technology, Pori

Jani Huhtanen asked:

> BTW. Does anyone know a proof that the only joint distribution that can be
> both spherically symmetric and such that the corresponding RVs are
> independent is gaussian distribution? Or equivalently proof for that the
> only separable spherically symmetric 2d (or kd) kernel is gaussian.

For the two-dimensional case, a proof is given in Chapter 6 of
Sheldon Ross's A First Course in Probability, Prentice-Hall.

dvsarwate@gmail.com wrote:

> Jani Huhtanen asked:
> 
>> BTW. Does anyone know a proof that the only joint distribution that can
>> be both spherically symmetric and such that the corresponding RVs are
>> independent is gaussian distribution? Or equivalently proof for that the
>> only separable spherically symmetric 2d (or kd) kernel is gaussian.
> 
> For the two-dimensional case, a proof is given in Chapter 6 of
> Sheldon Ross's A First Course in Probability, Prentice-Hall.

We don't have that in our library. Could you outline the approach in the
prove?

-- 
Jani Huhtanen
Tampere University of Technology, Pori

> R.V. are independent if the joint PDF of X and Y=  (PDF of X) * ( PDF
> of Y). This is the direct result of the fact that if X and Y are
> independent than conditioning does not change the PDF.

Yes thats what I meant by:

P(X,Y,Z,.....) = P(X)P(Y)P(Z)..... <-----> independence ------->
uncorrelated

However, the relation btw

>
> R.V are uncorrelated if Expectation {XY} =E{x}E{Y}.

and

spherical symmetry of joint pdf P(X,Y,Z,....) ----> uncorrelated (rxy,
ryz, rxz,..... all are dirac functions)

is not so clear!!! Any thoughts.

Again,

> As mentioned,
> independence is a stronger condition which implied uncorrelatedness
> (but not vice versa).
>

is corroborated by

P(X,Y,Z,.....) = P(X)P(Y)P(Z)..... <-----> independence ------->
uncorrelated

from my last post

(See
http://groups.google.com/group/comp.dsp/msg/da1f0b7a66e44138?hl=en&)

Partho

Ben wrote:

> 
>> R.V. are independent if the joint PDF of X and Y=  (PDF of X) * ( PDF
>> of Y). This is the direct result of the fact that if X and Y are
>> independent than conditioning does not change the PDF.
> 
> Yes thats what I meant by:
> 
> P(X,Y,Z,.....) = P(X)P(Y)P(Z)..... <-----> independence ------->
> uncorrelated
> 
> However, the relation btw
> 
>>
>> R.V are uncorrelated if Expectation {XY} =E{x}E{Y}.
> 
> and
> 
> spherical symmetry of joint pdf P(X,Y,Z,....) ----> uncorrelated (rxy,
> ryz, rxz,..... all are dirac functions)
> 
> is not so clear!!! Any thoughts.
> 

I was actually a bit sloppy. Spherical symmetry and zero mean imply
uncorrelatedness, but spherical symmetry by itself does not. Further
spherical symmetry implies equal variances of the RVs. For now lets assume
that the elements of random vector x = (x0, x1, ..., xn)^T have variances
equal to 1 and that they have zero mean.

If the elements of the x are not correlated then the covariance matrix is:

        E{x*x^T} = I

and any arbitrary rotation of the vector x results in uncorrelated random
vector y:

        y = A*x, where A is orthogonal (orthonormal basis)
        E{y*y^T} = A*E{x*x^T}*A^T = A*I*A^T = I

That is, uncorrelatedness is invariant under rotation. If the elements of
the x are correlated with each other then

        E{x*x^T} = B

and
        
        y = A*x, where A is orthogonal (orthonormal basis)
        E{y*y^T} = A*B*A^T = C =/= B.

Thus if the elemenst of the vector x are correlated, the correlation is
variant under rotation. 

Spherically symmetric distribution is invariant under rotation. The
correlation between the elements of spherically symmetrically distributed
random vector do not change under rotation. Thus, if the distribution is
spherically symmetric the corresponding random variables have to be
uncorrelated.

Please correct me if I was inaccurate or made a mistake.

-- 
Jani Huhtanen
Tampere University of Technology, Pori