xcorr - biased and unbiased

Started by puckman June 12, 2008
hello,
Could someone please tell me what is the difference between biased and
unbiased? And when do I use biased or unbiased? or any restriction to use
one of each? 

I was trying to implement Linear Predictor by using autocorrelation.
When I tested with sine wave, biased correlation provides a better shape,
but has a scaling problem in a peak of sinewave.
Unbiased has a less scaling problem, but for some position, the prediction

is extremely bad. 

Thank you.

David


On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote:
> hello, > Could someone please tell me what is the difference between biased and > unbiased? And when do I use biased or unbiased? or any restriction to use > one of each?
The biased correlation estimator is biased. The expression looks something like this (not checking details and using autocorrelation, as the details don't get so messy): 1 N-1 N-|k| rxx[k] = --- sum ------ x[n+k]x[k] N n=0 N The (N-|k|)/N term scales the elements in rxx[k] differently as a function of k: E[rxx[0]] = x^2[0] E[rxx[N-1]] = 1/N x^2[N-1] Since in general E[rxx[k]] =/= x^2[n] the estimator rxx[k] is biased when x[n] is stationary. The unbiased estimator rxx'[k], on the other hand, doesn't have that scaling term: 1 N-1 rxx'[k] = --- sum x[n+k]x[k] N n=0 In this case there is no svclaing term inside the sum, so E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. The difference is that the biased estimator has bounded variance whereas the unbiased estimater has not.
> I was trying to implement Linear Predictor by using autocorrelation. > When I tested with sine wave, biased correlation provides a better shape, > but has a scaling problem in a peak of sinewave. > Unbiased has a less scaling problem, but for some position, the prediction > > is extremely bad.
This is because the variance is unbounded. This basically means that results and predictions based on the unbiased estimator can become unstable. One uses the biased estimator as a matetr of course unless one has a very specific and justified reason not to. The justification would be that the bias introduced by the stable estimator is a worse problem than the instability of the unbiased estimator. I have yet to see a real-life case where that actually happens. Rune
Thank you very much, Rune,for kindly answer to my question!!

I am afraid that if this is trivial question, but I still don't have
confidence.

1. How could I possibly prove if the variance of the specific signal is
bounded or not?

2. "The justification would be that the bias introduced by the
stable estimator is a worse problem than the instability of the unbiased
estimator." 

Is there any example or published paper describing the use of unbiased
estimator? 

Thank you very much.

David

>On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote: >> hello, >> Could someone please tell me what is the difference between biased and >> unbiased? And when do I use biased or unbiased? or any restriction to
use
>> one of each? > >The biased correlation estimator is biased. The expression >looks something like this (not checking details and using >autocorrelation, as the details don't get so messy): > > 1 N-1 N-|k| >rxx[k] = --- sum ------ x[n+k]x[k] > N n=0 N > >The (N-|k|)/N term scales the elements in rxx[k] differently >as a function of k: > >E[rxx[0]] = x^2[0] >E[rxx[N-1]] = 1/N x^2[N-1] > >Since in general E[rxx[k]] =/= x^2[n] the estimator >rxx[k] is biased when x[n] is stationary. > >The unbiased estimator rxx'[k], on the other hand, doesn't >have that scaling term: > > 1 N-1 >rxx'[k] = --- sum x[n+k]x[k] > N n=0 > >In this case there is no svclaing term inside the sum, so >E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. > >The difference is that the biased estimator has bounded >variance whereas the unbiased estimater has not. > >> I was trying to implement Linear Predictor by using autocorrelation. >> When I tested with sine wave, biased correlation provides a better
shape,
>> but has a scaling problem in a peak of sinewave. >> Unbiased has a less scaling problem, but for some position, the
prediction
>> >> is extremely bad. > >This is because the variance is unbounded. This basically >means that results and predictions based on the unbiased >estimator can become unstable. > >One uses the biased estimator as a matetr of course unless >one has a very specific and justified reason not to. The >justification would be that the bias introduced by the >stable estimator is a worse problem than the instability >of the unbiased estimator. > >I have yet to see a real-life case where that actually >happens. > >Rune >
On 12 Jun, 21:08, "puckman" <kwa...@hotmail.com> wrote:
> Thank you very much, Rune,for kindly answer to my question!! > > I am afraid that if this is trivial question, but I still don't have > confidence. > > 1. How could I possibly prove if the variance of the specific signal is > bounded or not?
It's been 15 years since I looked into those proofs, in chapter 6 in the book by Therrien. Can't remember any other references off the top of my head, but I wouldn't be surprised if they can be found in some book by Kay.
> 2. "The justification would be that the bias introduced by the > stable estimator is a worse problem than the instability of the unbiased > estimator." > > Is there any example
You say the results are bad when you use it in your own work, that ought to be example enough...
> or published paper describing the use of unbiased > estimator?
...not that I can think of; people tend to publish the good or useful results. The best you can hope for is an example or exercise in the type of book that contains the formal discussion of the variance. Rune
Thank you for all your help!!~

David
Thank you for all your help!!~

David
On 12 Jun, 21:08, "puckman" <kwa...@hotmail.com> wrote:
> Thank you very much, Rune,for kindly answer to my question!! > > I am afraid that if this is trivial question, but I still don't have > confidence. > > 1. How could I possibly prove if the variance of the specific signal is > bounded or not?
It's been 15 years since I looked into those proofs, in chapter 6 in the book by Therrien. Can't remember any other references off the top of my head, but I wouldn't be surprised if they can be found in some book by Kay.
> 2. "The justification would be that the bias introduced by the > stable estimator is a worse problem than the instability of the unbiased > estimator." > > Is there any example
You say the results are bad when you use it in your own work, that ought to be example enough...
> or published paper describing the use of unbiased > estimator?
...not that I can think of; people tend to publish the good or useful results. The best you can hope for is an example or exercise in the type of book that contains the formal discussion of the variance. Rune
Thank you very much, Rune,for kindly answer to my question!!

I am afraid that if this is trivial question, but I still don't have
confidence.

1. How could I possibly prove if the variance of the specific signal is
bounded or not?

2. "The justification would be that the bias introduced by the
stable estimator is a worse problem than the instability of the unbiased
estimator." 

Is there any example or published paper describing the use of unbiased
estimator? 

Thank you very much.

David

>On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote: >> hello, >> Could someone please tell me what is the difference between biased and >> unbiased? And when do I use biased or unbiased? or any restriction to
use
>> one of each? > >The biased correlation estimator is biased. The expression >looks something like this (not checking details and using >autocorrelation, as the details don't get so messy): > > 1 N-1 N-|k| >rxx[k] = --- sum ------ x[n+k]x[k] > N n=0 N > >The (N-|k|)/N term scales the elements in rxx[k] differently >as a function of k: > >E[rxx[0]] = x^2[0] >E[rxx[N-1]] = 1/N x^2[N-1] > >Since in general E[rxx[k]] =/= x^2[n] the estimator >rxx[k] is biased when x[n] is stationary. > >The unbiased estimator rxx'[k], on the other hand, doesn't >have that scaling term: > > 1 N-1 >rxx'[k] = --- sum x[n+k]x[k] > N n=0 > >In this case there is no svclaing term inside the sum, so >E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. > >The difference is that the biased estimator has bounded >variance whereas the unbiased estimater has not. > >> I was trying to implement Linear Predictor by using autocorrelation. >> When I tested with sine wave, biased correlation provides a better
shape,
>> but has a scaling problem in a peak of sinewave. >> Unbiased has a less scaling problem, but for some position, the
prediction
>> >> is extremely bad. > >This is because the variance is unbounded. This basically >means that results and predictions based on the unbiased >estimator can become unstable. > >One uses the biased estimator as a matetr of course unless >one has a very specific and justified reason not to. The >justification would be that the bias introduced by the >stable estimator is a worse problem than the instability >of the unbiased estimator. > >I have yet to see a real-life case where that actually >happens. > >Rune >
On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote:
> hello, > Could someone please tell me what is the difference between biased and > unbiased? And when do I use biased or unbiased? or any restriction to use > one of each?
The biased correlation estimator is biased. The expression looks something like this (not checking details and using autocorrelation, as the details don't get so messy): 1 N-1 N-|k| rxx[k] = --- sum ------ x[n+k]x[k] N n=0 N The (N-|k|)/N term scales the elements in rxx[k] differently as a function of k: E[rxx[0]] = x^2[0] E[rxx[N-1]] = 1/N x^2[N-1] Since in general E[rxx[k]] =/= x^2[n] the estimator rxx[k] is biased when x[n] is stationary. The unbiased estimator rxx'[k], on the other hand, doesn't have that scaling term: 1 N-1 rxx'[k] = --- sum x[n+k]x[k] N n=0 In this case there is no svclaing term inside the sum, so E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. The difference is that the biased estimator has bounded variance whereas the unbiased estimater has not.
> I was trying to implement Linear Predictor by using autocorrelation. > When I tested with sine wave, biased correlation provides a better shape, > but has a scaling problem in a peak of sinewave. > Unbiased has a less scaling problem, but for some position, the prediction > > is extremely bad.
This is because the variance is unbounded. This basically means that results and predictions based on the unbiased estimator can become unstable. One uses the biased estimator as a matetr of course unless one has a very specific and justified reason not to. The justification would be that the bias introduced by the stable estimator is a worse problem than the instability of the unbiased estimator. I have yet to see a real-life case where that actually happens. Rune
hello,
Could someone please tell me what is the difference between biased and
unbiased? And when do I use biased or unbiased? or any restriction to use
one of each? 

I was trying to implement Linear Predictor by using autocorrelation.
When I tested with sine wave, biased correlation provides a better shape,
but has a scaling problem in a peak of sinewave.
Unbiased has a less scaling problem, but for some position, the prediction

is extremely bad. 

Thank you.

David