hello, Could someone please tell me what is the difference between biased and unbiased? And when do I use biased or unbiased? or any restriction to use one of each? I was trying to implement Linear Predictor by using autocorrelation. When I tested with sine wave, biased correlation provides a better shape, but has a scaling problem in a peak of sinewave. Unbiased has a less scaling problem, but for some position, the prediction is extremely bad. Thank you. David

# xcorr - biased and unbiased

Started by ●June 12, 2008

Posted by ●June 12, 2008

On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote:> hello, > Could someone please tell me what is the difference between biased and > unbiased? And when do I use biased or unbiased? or any restriction to use > one of each?The biased correlation estimator is biased. The expression looks something like this (not checking details and using autocorrelation, as the details don't get so messy): 1 N-1 N-|k| rxx[k] = --- sum ------ x[n+k]x[k] N n=0 N The (N-|k|)/N term scales the elements in rxx[k] differently as a function of k: E[rxx[0]] = x^2[0] E[rxx[N-1]] = 1/N x^2[N-1] Since in general E[rxx[k]] =/= x^2[n] the estimator rxx[k] is biased when x[n] is stationary. The unbiased estimator rxx'[k], on the other hand, doesn't have that scaling term: 1 N-1 rxx'[k] = --- sum x[n+k]x[k] N n=0 In this case there is no svclaing term inside the sum, so E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. The difference is that the biased estimator has bounded variance whereas the unbiased estimater has not.> I was trying to implement Linear Predictor by using autocorrelation. > When I tested with sine wave, biased correlation provides a better shape, > but has a scaling problem in a peak of sinewave. > Unbiased has a less scaling problem, but for some position, the prediction > > is extremely bad.This is because the variance is unbounded. This basically means that results and predictions based on the unbiased estimator can become unstable. One uses the biased estimator as a matetr of course unless one has a very specific and justified reason not to. The justification would be that the bias introduced by the stable estimator is a worse problem than the instability of the unbiased estimator. I have yet to see a real-life case where that actually happens. Rune

Posted by ●June 12, 2008

Thank you very much, Rune,for kindly answer to my question!! I am afraid that if this is trivial question, but I still don't have confidence. 1. How could I possibly prove if the variance of the specific signal is bounded or not? 2. "The justification would be that the bias introduced by the stable estimator is a worse problem than the instability of the unbiased estimator." Is there any example or published paper describing the use of unbiased estimator? Thank you very much. David>On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote: >> hello, >> Could someone please tell me what is the difference between biased and >> unbiased? And when do I use biased or unbiased? or any restriction touse>> one of each? > >The biased correlation estimator is biased. The expression >looks something like this (not checking details and using >autocorrelation, as the details don't get so messy): > > 1 N-1 N-|k| >rxx[k] = --- sum ------ x[n+k]x[k] > N n=0 N > >The (N-|k|)/N term scales the elements in rxx[k] differently >as a function of k: > >E[rxx[0]] = x^2[0] >E[rxx[N-1]] = 1/N x^2[N-1] > >Since in general E[rxx[k]] =/= x^2[n] the estimator >rxx[k] is biased when x[n] is stationary. > >The unbiased estimator rxx'[k], on the other hand, doesn't >have that scaling term: > > 1 N-1 >rxx'[k] = --- sum x[n+k]x[k] > N n=0 > >In this case there is no svclaing term inside the sum, so >E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. > >The difference is that the biased estimator has bounded >variance whereas the unbiased estimater has not. > >> I was trying to implement Linear Predictor by using autocorrelation. >> When I tested with sine wave, biased correlation provides a bettershape,>> but has a scaling problem in a peak of sinewave. >> Unbiased has a less scaling problem, but for some position, theprediction>> >> is extremely bad. > >This is because the variance is unbounded. This basically >means that results and predictions based on the unbiased >estimator can become unstable. > >One uses the biased estimator as a matetr of course unless >one has a very specific and justified reason not to. The >justification would be that the bias introduced by the >stable estimator is a worse problem than the instability >of the unbiased estimator. > >I have yet to see a real-life case where that actually >happens. > >Rune >

Posted by ●June 12, 2008

On 12 Jun, 21:08, "puckman" <kwa...@hotmail.com> wrote:> Thank you very much, Rune,for kindly answer to my question!! > > I am afraid that if this is trivial question, but I still don't have > confidence. > > 1. How could I possibly prove if the variance of the specific signal is > bounded or not?It's been 15 years since I looked into those proofs, in chapter 6 in the book by Therrien. Can't remember any other references off the top of my head, but I wouldn't be surprised if they can be found in some book by Kay.> 2. "The justification would be that the bias introduced by the > stable estimator is a worse problem than the instability of the unbiased > estimator." > > Is there any exampleYou say the results are bad when you use it in your own work, that ought to be example enough...> or published paper describing the use of unbiased > estimator?...not that I can think of; people tend to publish the good or useful results. The best you can hope for is an example or exercise in the type of book that contains the formal discussion of the variance. Rune

Posted by ●June 13, 2008

Posted by ●June 13, 2008

Posted by ●June 12, 2008

On 12 Jun, 21:08, "puckman" <kwa...@hotmail.com> wrote:> Thank you very much, Rune,for kindly answer to my question!! > > I am afraid that if this is trivial question, but I still don't have > confidence. > > 1. How could I possibly prove if the variance of the specific signal is > bounded or not?It's been 15 years since I looked into those proofs, in chapter 6 in the book by Therrien. Can't remember any other references off the top of my head, but I wouldn't be surprised if they can be found in some book by Kay.> 2. "The justification would be that the bias introduced by the > stable estimator is a worse problem than the instability of the unbiased > estimator." > > Is there any exampleYou say the results are bad when you use it in your own work, that ought to be example enough...> or published paper describing the use of unbiased > estimator?...not that I can think of; people tend to publish the good or useful results. The best you can hope for is an example or exercise in the type of book that contains the formal discussion of the variance. Rune

Posted by ●June 12, 2008

Thank you very much, Rune,for kindly answer to my question!! I am afraid that if this is trivial question, but I still don't have confidence. 1. How could I possibly prove if the variance of the specific signal is bounded or not? 2. "The justification would be that the bias introduced by the stable estimator is a worse problem than the instability of the unbiased estimator." Is there any example or published paper describing the use of unbiased estimator? Thank you very much. David>On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote: >> hello, >> Could someone please tell me what is the difference between biased and >> unbiased? And when do I use biased or unbiased? or any restriction touse>> one of each? > >The biased correlation estimator is biased. The expression >looks something like this (not checking details and using >autocorrelation, as the details don't get so messy): > > 1 N-1 N-|k| >rxx[k] = --- sum ------ x[n+k]x[k] > N n=0 N > >The (N-|k|)/N term scales the elements in rxx[k] differently >as a function of k: > >E[rxx[0]] = x^2[0] >E[rxx[N-1]] = 1/N x^2[N-1] > >Since in general E[rxx[k]] =/= x^2[n] the estimator >rxx[k] is biased when x[n] is stationary. > >The unbiased estimator rxx'[k], on the other hand, doesn't >have that scaling term: > > 1 N-1 >rxx'[k] = --- sum x[n+k]x[k] > N n=0 > >In this case there is no svclaing term inside the sum, so >E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. > >The difference is that the biased estimator has bounded >variance whereas the unbiased estimater has not. > >> I was trying to implement Linear Predictor by using autocorrelation. >> When I tested with sine wave, biased correlation provides a bettershape,>> but has a scaling problem in a peak of sinewave. >> Unbiased has a less scaling problem, but for some position, theprediction>> >> is extremely bad. > >This is because the variance is unbounded. This basically >means that results and predictions based on the unbiased >estimator can become unstable. > >One uses the biased estimator as a matetr of course unless >one has a very specific and justified reason not to. The >justification would be that the bias introduced by the >stable estimator is a worse problem than the instability >of the unbiased estimator. > >I have yet to see a real-life case where that actually >happens. > >Rune >

Posted by ●June 12, 2008

On 12 Jun, 13:27, "puckman" <kwa...@hotmail.com> wrote:> hello, > Could someone please tell me what is the difference between biased and > unbiased? And when do I use biased or unbiased? or any restriction to use > one of each?The biased correlation estimator is biased. The expression looks something like this (not checking details and using autocorrelation, as the details don't get so messy): 1 N-1 N-|k| rxx[k] = --- sum ------ x[n+k]x[k] N n=0 N The (N-|k|)/N term scales the elements in rxx[k] differently as a function of k: E[rxx[0]] = x^2[0] E[rxx[N-1]] = 1/N x^2[N-1] Since in general E[rxx[k]] =/= x^2[n] the estimator rxx[k] is biased when x[n] is stationary. The unbiased estimator rxx'[k], on the other hand, doesn't have that scaling term: 1 N-1 rxx'[k] = --- sum x[n+k]x[k] N n=0 In this case there is no svclaing term inside the sum, so E[rxx'[k]] = x^2[n] and rxx'[k] is unbiased. The difference is that the biased estimator has bounded variance whereas the unbiased estimater has not.> I was trying to implement Linear Predictor by using autocorrelation. > When I tested with sine wave, biased correlation provides a better shape, > but has a scaling problem in a peak of sinewave. > Unbiased has a less scaling problem, but for some position, the prediction > > is extremely bad.This is because the variance is unbounded. This basically means that results and predictions based on the unbiased estimator can become unstable. One uses the biased estimator as a matetr of course unless one has a very specific and justified reason not to. The justification would be that the bias introduced by the stable estimator is a worse problem than the instability of the unbiased estimator. I have yet to see a real-life case where that actually happens. Rune

Posted by ●June 12, 2008

hello, Could someone please tell me what is the difference between biased and unbiased? And when do I use biased or unbiased? or any restriction to use one of each? I was trying to implement Linear Predictor by using autocorrelation. When I tested with sine wave, biased correlation provides a better shape, but has a scaling problem in a peak of sinewave. Unbiased has a less scaling problem, but for some position, the prediction is extremely bad. Thank you. David