On 10/24/2010 02:39 PM, Muzaffer Kal wrote:> On Sun, 24 Oct 2010 17:20:33 -0400, Randy Yates<yates@ieee.org> > wrote: > >> Bryan<bryan.paul@gmail.com> writes: >> >>> On Oct 24, 9:33 am, Randy Yates<ya...@ieee.org> wrote: >>>> Tim Wescott<t...@seemywebsite.com> writes: >>>>> [...] >>>>> Try getting yourself a much bigger sample -- 1000 points would still >>>>> be small >>>> >>>> Tim, what criteria do you use for this statement? Is it just >>>> experience, or is there some analytical basis? >>>> -- >>>> Randy Yates % "I met someone who looks alot like you, >>>> Digital Signal Labs % she does the things you do, >>>> mailto://ya...@ieee.org % but she is an IBM."http://www.digitalsignallabs.com% 'Yours Truly, 2095', *Time*, ELO >>> >>> See 'Law of large numbers' >> >> Yeah, but how large is "large?" > > The "larger", the "better"OK, fine. Y'all made me go look in my book ("Mathematical Statistics", Freund/Walpole, Prentice Hall, 1987) If you estimate the variance of a vector of n samples of a normal random process with mean = 0 and variance = 1, then the variance of your variance estimate will be sqrt(2/(n-1)). That doesn't fall off all that rapidly -- for 100 samples your variance will be 0.1, to reduce the variance by another factor of ten requires 10000 samples, etc. So: larger, better. Really really larger, really better. Really really really really larger -- really really better. Progress is slow. Back in the late 1800's, early 1900's, experimenters tried to broadcast voice transmissions by amplitude modulating spark-gap transmitters. They succeeded in the amplitude modulation part, but failed at the broadcasting understandable voice part -- pondering the above relationships, and considering the bandwidth necessary to convey even a 3kHz wide signal with any degree of fidelity, may tell you why. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" was written for you. See details at http://www.wescottdesign.com/actfes/actfes.html
A test for stationarity assumption
Started by ●October 23, 2010
Reply by ●October 24, 20102010-10-24
Reply by ●October 24, 20102010-10-24
On Oct 24, 2:39=A0pm, Muzaffer Kal <k...@dspia.com> wrote:> On Sun, 24 Oct 2010 17:20:33 -0400, Randy Yates <ya...@ieee.org> > wrote: > > > > > > > > > > >Bryan <bryan.p...@gmail.com> writes: > > >> On Oct 24, 9:33=A0am, Randy Yates <ya...@ieee.org> wrote: > >>> Tim Wescott <t...@seemywebsite.com> writes: > >>> > [...] > >>> > Try getting yourself a much bigger sample -- 1000 points would stil=l> >>> > be small > > >>> Tim, what criteria do you use for this statement? Is it just > >>> experience, or is there some analytical basis? > >>> -- > >>> Randy Yates =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0% "I met someo=ne who looks alot like you,> >>> Digital Signal Labs =A0 =A0 =A0 =A0 =A0 =A0 =A0% =A0 =A0 =A0 =A0 =A0 ==A0 she does the things you do,> >>> mailto://ya...@ieee.org =A0 =A0 =A0 =A0 =A0% =A0 =A0 =A0 =A0 =A0 =A0 ==A0 =A0 =A0 =A0 but she is an IBM."http://www.digitalsignallabs.com%=A0 =A0= =A0 =A0'Yours Truly, 2095', *Time*, ELO =A0> > >> See 'Law of large numbers' > > >Yeah, but how large is "large?" > > The "larger", the "better" > -- > Muzaffer Kal > > DSPIA INC. > ASIC/FPGA Design Services > > http://www.dspia.comExactly, so testing with known distributions to compare with theoretical knowledge is the best way to determine roughly what is needed. For example, you may find it converges to the theoretical variance within a certain margin of error that you deem satisfactory after 100 samples. Of course, be mindful of the distribution's moments prior to attempting estimation to ensure convergence. For example, the Cauchy distribution's mean is undefined, so by definition, it has no higher order moments defined and thus the variance does not exist as well.
Reply by ●October 24, 20102010-10-24
On Oct 24, 5:19=A0pm, Tim Wescott <t...@seemywebsite.com> wrote:> > OK, fine. =A0Y'all made me go look in my book ("Mathematical Statistics", > Freund/Walpole, Prentice Hall, 1987) > > If you estimate the variance of a vector of n samples of a normal random > process with mean =3D 0 and variance =3D 1, then the variance of your > variance estimate will be sqrt(2/(n-1)). >Does the MATLAB function rand() return normal (i.e., Gaussian) random variables or uniformly distributed random variables in the interval (0, 1) ? If the latter, the variance of the variance estimate will behave somewhat differently (though I believe (as a WAG) that it will still be something like sqrt(b/(n-1)) where b will not be 2 as with normal variables). --Dilip nonMATLABi
Reply by ●October 24, 20102010-10-24
On Oct 24, 4:32=A0pm, dvsarwate <dvsarw...@yahoo.com> wrote:> On Oct 24, 5:19=A0pm, Tim Wescott <t...@seemywebsite.com> wrote: > > > > > OK, fine. =A0Y'all made me go look in my book ("Mathematical Statistics=",> > Freund/Walpole, Prentice Hall, 1987) > > > If you estimate the variance of a vector of n samples of a normal rando=m> > process with mean =3D 0 and variance =3D 1, then the variance of your > > variance estimate will be sqrt(2/(n-1)). > > Does the MATLAB function rand() return normal > (i.e., Gaussian) random variables or uniformly > distributed random variables in the interval (0, 1) ? > If the latter, the variance of the variance estimate > will behave somewhat differently (though I believe > (as a WAG) that it will still be something like > sqrt(b/(n-1)) where b will not be 2 as with normal > variables). > > --Dilip nonMATLABiIndeed, 'rand()' is uniformly distributed between 0 and 1. 'randn()' is normally distributed with the parameters (0, 1) for the mean and variance respectively.
Reply by ●October 24, 20102010-10-24
On Oct 24, 6:41=A0pm, Bryan <bryan.p...@gmail.com> wrote:> Indeed, 'rand()' is uniformly distributed between 0 and 1. 'randn()' > is normally distributed with the parameters (0, 1) for the mean and > variance respectively.And so the variance that is to be estimated is 1/12 which is less than 0.1. Since the parameter to be estimated is small, the variance of the estimate needs to be much smaller.... and, as has been noted already, a *large* number of samples would be needed to get a reliable estimate. In particular, the 100 samples used by the OP may not give a variance estimate close to the theoretical value of 1/12. --Dilip Sarwate
Reply by ●October 25, 20102010-10-25
On Oct 24, 6:32=A0pm, dvsarwate <dvsarw...@yahoo.com> quoted Tim Wescott <t...@seemywebsite.com> who consulted a text and wrote:>>........ >> then the variance of your >> variance estimate will be sqrt(2/(n-1)). >>.......I think this should have been "the *standard deviation* of your variance estimate will be ......" As a general rule, variances decrease inversely as n, and standard deviations inversely as sqrt(n). So, I suspect that Tim Wescott made a small mistake in describing what his textbook said. In my previous statement> If the latter, the variance of the variance estimate > will behave somewhat differently (though I believe > (as a WAG) that it will still be something like > sqrt(b/(n-1)) where b will not be 2 as with normal > variables).please replace the *first* occurrence of the word "variance" by "standard deviation". The error is regretted. --Dilip Sarwate
Reply by ●October 25, 20102010-10-25
On 10/25/2010 06:00 AM, dvsarwate wrote:> On Oct 24, 6:32 pm, dvsarwate<dvsarw...@yahoo.com> quoted > Tim Wescott<t...@seemywebsite.com> who consulted a text > and wrote: > >>> ........ >>> then the variance of your >>> variance estimate will be sqrt(2/(n-1)). >>> ....... > > I think this should have been > > "the *standard deviation* of your variance estimate > will be ......" > > As a general rule, variances decrease inversely > as n, and standard deviations inversely as sqrt(n). > So, I suspect that Tim Wescott made a small > mistake in describing what his textbook said. > > In my previous statement > >> If the latter, the variance of the variance estimate >> will behave somewhat differently (though I believe >> (as a WAG) that it will still be something like >> sqrt(b/(n-1)) where b will not be 2 as with normal >> variables). > > please replace the *first* occurrence of the word > "variance" by "standard deviation". The error is > regretted.I always get those terms mixed up. I always know what I'm calculating, but then I open my mouth and confuse people. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" was written for you. See details at http://www.wescottdesign.com/actfes/actfes.html
