On Monday, July 14, 2014 12:54:00 AM UTC-4, Tim Wescott wrote:> On Sun, 13 Jul 2014 18:35:48 -0700, julius wrote: > > > > > On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: > > > > > > > > >> Maybe I'll Monte-Carlo it (blech). > > >> > > >> > > > Most of the time, people who go down this route end up discovering not > > > the answer to their inquiry, rather the quality of their RNG (or lack > > > thereof). > > > > RNG? > > > > -- > > > > Tim Wescott > > Wescott Design Services > > http://www.wescottdesign.comRandom Number Generator. But you already have your answer, which is that distribution has nothing to do with color of the process, assuming stationarity etc.
Max absolute value of colored Gaussian noise
Started by ●July 12, 2014
Reply by ●July 14, 20142014-07-14
Reply by ●July 14, 20142014-07-14
On 7/14/14 10:59 AM, julius wrote:> > Random Number Generator. But you already have your answer, which is that distribution has nothing to do with color of the process, assuming stationarity etc.actually, the p.d.f. and the "color" (which i would assume means "power spectrum" or equivalently "autocorrelation), *are* related (the conditional or joint p.d.f.s determine both overall p.d.f. and autocorrelation) but, using an iterative process, can be independently specified for some RNG. i think if it were me, if i were tasked to produce an RNG with a given p.d.f. and a given power spectrum, i would first try to produce a gaussian-like p.d.f. (like a binomial distribution) with the target power spectrum (by simply filtering it), then run that through a simple monotonic memoryless non-linear function to get the p.d.f. i am seeking, *then* check over a large sample space what the apparent spectrum would be and compare that to the target, and if it deviates from the target spectrum, adjust the specifications of the filter and do it again. Tim, i am digging out some grad school textbooks to see if i can answer your question as stated. L8r, -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
Reply by ●July 14, 20142014-07-14
On Sun, 13 Jul 2014 20:00:37 +0200, Piergiorgio Sartor wrote:> On 2014-07-13 19:42, Tim Wescott wrote: > [...] >> Yes, the resulting probability density will have a tail that extends >> off to infinity. But the probability that any one sample of the vector >> is at infinity is vanishingly small. > > Of course is small, but that's exactly the difference between reality > and theory. > In theory, it will be +inf, in reality it will be well between six > sigma. > >> If we were to apply your attitude, then when we needed to go to the >> store to get eggs, we would look out at the car and say "nope, if I >> stand next to it in the driveway it may not be there because the PDF of >> its distance from its rest position goes to infinity. I guess I'll >> just sit here without eating dinner". THAT is mixing theory with >> reality. > > You are the one saying that the oscilloscope does not show infinity. > That's the reality, the theory is that you will have infinity. > > Objecting that the measure is not +inf *is* > mixing reality and theory.Neither the mean nor the variance is infinity, and as far as I can tell the measure is useful. I don't give a crap about a theoretically infinite tail, if the measure is useful. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 14, 20142014-07-14
On Mon, 14 Jul 2014 07:59:57 -0700, julius wrote:> On Monday, July 14, 2014 12:54:00 AM UTC-4, Tim Wescott wrote: >> On Sun, 13 Jul 2014 18:35:48 -0700, julius wrote: >> >> >> >> > On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: >> >> >> > >> >> > >> >> Maybe I'll Monte-Carlo it (blech). >> >> >> >> >> >> >> >> > Most of the time, people who go down this route end up discovering >> > not >> >> > the answer to their inquiry, rather the quality of their RNG (or lack >> >> > thereof). >> >> >> >> RNG? >> >> >> >> -- >> >> >> >> Tim Wescott >> >> Wescott Design Services >> >> http://www.wescottdesign.com > > Random Number Generator. But you already have your answer, which is that > distribution has nothing to do with color of the process, assuming > stationarity etc.Actually I'm certain that the color of the process does impact the end measurement -- the measurement mean certainly rises the longer a vector of independent random variables you use; this indicates to me that the more correlated the variables are, the lower the mean is going to be, since the vector is going to act like it has fewer independent variables. The language is wishy-washy, but I hope you can get the sense through the hand waving. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 14, 20142014-07-14
On Sunday, July 13, 2014 5:46:15 PM UTC+12, Tim Wescott wrote:> On Sat, 12 Jul 2014 21:47:51 -0400, robert bristow-johnson wrote: > > > > > On 7/12/14 6:03 PM, Tim Wescott wrote: > > >> I need a number, and I'm feeling lazy; has anyone worked this out > > >> recently? > > >> > > >> I want to know the distribution of the maximum of the absolute value of > > >> a vector of samples of a colored, zero-mean Gaussian process. > > >> > > >> Or, stated another way, I want to shove white noise into a filter, then > > >> examine a finite chunk of the filter output for it's maximum absolute > > >> value. > > >> > > >> Anyone know the answer? Or should I sharpen my pencil and get to work? > > >> > > >> > > > well, one thing, Tim, is that without some additional restriction about > > > how the process that colors the Gaussian noise and how the gaussian > > > p.d.f. random variable is generated. like we know that adding together > > > 12 independent uniform p.d.f. (1 unit width) random variables together > > > get a binomial distribution that well approximates a gaussian of 1 unit > > > variance. and (assuming zero mean), it never gets beyond -6 and +6. > > > > > > so maybe you wanna know where the 99% likelihood limit level is. or > > > something like that. > > > > No, I really am looking for mean and variance. And I think the only > > missing restriction, which ought to be obvious, is that the coloration on > > the noise must have a finite bandwidth (i.e., if you model the process as > > white noise feeding a filter, the filter must have a finite noise > > bandwidth). > > > > But, clearly you haven't done the math on this any time recently either. > > > > -- > > Tim Wescott > > Control system and signal processing consulting > > www.wescottdesign.comIt's mean is easy to find - just like a step input, the variance depends on whether the colouring process is IIR or FIR. For FIR it's easy, the variance is just the sum of squares of the filter coefficients times the variance of the input. For IIR you need one of those contour integrals or a table of integrals around the unit circle. For 2nd order it is quite easy to find.
Reply by ●July 14, 20142014-07-14
On Mon, 14 Jul 2014 11:49:20 -0700, gyansorova wrote:> On Sunday, July 13, 2014 5:46:15 PM UTC+12, Tim Wescott wrote: >> On Sat, 12 Jul 2014 21:47:51 -0400, robert bristow-johnson wrote: >> >> >> >> > On 7/12/14 6:03 PM, Tim Wescott wrote: >> >> >> I need a number, and I'm feeling lazy; has anyone worked this out >> >> >> recently? >> >> >> >> >> >> I want to know the distribution of the maximum of the absolute value >> >> of >> >> >> a vector of samples of a colored, zero-mean Gaussian process. >> >> >> >> >> >> Or, stated another way, I want to shove white noise into a filter, >> >> then >> >> >> examine a finite chunk of the filter output for it's maximum >> >> absolute >> >> >> value. >> >> >> >> >> >> Anyone know the answer? Or should I sharpen my pencil and get to >> >> work? >> >> >> >> >> >> >> >> > well, one thing, Tim, is that without some additional restriction >> > about >> >> > how the process that colors the Gaussian noise and how the gaussian >> >> > p.d.f. random variable is generated. like we know that adding >> > together >> >> > 12 independent uniform p.d.f. (1 unit width) random variables >> > together >> >> > get a binomial distribution that well approximates a gaussian of 1 >> > unit >> >> > variance. and (assuming zero mean), it never gets beyond -6 and +6. >> >> >> > >> > so maybe you wanna know where the 99% likelihood limit level is. or >> >> > something like that. >> >> >> >> No, I really am looking for mean and variance. And I think the only >> >> missing restriction, which ought to be obvious, is that the coloration >> on >> >> the noise must have a finite bandwidth (i.e., if you model the process >> as >> >> white noise feeding a filter, the filter must have a finite noise >> >> bandwidth). >> >> >> >> But, clearly you haven't done the math on this any time recently >> either. >> >> >> >> -- >> >> Tim Wescott >> >> Control system and signal processing consulting >> >> www.wescottdesign.com > > It's mean is easy to find - just like a step input, the variance depends > on whether the colouring process is IIR or FIR. For FIR it's easy, the > variance is just the sum of squares of the filter coefficients times the > variance of the input. For IIR you need one of those contour integrals > or a table of integrals around the unit circle. For 2nd order it is > quite easy to find.The metric I'm taking is the maximum of the absolute value of the elements of the vector. Not the variance of each element. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 14, 20142014-07-14
On Mon, 14 Jul 2014 12:31:45 -0500, Tim Wescott <tim@seemywebsite.really> wrote:>On Mon, 14 Jul 2014 07:59:57 -0700, julius wrote: > >> On Monday, July 14, 2014 12:54:00 AM UTC-4, Tim Wescott wrote: >>> On Sun, 13 Jul 2014 18:35:48 -0700, julius wrote: >>> >>> >>> >>> > On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: >>> >>> >>> > >>> >>> > >>> >> Maybe I'll Monte-Carlo it (blech). >>> >>> >>> >> >>> >>> >> >>> > Most of the time, people who go down this route end up discovering >>> > not >>> >>> > the answer to their inquiry, rather the quality of their RNG (or lack >>> >>> > thereof). >>> >>> >>> >>> RNG? >>> >>> >>> >>> -- >>> >>> >>> >>> Tim Wescott >>> >>> Wescott Design Services >>> >>> http://www.wescottdesign.com >> >> Random Number Generator. But you already have your answer, which is that >> distribution has nothing to do with color of the process, assuming >> stationarity etc. > >Actually I'm certain that the color of the process does impact the end >measurement -- the measurement mean certainly rises the longer a vector >of independent random variables you use; this indicates to me that the >more correlated the variables are, the lower the mean is going to be, >since the vector is going to act like it has fewer independent variables. > >The language is wishy-washy, but I hope you can get the sense through the >hand waving.To be honest, I've had a very difficult time following what you've written in this thread. I'm a little surprised that other people have apparently been able to sort some of it out. What you wrote above doesn't parse for me at all. Mean of what? The vector? How is that at all dependent on the length of the vector if the process is stationary? Correlation or dependence drives the mean? No grok.>-- > >Tim Wescott >Wescott Design Services >http://www.wescottdesign.com >Eric Jacobsen Anchor Hill Communications http://www.anchorhill.com
Reply by ●July 14, 20142014-07-14
Tim Wescott <tim@seemywebsite.please> writes:> [...] > But, clearly you haven't done the math on this any time recently either.It's not my math, it's my reading. My mind skipped the word "distribution" in your original post. I.e., the DISTRIBUTION of the maximum value of blah blah. Which is admittedly a pretty serious error to make. Meh, maybe I was tired.. -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
Reply by ●July 14, 20142014-07-14
On Mon, 14 Jul 2014 19:03:30 +0000, Eric Jacobsen wrote:> On Mon, 14 Jul 2014 12:31:45 -0500, Tim Wescott > <tim@seemywebsite.really> wrote: > >>On Mon, 14 Jul 2014 07:59:57 -0700, julius wrote: >> >>> On Monday, July 14, 2014 12:54:00 AM UTC-4, Tim Wescott wrote: >>>> On Sun, 13 Jul 2014 18:35:48 -0700, julius wrote: >>>> >>>> >>>> >>>> > On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: >>>> >>>> >>>> >>>> > >>>> >>>> >> Maybe I'll Monte-Carlo it (blech). >>>> >>>> >>>> >>>> >> >>>> >>>> > Most of the time, people who go down this route end up discovering >>>> > not >>>> >>>> > the answer to their inquiry, rather the quality of their RNG (or >>>> > lack >>>> >>>> > thereof). >>>> >>>> >>>> >>>> RNG? >>>> >>>> >>>> >>>> -- >>>> >>>> >>>> >>>> Tim Wescott >>>> >>>> Wescott Design Services >>>> >>>> http://www.wescottdesign.com >>> >>> Random Number Generator. But you already have your answer, which is >>> that distribution has nothing to do with color of the process, >>> assuming stationarity etc. >> >>Actually I'm certain that the color of the process does impact the end >>measurement -- the measurement mean certainly rises the longer a vector >>of independent random variables you use; this indicates to me that the >>more correlated the variables are, the lower the mean is going to be, >>since the vector is going to act like it has fewer independent >>variables. >> >>The language is wishy-washy, but I hope you can get the sense through >>the hand waving. > > To be honest, I've had a very difficult time following what you've > written in this thread. I'm a little surprised that other people have > apparently been able to sort some of it out. > > What you wrote above doesn't parse for me at all. > > Mean of what? The vector? How is that at all dependent on the length > of the vector if the process is stationary? > > Correlation or dependence drives the mean? > > No grok.My goal is to know the probability distribution of a measure of the "magnitude" of noise, said measure being the maximum absolute value found in a vector of samples of a colored (i.e., filtered white noise, autocorrelation != impulse, etc.) random process. It is the mean (and variance, and pdf if I can get it) of this measure that I am seeking. The mean of such a metric DOES go up as the number of points in the vector go up, and you can see how it does intuitively: the more points you include, the more chances there are for the vector to have one really high-magnitude member. I'm pretty sure the variance falls with increasing vector length, but I'm not sure. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 14, 20142014-07-14
Tim Wescott <tim@seemywebsite.really> writes:> On Mon, 14 Jul 2014 19:03:30 +0000, Eric Jacobsen wrote: > >> On Mon, 14 Jul 2014 12:31:45 -0500, Tim Wescott >> <tim@seemywebsite.really> wrote: >> >>>On Mon, 14 Jul 2014 07:59:57 -0700, julius wrote: >>> >>>> On Monday, July 14, 2014 12:54:00 AM UTC-4, Tim Wescott wrote: >>>>> On Sun, 13 Jul 2014 18:35:48 -0700, julius wrote: >>>>> >>>>> >>>>> >>>>> > On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: >>>>> >>>>> >>>>> >>>>> > >>>>> >>>>> >> Maybe I'll Monte-Carlo it (blech). >>>>> >>>>> >>>>> >>>>> >> >>>>> >>>>> > Most of the time, people who go down this route end up discovering >>>>> > not >>>>> >>>>> > the answer to their inquiry, rather the quality of their RNG (or >>>>> > lack >>>>> >>>>> > thereof). >>>>> >>>>> >>>>> >>>>> RNG? >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> Tim Wescott >>>>> >>>>> Wescott Design Services >>>>> >>>>> http://www.wescottdesign.com >>>> >>>> Random Number Generator. But you already have your answer, which is >>>> that distribution has nothing to do with color of the process, >>>> assuming stationarity etc. >>> >>>Actually I'm certain that the color of the process does impact the end >>>measurement -- the measurement mean certainly rises the longer a vector >>>of independent random variables you use; this indicates to me that the >>>more correlated the variables are, the lower the mean is going to be, >>>since the vector is going to act like it has fewer independent >>>variables. >>> >>>The language is wishy-washy, but I hope you can get the sense through >>>the hand waving. >> >> To be honest, I've had a very difficult time following what you've >> written in this thread. I'm a little surprised that other people have >> apparently been able to sort some of it out. >> >> What you wrote above doesn't parse for me at all. >> >> Mean of what? The vector? How is that at all dependent on the length >> of the vector if the process is stationary? >> >> Correlation or dependence drives the mean? >> >> No grok. > > My goal is to know the probability distribution of a measure of the > "magnitude" of noise, said measure being the maximum absolute value found > in a vector of samples of a colored (i.e., filtered white noise, > autocorrelation != impulse, etc.) random process. > > It is the mean (and variance, and pdf if I can get it) of this measure > that I am seeking. > > The mean of such a metric DOES go up as the number of points in the > vector go up, and you can see how it does intuitively: the more points > you include, the more chances there are for the vector to have one really > high-magnitude member. > > I'm pretty sure the variance falls with increasing vector length, but I'm > not sure.Simulate and histogram it, with plenty of dynamic range. -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
