On Saturday, July 12, 2014 5:03:55 PM UTC-5, Tim Wescott wrote:> I need a number, and I'm feeling lazy; has anyone worked this out > > recently? > > > > I want to know the distribution of the maximum of the absolute value of a > > vector of samples of a colored, zero-mean Gaussian process. > > > > Or, stated another way, I want to shove white noise into a filter, then > > examine a finite chunk of the filter output for it's maximum absolute > > value. > > > > Anyone know the answer? Or should I sharpen my pencil and get to work? > > > > -- > > > > Tim Wescott > > Wescott Design Services > > http://www.wescottdesign.comAfter filtering, you have n zero-mean jointly Gaussian random variables Xi that are correlated, not independent. You are asking for the distribution of the maximum magnitude of these random variables, that is, Z = max(|X1|, |X2|, ... , |Xn|). Then, P(Z < a) = P(-a < X1 < a, -a < X2 < a, ... , -a < Xn < a). This probability can be expressed in terms of the joint CDF (long, messy formula) or a n-dimensional integral of the joint pdf of the Xi. But, _computing_ the value is a non-trivial task because the Xi are correlated. For _independent_ Xi, we have that P(Z < a) = [Phi(a) - Phi(-a)]^n = [2 Phi(a) - 1]^n from which we can write down the pdf of Z and use it to calculate the mean and variance. But it still is a lot of work. Dilip Sarwate
Max absolute value of colored Gaussian noise
Started by ●July 12, 2014
Reply by ●July 13, 20142014-07-13
Reply by ●July 13, 20142014-07-13
On Sun, 13 Jul 2014 11:23:18 +0200, Piergiorgio Sartor wrote:> On 2014-07-13 07:43, Tim Wescott wrote: > [...] >> Well, a quick disproof can be had by looking at the output of a lowpass >> or bandpass filter on an oscilloscope. Since any such filter made with >> real components is going to have an output signal as the one I >> describe, >> you can see if it's infinite or not. > > You're mixing reality with theory. > > A Gaussian random process (white or colored) is *unlimited*, that is it > max/min are +/- infinity. And it stays like that after (linear) > filtering. > > Of course, such thing does not really exists, it is only convenient for > modeling. > > Different story is mean and *variance*. > > For this, it was long time ago and I'm not anymore sure. Maybe you're > preferred search engine can help. > > bye,No I am not mixing reality with theory. I may not be stating the problem well, but I'm not mixing reality with theory. I am looking for the probability density of the maximum of the absolute value of a finite-length vector of zero-mean Gaussian variables which are related to one another through a known covariance matrix. Yes, the resulting probability density will have a tail that extends off to infinity. But the probability that any one sample of the vector is at infinity is vanishingly small. If we were to apply your attitude, then when we needed to go to the store to get eggs, we would look out at the car and say "nope, if I stand next to it in the driveway it may not be there because the PDF of its distance from its rest position goes to infinity. I guess I'll just sit here without eating dinner". THAT is mixing theory with reality. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 13, 20142014-07-13
On Sun, 13 Jul 2014 02:14:32 -0500, mnentwig wrote:> Hi, > >>> examine a finite chunk of the filter output > > if I think of the observation interval as an OFDM symbol, the maximum is > finite, relative to the total energy in the observation interval. > A maximum occurs when all the "subcarriers" are in phase. > A white power spectrum after the filter maximizes the sum, if energy is > normalized for the "colored" signal (that is, after the "filter"). > > The question is now, how do I normalize the energy. If I think of the > worst possible "colored" noise chunk, it is white and a pulse ("Gauss" > doesn't matter as I know the exact signal vector, except absolute > phase). > So it may be possible to simplify the problem, stating "the highest > possible peak of x seconds of signal bandlimited to y Hz is z". > Don't know if that's what you want. > > In OFDM we try to avoid that kind of symbol (and put lightning rods on > the towers...).Well, I'm not looking for the _absolute_ highest, because as stated that's infinite. I'm looking for the PDF of the maximum, or at least its mean and variance. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 13, 20142014-07-13
On Sun, 13 Jul 2014 07:26:53 -0700, dvsarwate wrote:> On Saturday, July 12, 2014 5:03:55 PM UTC-5, Tim Wescott wrote: >> I need a number, and I'm feeling lazy; has anyone worked this out >> >> recently? >> >> >> >> I want to know the distribution of the maximum of the absolute value of >> a >> >> vector of samples of a colored, zero-mean Gaussian process. >> >> >> >> Or, stated another way, I want to shove white noise into a filter, then >> >> examine a finite chunk of the filter output for it's maximum absolute >> >> value. >> >> >> >> Anyone know the answer? Or should I sharpen my pencil and get to work? >> >> >> >> -- >> >> >> >> Tim Wescott >> >> Wescott Design Services >> >> http://www.wescottdesign.com > > After filtering, you have n zero-mean jointly Gaussian random variables > Xi that are correlated, not independent. > You are asking for the distribution of the maximum magnitude of these > random variables, that is, > > Z = max(|X1|, |X2|, ... , |Xn|). > > Then, > > P(Z < a) = P(-a < X1 < a, -a < X2 < a, ... , -a < Xn < a). > > This probability can be expressed in terms of the joint CDF (long, messy > formula) or a n-dimensional integral of the joint pdf of the Xi. But, > _computing_ the value is a non-trivial task because the Xi are > correlated. For _independent_ Xi, we have that > > P(Z < a) = [Phi(a) - Phi(-a)]^n = [2 Phi(a) - 1]^n > > from which we can write down the pdf of Z and use it to calculate the > mean and variance. But it still is a lot of work. > > Dilip SarwateI kind of got as far as you have in the first five minutes -- which was why I was hoping that someone may have worked it out already, or seen it worked out. Maybe I'll Monte-Carlo it (blech). -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 13, 20142014-07-13
On 2014-07-13 19:42, Tim Wescott wrote: [...]> Yes, the resulting probability density will have a tail that extends off > to infinity. But the probability that any one sample of the vector is at > infinity is vanishingly small.Of course is small, but that's exactly the difference between reality and theory. In theory, it will be +inf, in reality it will be well between six sigma.> If we were to apply your attitude, then when we needed to go to the store > to get eggs, we would look out at the car and say "nope, if I stand next > to it in the driveway it may not be there because the PDF of its distance > from its rest position goes to infinity. I guess I'll just sit here > without eating dinner". THAT is mixing theory with reality.You are the one saying that the oscilloscope does not show infinity. That's the reality, the theory is that you will have infinity. Objecting that the measure is not +inf *is* mixing reality and theory. bye, -- piergiorgio
Reply by ●July 13, 20142014-07-13
On Sun, 13 Jul 2014 07:26:53 -0700, dvsarwate wrote:> On Saturday, July 12, 2014 5:03:55 PM UTC-5, Tim Wescott wrote: >> I need a number, and I'm feeling lazy; has anyone worked this out >> >> recently? >> >> >> >> I want to know the distribution of the maximum of the absolute value of >> a >> >> vector of samples of a colored, zero-mean Gaussian process. >> >> >> >> Or, stated another way, I want to shove white noise into a filter, then >> >> examine a finite chunk of the filter output for it's maximum absolute >> >> value. >> >> >> >> Anyone know the answer? Or should I sharpen my pencil and get to work? >> >> >> >> -- >> >> >> >> Tim Wescott >> >> Wescott Design Services >> >> http://www.wescottdesign.com > > After filtering, you have n zero-mean jointly Gaussian random variables > Xi that are correlated, not independent. > You are asking for the distribution of the maximum magnitude of these > random variables, that is, > > Z = max(|X1|, |X2|, ... , |Xn|). > > Then, > > P(Z < a) = P(-a < X1 < a, -a < X2 < a, ... , -a < Xn < a). > > This probability can be expressed in terms of the joint CDF (long, messy > formula) or a n-dimensional integral of the joint pdf of the Xi. But, > _computing_ the value is a non-trivial task because the Xi are > correlated. For _independent_ Xi, we have that > > P(Z < a) = [Phi(a) - Phi(-a)]^n = [2 Phi(a) - 1]^n > > from which we can write down the pdf of Z and use it to calculate the > mean and variance. But it still is a lot of work. > > Dilip SarwateActually I took your P(Z < a) formula (I had arrived at that last night), and just did some experimentation. For A with normally distributed independent variables and n ranging from 2 to 25, the CDF hits 50% between 0.7 and 1.55, it hits 90% between and 1.4 and 2.05 or so, and the probability of it being greater than 3 is small enough that I can disregard it (1/1800 or so for n = 25). So, while this isn't my answer, it's close enough that I can proceed with other things. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 13, 20142014-07-13
On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote:> > Maybe I'll Monte-Carlo it (blech). >Most of the time, people who go down this route end up discovering not the answer to their inquiry, rather the quality of their RNG (or lack thereof).
Reply by ●July 13, 20142014-07-13
On Sun, 13 Jul 2014 18:35:48 -0700 (PDT), julius <juliusk@gmail.com> wrote:>On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: > >> >> Maybe I'll Monte-Carlo it (blech). >> > >Most of the time, people who go down this route end up discovering not the answer to their inquiry, rather the quality of their RNG (or lack thereof).Yeah, I think a bit of a measure of the advancement of signal processing and cryptography/security is how much the quality of the rng gets pushed along. I'm encouraged to see some of the computing stuff that's coming out to support this. Things look like they're gonna change in a good way. And Tim should able to do his sims. ;) Eric Jacobsen Anchor Hill Communications http://www.anchorhill.com
Reply by ●July 14, 20142014-07-14
On Sun, 13 Jul 2014 18:35:48 -0700, julius wrote:> On Sunday, July 13, 2014 1:46:39 PM UTC-4, Tim Wescott wrote: > > >> Maybe I'll Monte-Carlo it (blech). >> >> > Most of the time, people who go down this route end up discovering not > the answer to their inquiry, rather the quality of their RNG (or lack > thereof).RNG? -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 14, 20142014-07-14
On Sun, 13 Jul 2014 20:00:37 +0200, Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> wrote:>On 2014-07-13 19:42, Tim Wescott wrote: >[...] >> Yes, the resulting probability density will have a tail that extends off >> to infinity. But the probability that any one sample of the vector is at >> infinity is vanishingly small. > >Of course is small, but that's exactly the difference >between reality and theory. >In theory, it will be +inf, in reality it will be well >between six sigma. > >> If we were to apply your attitude, then when we needed to go to the store >> to get eggs, we would look out at the car and say "nope, if I stand next >> to it in the driveway it may not be there because the PDF of its distance >> from its rest position goes to infinity. I guess I'll just sit here >> without eating dinner". THAT is mixing theory with reality. > >You are the one saying that the oscilloscope >does not show infinity. >That's the reality, the theory is that you >will have infinity. > >Objecting that the measure is not +inf *is* >mixing reality and theory.Is part of the problem in this discussion related to the fact that in the real world it's pretty hard to come up with a process that has tails that go to infinity, and that we can prove go to infinity? When you create a (quasi) Gaussian distribution to feed into a real-world D/A, you are limited to 16 or 24 bits (or whatever). You have to decide where that limit is going to fall on the distribution tail, and accept truncation beyond that. The farther out you want to run your tails, the lower the overall amplitude of the signal, so this is a non-trivial issue. Best regards, Bob Masta DAQARTA v7.60 Data AcQuisition And Real-Time Analysis www.daqarta.com Scope, Spectrum, Spectrogram, Sound Level Meter Frequency Counter, Pitch Track, Pitch-to-MIDI FREE Signal Generator, DaqMusiq generator Science with your sound card!
