Double integrated white noise

On Tue, 26 Feb 2008 20:22:59 -0600, bharat pathak wrote:
(top posting fixed)
>>I must be disagreeable today -- I'm certainly disagreeing enough.
>>
>>Depending on how you constrain your z transform a transfer function with
> 
>>a pole outside just about any region you wish to define on the complex
>>plane "doesn't exist".
>>
>>But then, depending on how you constrain your z transform you can find a
> 
>>region of convergence for just about any pole anywhere.  With careful
>>dancing around the math and a creative view of causality you can get
>>just about any z transform that consists of exponents, steps and ramps
>>to converge.
>>
>>But that's not important.  What's important is that you can _use_ a
>>transfer function with a pole at z=1 to make accurate predictions, both
>>theoretically and through experimentation.  So no matter how creatively
>>you have to state your case to a mathematician, when you're doing
>>engineering work you can just take the thing at face value.
>>
>>The only thing that you _must_ pay attention to is that if you are going
> 
>>to do like Marc did and start using an integrator (or a double
>>integrator) all by itself you have to account for the instability, and
>>the resulting behavior that for most inputs the outputs will go to
>>infinity.  Marc did this with windowing, which is probably about as
>>proper as you can get given that the system as a whole is much more
>>amenable to theoretical analysis than to experiment.
>>
> Tim,
> 
>    Good that you are disagreeing. Atleast this way I will understand
>    better.
> 
>    An integrator is an unstable system atleast for z=1. So any small
>    amount of DC at input is going to make it to go into oscillations.
>    Only the period of oscillations will change based on how low is the
>    DC value.

Check your math.  An single integrator is unstable in the BIBO sense, 
because a (bounded) DC input will make the output grow without bound.  It 
does not, however, oscillate.  A single integrator, however, _is_ stable 
in the sense that there are no initial conditions to the system that will 
cause the output to grow without bound with a zero input -- when I was in 
school they referred to this sort of thing as a 'metastable' system.

Note that a _double_ integrator is not 'metastable' -- a nonzero state in 
the first integrator will cause the output to grow without bound unless 
you get just the right input to the system.  Note also that a system with 
any collection of non-identical poles on the stability boundary will be 
metastable (and pretty weird, too).

>    How does windowing of gaussian data helps in removingof DC from the
>    incoming signal to avoid oscillations.

It doesn't do anything to the behavior of the system, it just changes the 
way that the FFT 'sees' the data.

The FFT acts as if the input data is periodic, so any discontinuities in 
the data between the first and last samples appears in the FFT result as 
a signal, just as if that same discontinuity had been smack in the 
middle.  This is the whole point of windowing.  In this case, the input 
to the system is random noise, and the two integrators will generate a 
considerable bias at the end of the run.  This bias must be dealt with 
before anything meaningful can come out of the FFT, and windowing is one 
way to accomplish this.

>    In order to remove DC completely from the incoming signal, you need a
>    differentiator upfront. i.e. hz = [1 -1] and hence if you look at all
>    CIC systems they are a combination of differentiator and integrators.
>    To remove DC completely you need a zero at z = 1, which essentially a
>    differentiator does. It offers -infinite db attenuation at dc.
> 
That is correct, but not my point at all.  In this case you don't need to 
remove the DC, but you do have to deal with it correctly.


-- 
Tim Wescott
Control systems and communications consulting
http://www.wescottdesign.com

Need to learn how to apply control theory in your embedded system?
"Applied Control Theory for Embedded Systems" by Tim Wescott
Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html

Tim Wescott wrote:
> bharat pathak wrote:
>>> Looks like I can answer my own question here. The anomaly comes from the
>>
>>>  implicit rectangular windowing of the finite number of samples. 
>>> Applying a suitable window to the data seems to give the expected
>> results.
>>
>> Marc,
>>
>>    you are feeding random data. hence there is no concept of spectral
>>    leakage as spectral leakage will always be there if u window or not.
>>
>>    windowing is helpful if you have sinetones at input and then u    
>> chop-off some portion of it, and try to compute FFT.
>>
> I think in this case windowing _is_ indicated.  Mostly because Marc 
> clearly saw better results when he applied it, but also because of the 
> nature of the sample he's FFT-ing.

Thanks Tim, it's good to know that I was (somewhat) on the right track, 
but possibly for the wrong reasons.

> 
> If you take an FFT of white noise, then spectral leakage is immaterial 
> -- if all samples are independent, then there will be no artificial 
> discontinuity between the 1st sample and the Nth.
> 
> However, if you take an FFT of colored noise, which should have some 
> correlation between adjacent samples, then the step between the 1st and 
> Nth samples will, indeed, splatter over the spectrum at 20dB/decade. 
> Moreover, the lower the frequency content of the noise, the worse this 
> leakage will be.
> 
> This gets particularly bad when you try this with data from integrated 
> white noise, because the Nth sample is a random walk away from the 1st; 
> with such high variance your expected step is large.  With a double 
> integrator your expected step is _huge_.
> 
> So, Marc -- window the data, with my blessing.  It may even me more 
> valid to remove the mean, the trend, and the x^2 trend from the noise 
> before doing your FFT, then augment the FFT output with an estimate of 
> the effect of the mean, trend and x^2 trend on the spectrum.
> 

I tried removing the mean and linear trend (and adding in the expected 
FFT), and it didn't have a significant effect on the results. I will try 
again, removing the x^2 trend and see if that changes things.

Cheers

Marc

Tim,

>Check your math.  An single integrator is unstable in the BIBO sense, 
>because a (bounded) DC input will make the output grow without bound.  It

>does not, however, oscillate.  A single integrator, however, _is_ stable

>in the sense that there are no initial conditions to the system that will

>cause the output to grow without bound with a zero input -- when I was in

>school they referred to this sort of thing as a 'metastable' system.

  I checked my math. When it comes to implementation, either you
  will model it on machine or hardware. both of them have precision
  limitation (fixed or floating), when the output grows beyond the
  dynamic range it rolls over and this causes large cycle oscillations.
  I was referring to this. Yes definitely on peice of paper there are
  no oscillations. Metastability is a condition of a Flip-Flop and when
  they put 2 flip-flops the probablility of occurence becomes way too
  low, to see it in a lifetime of the device. Can you cite a reference
  where "metastability of digital integrator" is mentioned. I would
  like to enhance my understanding.

>The FFT acts as if the input data is periodic, so any discontinuities in

>the data between the first and last samples appears in the FFT result as

>a signal, just as if that same discontinuity had been smack in the 
>middle.  This is the whole point of windowing.  In this case, the input 
>to the system is random noise, and the two integrators will generate a 
>considerable bias at the end of the run.  This bias must be dealt with 
>before anything meaningful can come out of the FFT, and windowing is one

>way to accomplish this.

   Where is the question of "discontinuity of random data" at boundaries.
   I don't understand this please explain. The data itself is random
   in nature, so according to me discntinuity at boundaries do not make
   sense. In case if you have any document reference, I would like to
read.

Thanks and Regards
Bharat Pathak

Arithos Designs
~dsp Simplified
www.Arithos.com

Video IP development........DSP Design Consulting........DSP Training.



>

Tim Wescott wrote:

   ...

> Note that a _double_ integrator is not 'metastable' -- a nonzero state in 
> the first integrator will cause the output to grow without bound unless 
> you get just the right input to the system.  Note also that a system with 
> any collection of non-identical poles on the stability boundary will be 
> metastable (and pretty weird, too).

An interesting case of metastability is a pair of cascaded integrators 
in which each is driven by the other's output (with an inversion in 
there somewhere). An analog implementation is a bit difficult to 
stabilize, but digitally, that's fairly easy. Do you recognize the 
quadrature oscillator?

   ...

Jerry
-- 
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

bharat pathak wrote:

   ...

>   I checked my math. When it comes to implementation, either you
>   will model it on machine or hardware. both of them have precision
>   limitation (fixed or floating), when the output grows beyond the
>   dynamic range it rolls over and this causes large cycle oscillations.

The roll-over is a ninlinearity. Linear theory doesn't apply then.

   ...

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Jerry Avins wrote:
> Tim Wescott wrote:
> 
>   ...
> 
>> Note that a _double_ integrator is not 'metastable' -- a nonzero state 
>> in the first integrator will cause the output to grow without bound 
>> unless you get just the right input to the system.  Note also that a 
>> system with any collection of non-identical poles on the stability 
>> boundary will be metastable (and pretty weird, too).
> 
> An interesting case of metastability is a pair of cascaded integrators 
> in which each is driven by the other's output (with an inversion in 
> there somewhere). An analog implementation is a bit difficult to 
> stabilize, but digitally, that's fairly easy. Do you recognize the 
> quadrature oscillator?
> 
Indeed, and it's one of the cases of a system with a pair of 
non-identical poles on the stability boundary.

To be completely correct you shouldn't multiply by -1 in one stage.  You 
should multiply by i in two.  That gets a bit difficult when you're 
working with op-amps, of course.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

bharat pathak wrote:
> Tim,
> 
>> Check your math.  An single integrator is unstable in the BIBO sense, 
>> because a (bounded) DC input will make the output grow without bound.  It
> 
>> does not, however, oscillate.  A single integrator, however, _is_ stable
> 
>> in the sense that there are no initial conditions to the system that will
> 
>> cause the output to grow without bound with a zero input -- when I was in
> 
>> school they referred to this sort of thing as a 'metastable' system.
> 
>   I checked my math. When it comes to implementation, either you
>   will model it on machine or hardware. both of them have precision
>   limitation (fixed or floating), when the output grows beyond the
>   dynamic range it rolls over and this causes large cycle oscillations.
>   I was referring to this. Yes definitely on peice of paper there are
>   no oscillations.

I was referring to theory.  In practice you may see oscillations, you 
may see the number getting "stuck", you may see resistors turn to smoke 
-- there are many behaviors, all of them beyond the realm of linear 
system analysis.  For Marc's problem he only needs to limit the time 
that he simulates, and nothing will overflow.

>   Metastability is a condition of a Flip-Flop and when
>   they put 2 flip-flops the probablility of occurence becomes way too
>   low, to see it in a lifetime of the device. Can you cite a reference
>   where "metastability of digital integrator" is mentioned. I would
>   like to enhance my understanding.

The word "metastable" means a lot of different things depending on 
context -- check Wikipedia for a few of them.  The definition that I 
remember refers to continuous-time integrators, 'on paper' if you will.

>> The FFT acts as if the input data is periodic, so any discontinuities in
> 
>> the data between the first and last samples appears in the FFT result as
> 
>> a signal, just as if that same discontinuity had been smack in the 
>> middle.  This is the whole point of windowing.  In this case, the input 
>> to the system is random noise, and the two integrators will generate a 
>> considerable bias at the end of the run.  This bias must be dealt with 
>> before anything meaningful can come out of the FFT, and windowing is one
> 
>> way to accomplish this.
> 
>    Where is the question of "discontinuity of random data" at boundaries.
>    I don't understand this please explain. The data itself is random
>    in nature, so according to me discntinuity at boundaries do not make
>    sense. In case if you have any document reference, I would like to
> read.
> 
OK.  This will be the third time I've stated this in this thread, so 
please pay attention:

   The FFT behaves as if the input data is cyclic, with the
   last point in the data being effectively adjoint to the first.

With most data that's going into an FFT the actual data isn't cyclical, 
or it isn't cyclical with a period exactly equal to the length of the 
FFT.  Thus, the inevitable difference between the first and last points 
in the data array generates a false response in the FFT, because the FFT 
treats it as whopping huge discontinuity.

This is the whole reason you need to window data going into an FFT -- to 
get rid of the discontinuity that the algorithm will see.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

>With most data that's going into an FFT the actual data isn't cyclical, 
>or it isn't cyclical with a period exactly equal to the length of the 
>FFT.  Thus, the inevitable difference between the first and last points 
>in the data array generates a false response in the FFT, because the FFT

>treats it as whopping huge discontinuity.

Agreed Tim,

       My only doubt is.

       Do you need to window the data if it is gaussianly distributed?

       All this windowing makes sense for periodic data, because only
       for that you will get discontinuity at boundaries. A random data 
       is random anyway. So what additional advantage will you get when 
       you window random data?? It is anyway discontinous throughout.

       Please clarify.

Regards
Bharat Pathak

Arithos Designs
www.Arithos.com

Tim Wescott wrote:

   ...

> The word "metastable" means a lot of different things depending on 
> context -- check Wikipedia for a few of them.  The definition that I 
> remember refers to continuous-time integrators, 'on paper' if you will.

Let's not for the metastable state a flip-flop can hang up in if the 
toggle conditions aren't all met.

   ...

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

bharat pathak wrote:
>> With most data that's going into an FFT the actual data isn't cyclical, 
>> or it isn't cyclical with a period exactly equal to the length of the 
>> FFT.  Thus, the inevitable difference between the first and last points 
>> in the data array generates a false response in the FFT, because the FFT
> 
>> treats it as whopping huge discontinuity.
> 
> Agreed Tim,
> 
>        My only doubt is.
> 
>        Do you need to window the data if it is gaussianly distributed?
> 
>        All this windowing makes sense for periodic data, because only
>        for that you will get discontinuity at boundaries. A random data 
>        is random anyway. So what additional advantage will you get when 
>        you window random data?? It is anyway discontinous throughout.
> 
>        Please clarify.

Oh come on; the explanation was given already. Low frequencies 
predominate in i/f^2 noise. Therefore, samples can't be completely 
uncorrelated from their neighbors. Can you take it from there?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������