Causality

On Fri, 11 Oct 2013 20:18:09 -0400, Randy Yates wrote:

> gyansorova@gmail.com writes:
> 
>> Somebody told me that a differentiator is classified as an uncausal
>> system because it has more zeros than poles. This would mean that it
>> needs information from the future to effect the present which is
>> nonsense. Whilst I agree you cannot implement a pure differentiator,
>> you can implement an approximation over a limited frequency range and
>> they have to be causal.
> 
> Hi,
> 
> I don't see a question here, but you seem to be implying the question,
> "Is the theoretical differentiator causal?"
> 
> Tim answered this question (sorta). I'd also like to think I'm answering
> it here but at the end of my day, I don't know.
> 
> Since the impulse response of a system is the output of the system when
> a dirac delta function \delta(t) (note 1) is applied, the impulse
> response of a differentiator "system" is the derivative of \delta(t), or
> \delta'(t).
> This is confirmed by Bracewell's statement [bracewell], p. 82, that
> 
>   \delta' \convolve f = f'(x).
> 
> An impulse response h(t) is causal iff h(t) = 0, t < 0.
> 
> So is \delta'(t) = 0 when t < 0? Hmmm.
> 
> I think this falls into the same camp as asking "What is \delta(0)?"
> This \delta'(t) thingie isn't a function (it's a "generalized function,"
> which I've not studied in math), so I don't believe you can really ask
> if \delta'(t)^- = 0 either.
> 
> Here's a whole other take on it without using DSP or linear system
> theory. Consider the class of functions f(x) that are rational functions
> (I use such a constraint just to get around the arguments over whether
> f(x) even HAS a derivative). In order to compute the derivative at x0,
> you must have knowledge of the function f(x) on both sides of x0. So
> from this POV, the derivative "system" is NOT causal.
> 
> --Randy
> 
> Notes -----
> 1. I'm using LaTeX-ish strings in this post. http://www.tug.org
> 
> @BOOK{bracewell,
>   title = "{The Fourier Transform and Its Applications}",
>   author = "{Ronald~N.~Bracewell}",
>   publisher = "McGraw-Hill",
>   edition = "second",
>   year = "1986"}

I'm kinda sorta repeating a point I kinda sorta didn't make before:

I think the real problem is that the concepts of causality and 
differentiation don't fit well in the same space.  Differentiation (in 
time, at least) depends on taking things in the limit of infinitesimal 
time, while causality depends on taking things in terms of finite 
intervals.  As soon as you try to mix a definition of causality (which 
requires making distinctions about finite intervals of time) with 
differentiation (which requires making distinctions using infinitesimal 
intervals of time), then difficulties are just going to arise.

You could define an approximation for differentiation that is noncausal 
for any finite delta-t, then take the limit as delta-t goes to zero to 
show that (a) it's really differentiation, and (b), it's noncausal.  But 
I could then define an approximation for differentiation that is strictly 
causal always, yet goes to true differentiation as delta-t approaches 
zero.

If we had a refereed argument about the whole thing that wasn't declared 
a draw, it'd probably be because you won.  But you'd never convince me 
that the notion of having an honest to gosh naked differentiator in a 
real system made any more sense than having an honest to gosh primordial 
black hole in there -- you can argue all you want about what might be 
happening in the fine detail, but in practice it can't be achieved, and 
if you did you'd just have a big smoking hole where your system used to 
be, so who cares?

You could easily define a system of looking at these things that would 
make a naked differentiator non-causal.  Perhaps when you really flog the 
math you end up finding that the only such systems that maintain 
consistency are the ones that make naked differentiators non-causal.  But 
when all is said and done (and this _is_ the point I made before), 
causality is by no means the biggest fish you have to fry when the 
question of putting a naked differentiator into a system comes up.

Since we're engineers, I think the practical considerations of finite 
amounts of noise applied to infinite gain at high frequencies, not to 
mention the difficulty of achieving truly infinite bandwidth in the first 
place, rule out worries about whether the impossible result will be 
causal or not.  It don't matter, because it ain't possible!

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com

Tim Wescott <tim@seemywebsite.please> wrote:

(snip)
>>> Somebody told me that a differentiator is classified as an uncausal
>>> system because it has more zeros than poles. This would mean that it
>>> needs information from the future to effect the present which is
>>> nonsense. 

(snip)
> I'm kinda sorta repeating a point I kinda sorta didn't make before:
 
> I think the real problem is that the concepts of causality and 
> differentiation don't fit well in the same space.  Differentiation (in 
> time, at least) depends on taking things in the limit of infinitesimal 
> time, while causality depends on taking things in terms of finite 
> intervals.  As soon as you try to mix a definition of causality (which 
> requires making distinctions about finite intervals of time) with 
> differentiation (which requires making distinctions using infinitesimal 
> intervals of time), then difficulties are just going to arise.

I was thinking about since differential equations have derivatives,
and they are causal (because much of physics depends on differential
equations in time), but differential equations really work the other
way around. 

They predict (give the future value of the system) such that the
derivative is equal to some function of the current value.

We write them as derivatives, but physically many make more sense
as integrals.

But consider Maxwell's equations (which pretty much all of 
electronics is based on). 

Ignoring constants and in 1D, you have dE/dx=-dB/dt so the
derivative of one depends on the derivative of the other.
If you look at it that way, the uncausality of one derivative
is canceled by the other.

-- glen

On Sat, 12 Oct 2013 15:40:20 +0000, glen herrmannsfeldt wrote:

> Tim Wescott <tim@seemywebsite.please> wrote:
> 
> (snip)
>>>> Somebody told me that a differentiator is classified as an uncausal
>>>> system because it has more zeros than poles. This would mean that it
>>>> needs information from the future to effect the present which is
>>>> nonsense.
> 
> (snip)
>> I'm kinda sorta repeating a point I kinda sorta didn't make before:
>  
>> I think the real problem is that the concepts of causality and
>> differentiation don't fit well in the same space.  Differentiation (in
>> time, at least) depends on taking things in the limit of infinitesimal
>> time, while causality depends on taking things in terms of finite
>> intervals.  As soon as you try to mix a definition of causality (which
>> requires making distinctions about finite intervals of time) with
>> differentiation (which requires making distinctions using infinitesimal
>> intervals of time), then difficulties are just going to arise.
> 
> I was thinking about since differential equations have derivatives, and
> they are causal (because much of physics depends on differential
> equations in time), but differential equations really work the other way
> around.
> 
> They predict (give the future value of the system) such that the
> derivative is equal to some function of the current value.
> 
> We write them as derivatives, but physically many make more sense as
> integrals.

I had an instructor make the comment once that he really preferred 
integral equations.  He seemed to feel that there were fewer and more 
obvious pitfalls involved.

> But consider Maxwell's equations (which pretty much all of electronics
> is based on).
> 
> Ignoring constants and in 1D, you have dE/dx=-dB/dt so the derivative of
> one depends on the derivative of the other.
> If you look at it that way, the uncausality of one derivative is
> canceled by the other.

And yet, if you were to solve them numerically, I think you'd find 
yourself integrating both sides, i.e.

int{E dt} = -int{B dx}

Certainly when I was taking E&M we saw the integral forms just as often 
as the derivative forms.

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com

On Sunday, October 13, 2013 5:46:03 AM UTC+13, Tim Wescott wrote:
> On Sat, 12 Oct 2013 15:40:20 +0000, glen herrmannsfeldt wrote:
> 
> 
> 
> > Tim Wescott <tim@seemywebsite.please> wrote:
> 
> > 
> 
> > (snip)
> 
> >>>> Somebody told me that a differentiator is classified as an uncausal
> 
> >>>> system because it has more zeros than poles. This would mean that it
> 
> >>>> needs information from the future to effect the present which is
> 
> >>>> nonsense.
> 
> > 
> 
> > (snip)
> 
> >> I'm kinda sorta repeating a point I kinda sorta didn't make before:
> 
> >  
> 
> >> I think the real problem is that the concepts of causality and
> 
> >> differentiation don't fit well in the same space.  Differentiation (in
> 
> >> time, at least) depends on taking things in the limit of infinitesimal
> 
> >> time, while causality depends on taking things in terms of finite
> 
> >> intervals.  As soon as you try to mix a definition of causality (which
> 
> >> requires making distinctions about finite intervals of time) with
> 
> >> differentiation (which requires making distinctions using infinitesimal
> 
> >> intervals of time), then difficulties are just going to arise.
> 
> > 
> 
> > I was thinking about since differential equations have derivatives, and
> 
> > they are causal (because much of physics depends on differential
> 
> > equations in time), but differential equations really work the other way
> 
> > around.
> 
> > 
> 
> > They predict (give the future value of the system) such that the
> 
> > derivative is equal to some function of the current value.
> 
> > 
> 
> > We write them as derivatives, but physically many make more sense as
> 
> > integrals.
> 
> 
> 
> I had an instructor make the comment once that he really preferred 
> 
> integral equations.  He seemed to feel that there were fewer and more 
> 
> obvious pitfalls involved.
> 
> 
> 
> > But consider Maxwell's equations (which pretty much all of electronics
> 
> > is based on).
> 
> > 
> 
> > Ignoring constants and in 1D, you have dE/dx=-dB/dt so the derivative of
> 
> > one depends on the derivative of the other.
> 
> > If you look at it that way, the uncausality of one derivative is
> 
> > canceled by the other.
> 
> 
> 
> And yet, if you were to solve them numerically, I think you'd find 
> 
> yourself integrating both sides, i.e.
> 
> 
> 
> int{E dt} = -int{B dx}
> 
> 
> 
> Certainly when I was taking E&M we saw the integral forms just as often 
> 
> as the derivative forms.
> 
> 
> 
> -- 
> 
> Tim Wescott
> 
> Control system and signal processing consulting
> 
> www.wescottdesign.com

Certainly when I see things like this

http://radhesh.wordpress.com/2008/05/11/pid-controller-simplified/

I wonder about our education system (complete with Matlab code and showing it working with a pure differentiator apparently!)

and this from a Uni


http://www.facstaff.bucknell.edu/mastascu/econtrolhtml/PID/PID3.html

Of course I am not complaining because I was taught that rubbish too.
It wasn't until I was in industry that a real engineer pointed out to me
that (in electro-mech systems at least) all systems have a structural resonance
and you cannot use differentiators unless it is of the phase-lead variety (ie limited with a frequency band) in cascade. I presume that for those people that have the problem of larger plants in the chemical industry (ie the fiddlers who tweek PID gains) that they must have at least a low-pass filter on the derivative action whether it be analogue or digital.

On Sat, 12 Oct 2013 17:45:24 -0700, gyansorova wrote:

> On Sunday, October 13, 2013 5:46:03 AM UTC+13, Tim Wescott wrote:
>> On Sat, 12 Oct 2013 15:40:20 +0000, glen herrmannsfeldt wrote:
>> 
>> 
>> 
>> > Tim Wescott <tim@seemywebsite.please> wrote:
>> 
>> 
>> > 
>> > (snip)
>> 
>> >>>> Somebody told me that a differentiator is classified as an
>> >>>> uncausal
>> 
>> >>>> system because it has more zeros than poles. This would mean that
>> >>>> it
>> 
>> >>>> needs information from the future to effect the present which is
>> 
>> >>>> nonsense.
>> 
>> 
>> > 
>> > (snip)
>> 
>> >> I'm kinda sorta repeating a point I kinda sorta didn't make before:
>> 
>> 
>> >  
>> >> I think the real problem is that the concepts of causality and
>> 
>> >> differentiation don't fit well in the same space.  Differentiation
>> >> (in
>> 
>> >> time, at least) depends on taking things in the limit of
>> >> infinitesimal
>> 
>> >> time, while causality depends on taking things in terms of finite
>> 
>> >> intervals.  As soon as you try to mix a definition of causality
>> >> (which
>> 
>> >> requires making distinctions about finite intervals of time) with
>> 
>> >> differentiation (which requires making distinctions using
>> >> infinitesimal
>> 
>> >> intervals of time), then difficulties are just going to arise.
>> 
>> 
>> > 
>> > I was thinking about since differential equations have derivatives,
>> > and
>> 
>> > they are causal (because much of physics depends on differential
>> 
>> > equations in time), but differential equations really work the other
>> > way
>> 
>> > around.
>> 
>> 
>> > 
>> > They predict (give the future value of the system) such that the
>> 
>> > derivative is equal to some function of the current value.
>> 
>> 
>> > 
>> > We write them as derivatives, but physically many make more sense as
>> 
>> > integrals.
>> 
>> 
>> 
>> I had an instructor make the comment once that he really preferred
>> 
>> integral equations.  He seemed to feel that there were fewer and more
>> 
>> obvious pitfalls involved.
>> 
>> 
>> 
>> > But consider Maxwell's equations (which pretty much all of
>> > electronics
>> 
>> > is based on).
>> 
>> 
>> > 
>> > Ignoring constants and in 1D, you have dE/dx=-dB/dt so the derivative
>> > of
>> 
>> > one depends on the derivative of the other.
>> 
>> > If you look at it that way, the uncausality of one derivative is
>> 
>> > canceled by the other.
>> 
>> 
>> 
>> And yet, if you were to solve them numerically, I think you'd find
>> 
>> yourself integrating both sides, i.e.
>> 
>> 
>> 
>> int{E dt} = -int{B dx}
>> 
>> 
>> 
>> Certainly when I was taking E&M we saw the integral forms just as often
>> 
>> as the derivative forms.
>> 
>> 
>> 
>> --
>> 
>> Tim Wescott
>> 
>> Control system and signal processing consulting
>> 
>> www.wescottdesign.com
> 
> Certainly when I see things like this
> 
> http://radhesh.wordpress.com/2008/05/11/pid-controller-simplified/
> 
> I wonder about our education system (complete with Matlab code and
> showing it working with a pure differentiator apparently!)
> 
> and this from a Uni
> 
> 
> http://www.facstaff.bucknell.edu/mastascu/econtrolhtml/PID/PID3.html
> 
> Of course I am not complaining because I was taught that rubbish too.
> It wasn't until I was in industry that a real engineer pointed out to me
> that (in electro-mech systems at least) all systems have a structural
> resonance and you cannot use differentiators unless it is of the
> phase-lead variety (ie limited with a frequency band) in cascade. I
> presume that for those people that have the problem of larger plants in
> the chemical industry (ie the fiddlers who tweek PID gains) that they
> must have at least a low-pass filter on the derivative action whether it
> be analogue or digital.

When I teach seminars on control, I present differentiator action as a 
useful but dangerous thing, that will give you problems with high 
frequency noise amplification and high frequency instabilities.

And yes, usually the answer is a band-limited differentiator, or lead-
lag, or whatever you want to call it.

Perversely, in a sampled-time system sometimes an answer is to sample 
slower, but that's because you're essentially using the zero-hold action 
as the bandlimit in the derivative action.  It's not a _good_ answer by 
any means.

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com