Window of understanding opened ONCE WAS [Re: Imp.inv/bilinear trans. vs. expm()]

Pardon pun;}

Rune Allnor wrote:
 > Simo S&#4294967295;rkk&#4294967295; <simo.sarkka@hut.fi> wrote in message 
news:<Pine.OSF.4.58.0408271231210.280764@zeus.lce.hut.fi>...
 >
 >>Hi,
 >>
 >>What is the reason to that dicrete IIR filters are usually
 >>designed using impulse invariant transformation (= Euler
 >>integration) or bilinear transformation (= trapetzoidal
 >>integration) instead of the closed form solution that
 >>can be computed by the matrix exponential function?
 >
 >
 > You need to keep an eye on the history of DSP to see the full reasons.
 > When DSP was in its infancy (form an operational, not necessarily
 > theoretical, point of view), the engineers who designed and implemented
 > the filters were trained as "analog" engineers, and had little if any
 > formal or theoretical training in designing discrete-time systems.
 > These engineers were trained to think in terms of Butterorth, 
Chebychev,
 > etc., filters, and knew how to design the filters from an *analog* 
spec.
 >
 > [snip]
 >
 > Once DSP "matured" in the sense that DSP was taught in courses in
 > universities and engineering schools, IIR design by analog template
 > was kept as a link to the world of continuous time, while purely
 > discrete-time design methods were developed.[snip]

Now I understand why experts confuse me and I them when I ask DSP 
Newbie questions.

I have a profoundly analog formal background ( computers have lots of 
6J6's, 12AX7's and 5U4's don't they ;) which I never used in 'real' world.

I was "out of it" ( yeah another pun ) for ~ 30 yrs.

Now there a computational tools ( eg Scilab ) that make strange 
computations possible.

I've asked questions about "arbitrarily shaped frequency response 
filters". I kept getting answers assuming "arbitrary" referred to 
amount of ripple in passband/stopband. What I *MEANT* was an 
*ARBITRARY* form of response vs frequency characteristic. Correlation 
to a *physically* realizable system being irrelevant.

Similarly for windowing functions.

When I've said an "arbitrarily shaped function" {be it time or 
frequency domain} my only implied restriction was that it be 
continuous ( would 'smooth' be better term).

Richard Owlett <rowlett@atlascomm.net> wrote in message news:<10j70psl0g15r9d@corp.supernews.com>...

> I've asked questions about "arbitrarily shaped frequency response 
> filters". I kept getting answers assuming "arbitrary" referred to 
> amount of ripple in passband/stopband. What I *MEANT* was an 
> *ARBITRARY* form of response vs frequency characteristic. Correlation 
> to a *physically* realizable system being irrelevant.
> 
> Similarly for windowing functions.
> 
> When I've said an "arbitrarily shaped function" {be it time or 
> frequency domain} my only implied restriction was that it be 
> continuous ( would 'smooth' be better term).

"Continuous" is not a good term to use when discussing discrete-time 
systems... 

As for "arbitrary" filters, some clarification of the terminology 
might be in useful. First, there are methods available for designing 
"arbitrary" filters in frequency domain (see below). However, the 
time-domain response of these filters are left unspecified, except 
for the resulting filters being linear-phase FIRs. 

One could do some fancy fiddeling in time domain (seismic data 
processing is infamous for that), with un-specified consequences 
in frequency domain. 

So you can do "arbitrary" stuff in *either* time domain *or* 
frequency domain. The price paid for fancy action in one domain 
is loss of control, to some extent, in the other [*]. 

Having said that, the matlab signal processing toolbox provides
the Remez algorithm for designing "arbitrary" filters in frequency 
domain:

>> help remez

 REMEZ Parks-McClellan optimal equiripple FIR filter design.
    B=REMEZ(N,F,A) returns a length N+1 linear phase (real, symmetric
    coefficients) FIR filter which has the best approximation to the
    desired frequency response described by F and A in the minimax
    sense. F is a vector of frequency band edges in pairs, in ascending
    order between 0 and 1. 1 corresponds to the Nyquist frequency or half
    the sampling frequency. A is a real vector the same size as F
    which specifies the desired amplitude of the frequency response of the
    resultant filter B. The desired response is the line connecting the
    points (F(k),A(k)) and (F(k+1),A(k+1)) for odd k; REMEZ treats the
    bands between F(k+1) and F(k+2) for odd k as "transition bands" or
    "don't care" regions. Thus the desired amplitude is piecewise linear
    with transition bands. The maximum error is minimized.
 
Scilab, or some DSP extension of it, might have available some version 
of the Remez algorith. 

Another possible method to design not quite as arbitrary but still 
flexible filters, is demonstrated in chapter 7.3 of 

Oppenheim & Schafer: Discrete-Time Signal Processing
        Prentice-Hall, 1999.

Rune

Rune Allnor may be paraphrased by:

 > ... some clarification of the terminology might be in useful.

OK, 'might' much to mild ;}

Rune stated:

 >
 > "Continuous" is not a good term to use when discussing discrete-time
 > systems...
 >


OK, just how does one UNAMBIGUOUSLY refer to digitized version of a 
analog domain function function which meets ALL traditional 
definitions of 'continuous'
Specifically:
1. satisfys "Mean value theorem"
   [ Now, ~40 yrs later I'm beginning to understand why my 1st 
semester calculus instructor thought it was important. ]
2. infinetly (spelling?) differentiable

On Tue, 31 Aug 2004 14:13:19 -0500, Richard Owlett
<rowlett@atlascomm.net> wrote:
> Rune Allnor may be paraphrased by:
>
> > ... some clarification of the terminology might be in useful.
>
> OK, 'might' much to mild ;}
>
> Rune stated:
>
> >
> > "Continuous" is not a good term to use when discussing discrete-time
> > systems...
> >
>
>
> OK, just how does one UNAMBIGUOUSLY refer to digitized version of a 
> analog domain function function which meets ALL traditional 
> definitions of 'continuous'
> Specifically:
> 1. satisfys "Mean value theorem"
>    [ Now, ~40 yrs later I'm beginning to understand why my 1st 
> semester calculus instructor thought it was important. ]

The MVT is defined in terms of differentiable functions, not the other
way 'round.

> 2. infinetly (spelling?) differentiable
>

A function f is said to be continuous at the point x_0 if:

Lim       f(x)    = f(x_0)
x --> x_0

A function need not be continuously differentiable, and in fact a
function can be continuous ONLY at x_0.

Continuity and differentiability are different things.  DAGS on the
Blancmange function which is continuous everywhere but differentiable
nowhere.

If you take something called either "Real Analysis" or "Advanced
Calculus" you run into all sorts of odd-duck functions as examples such
as.

"Define f such that f is defined everywhere but continuous nowhere"
which yields

              f = 0 if x is rational
                  1 if x is irrational

Which can easily be shown to satisfy the conditions of the problem.

Richard Owlett wrote:

> Rune Allnor may be paraphrased by:
> 
>  > ... some clarification of the terminology might be in useful.
> 
> OK, 'might' much to mild ;}
> 
> Rune stated:
> 
>  >
>  > "Continuous" is not a good term to use when discussing discrete-time
>  > systems...
>  >
> 
> 
> OK, just how does one UNAMBIGUOUSLY refer to digitized version of a 
> analog domain function function which meets ALL traditional definitions 
> of 'continuous'

You just did. There's no single word.

> Specifically:
> 1. satisfys "Mean value theorem"
>   [ Now, ~40 yrs later I'm beginning to understand why my 1st semester 
> calculus instructor thought it was important. ]
> 2. infinetly (spelling?) differentiable

Why infinitely differentiable? Even the first derivative of a continuous
function need not be continuous. Indeed, it may not even exist at some
points. If a function has a value at a particular point that depends on
the whether the independent variable is increasing or decreasing, the
function is continuous; otherwise not.

A digitized signal can be continuous only if it never varies or if there
are an infinite number of quantization levels crammed into a finite
dynamic range using an infinite sampling frequency. Few busses are wide
enough and few devices fast enough to accommodate the second kind. :^) A
signal that doesn't meet either of those criteria necessarily has gaps
and/or jumps, breaking its discontinuity.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Richard Owlett <rowlett@atlascomm.net> wrote in message news:<10j9j7d1mi7t3aa@corp.supernews.com>...
> Rune Allnor may be paraphrased by:
> 
>  > ... some clarification of the terminology might be in useful.
> 
> OK, 'might' much to mild ;}
> 
> Rune stated:
> 
>  >
>  > "Continuous" is not a good term to use when discussing discrete-time
>  > systems...
>  >
> 
> 
> OK, just how does one UNAMBIGUOUSLY refer to digitized version of a 
> analog domain function function which meets ALL traditional 
> definitions of 'continuous'

One can't. Rick's chapter 1 has a very nice discussion about these things, 
in particular his figure 1-1. Some aspects of the continuous-time signal
are lost when we sample it. One can only hope the important aspects are 
preserved. That's why the Nyquist sampling criterion is so important. 

> Specifically:
> 1. satisfys "Mean value theorem"
>    [ Now, ~40 yrs later I'm beginning to understand why my 1st 
> semester calculus instructor thought it was important. ]

The versions of the MVT I know of, explicitly recuire the functions 
to be continuous. A discrete-time signal is not continuous, so the 
MVT can not be applied. At least not the naive versions I know of. 

I don't know where the MVT fits in, in mathematics. If it is derived 
from concepts of general Hilbert space theory, there may exist a 
discrete-time version of it somewhere. If so, I don't know about it.
If the MVT is derived from concepts of the theory of continuous
functions, there is probably no way to extend it to discrete functions. 

> 2. infinetly (spelling?) differentiable

Well... it might be *formally* possible to use some sort of continuous 
representation of the discrete signal, something along the lines of 

           inf      inf
   x[n]  = sum    integral  x(tau)D(tau-nT) dtau                  (1)
          n=-inf   -inf

x[n]  - discrete sequence
x(t)  - continuous function
D(t)  - Dirac's delta function
T     - Sampling period

I think I have seen somewhere (I can't remember the reference off the 
top of my head) that the Delta function is *formally* differentiable 
as something like

  dD(t)/dt= lim D(t+e) - D(t-e),
            e->0 

but I can't see how this can be *practically* useful in any meaningful 
way.

Rune

Charles wrote:
...

> ...and in fact a function can be continuous ONLY at x_0.

That doesn't make sense. If a function is continuous in x_0, it is also 
continuous in a small open interval containing x_0.

Richard Owlett wrote:
> Rune Allnor may be paraphrased by:
> 
>  > ... some clarification of the terminology might be in useful.
> 
> OK, 'might' much to mild ;}
> 
> Rune stated:
> 
>  >
>  > "Continuous" is not a good term to use when discussing discrete-time
>  > systems...
>  >
> 
> 
> OK, just how does one UNAMBIGUOUSLY refer to digitized version of a 
> analog domain function function which meets ALL traditional definitions 
> of 'continuous'
> Specifically:
> 1. satisfys "Mean value theorem"
>   [ Now, ~40 yrs later I'm beginning to understand why my 1st semester 
> calculus instructor thought it was important. ]
> 2. infinetly (spelling?) differentiable

Richard,

1. Bandlimited integrable functions are smooth (ie. infinitely often 
differentiable). If you periodically extend the spectrum, and calculate 
the Fourier series coefficients of this new periodic spectrum, these 
coefficients are a (temporal) sampling of the bandlimited function.

2. Conversely, if you have a sumable stream of numbers and calculate the 
DTFT you get a periodic spectrum. If you (symmetrically about 0) ideally 
bandpass this spectrum (by multiplying the spectrum with a rectangle 
function), and calculate the inverse FT of the bandpassed spectrum you 
get a smooth, real valued function.

Therefore it makes perfect sense to speak of the continuity, or 
differentiability, or differential, or whatever, of a discrete set of 
numbers if you mean the corresponding property of the uniquely defined 
smooth function which you get via 2 (commonly known as 
digital-to-analogue conversion).

If you are working with a set of numbers obtained by sampling a function 
(as in 1), the uniqueness is only preserved if you make sure that the 
Nyquist criterium was met, ie. you periodically extended the spectrum 
such that no overlap occured.

FWIW,
Andor

On Wed, 01 Sep 2004 18:55:09 +0200, Andor Bariska <an2or@nospam.net> wrote:
> Charles wrote:
> ...
>
>> ...and in fact a function can be continuous ONLY at x_0.
>
> That doesn't make sense. If a function is continuous in x_0, it is also 
> continuous in a small open interval containing x_0.
>

Ah . .right.  I mangled that bit.

U-CDK_CHARLES\Charles wrote:
> On Wed, 01 Sep 2004 18:55:09 +0200, Andor Bariska <an2or@nospam.net> wrote:
> 
>>Charles wrote:
>>...
>>
>>
>>>...and in fact a function can be continuous ONLY at x_0.
>>
>>That doesn't make sense. If a function is continuous in x_0, it is also 
>>continuous in a small open interval containing x_0.
>>
> 
> 
> Ah . .right.  I mangled that bit.
> 

Let f(x) = x if x is rational, and f(x) = -x if x is irrational.  Then
f is continuous at 0 but discontinuous everywhere else.  Now back to
your regularly-scheduled DSP programming....

Bob Beaudoin