Basis for digital filters

Les Cargill <lcargill99@comcast.com> wrote:

(snip)

> Interesting. Still, there's a time constant delay for this "integrator" 
> - they even specified emphasis filters in tape and LP processing by 
> units of time. Indeed, I've thought about "fraction of a sample"
> digital delay as a model for capacitors.

> I'll have to think about that one.

It does seem usual to specify some filter by RC time constant
instead of frequency. I suppose that makes it easier for people
building the filters.

-- glen

Tim Wescott wrote:
> On Mon, 01 Oct 2012 18:55:57 -0500, Les Cargill wrote:
>
>> robert bristow-johnson wrote:
>>> On 10/1/12 1:23 PM, Les Cargill wrote:
>>>> Crandles wrote:
>>>>> So I was wondering the other day why is it that recombining past
>>>>> input samples (and output)with different weights ends up creating a
>>>>> digital filter? I vaguely remember asking a prof this once and his
>>>>> response was something along the lines of "constructive and
>>>>> destructive interference".
>>>>> Could anyone elaborate on this for me (or provide me a link to read)?
>>> ...
>>>>
>>>> Analog filters made of inductors and capacitors also delay (components
>>>> of ) signals as well. The "why" of a filter is "that is how they add
>>>> back together."
>>>
>>> here is my philosophical spin on this:
>>>
>>> LTI Analog filters are made up of 3 building blocks:
>>>
>>>      1.  Adders   (signals are added together) 2.  Scalers  (signals are
>>>      multiplied by a constant value) 3.  Integrators   (by use of
>>>      capacitors)
>>>
>>>
>>> LTI Digial filters are made up of these 3 building blocks:
>>>
>>>      1.  Adders   (signals are added together) 2.  Scalers  (signals are
>>>      multiplied by a constant value) 3.  Delays
>>>
>>>
>>> you can see that 1. and 2. are the same in analog and digital filters.
>>> you can also see that neither 1. nor 2. do anything different for
>>> different frequencies.  they do not discriminate between frequencies.
>>> this is because they are "memoryless".  so you cannot make something
>>> that discriminates between frequencies with only memoryless components.
>>>
>>> if you view it on a time scale that is normalized to the period of a
>>> sinusoid, when low frequency or high frequency signals are added or
>>> scaled, there is no difference to their size nor phase whether they
>>> were low or high frequency.  but this is not the case if they are
>>> integrated (the high frequency sinusoids will come out smaller than the
>>> low) nor when they are delayed (the high frequency sinusoid will
>>> experience a greater phase shift than the low).
>>>
>>> so, in order to make a linear, time-invariant *filter* that
>>> discriminates between sinusoids of different frequencies, you need a
>>> component inside that filter that discriminates and in an analog filter
>>> it's the integrator (usually made outa a capacitor) and in a digital
>>> filter it's a delay element, even a one-sample delay.
>>>
>>>
>>>
>> Interesting. Still, there's a time constant delay for this "integrator"
>> - they even specified emphasis filters in tape and LP processing by
>> units of time. Indeed, I've thought about "fraction of a sample" digital
>> delay as a model for capacitors.
>>
>> I'll have to think about that one.
>
> I suspect you're confusing a 1st-order lowpass filter's "time constant"
> with integral action.  If you view an integrator as a 1st-order lowpass
> filter, it's time constant is infinite.
>

Pretty much. If you took my logic far enough, it'd be a comb
filter, which doesn't make any sense at all.

You just can't talk about this stuff without invoking phase domain.

> A filter's time constant means something -- but it's different from a
> delay.
>

True enough.

--
Les Cargill

On 10/1/2012 8:54 AM, Crandles wrote:
> So I was wondering the other day why is it that recombining past input
> samples (and output)with different weights ends up creating a digital
> filter? I vaguely remember asking a prof this once and his response was
> something along the lines of "constructive and destructive interference".
> Could anyone elaborate on this for me (or provide me a link to read)? Don't
> be afraid to get too mathematical either. I figured this is kind of
> important as it's the basis for a lot of work I do :)
>
> Thanks,
> Graham
>

In high school we learned about "moving average filters".  Later we
re-learned them as FIR filters with N taps and filter coefficients
of 1/N.   AKA "boxcar" filters.

Even a first order IIR filter is easy to understand in the time domain.

For example, a first order exponential filter:

Y[n] = (alpha)X[n] + (1-alpha)Y[n-1] ;     where 0 < a < 1

If alpha = 0.01, you could read the equation to say: "The new output
is 99% the of the previous output plus (only) 1% of the new input".

I think Digital Filters are easier to understand in the time domain.
Maybe less so in the frequency domain.

Rob.

On 10/1/12 11:29 PM, Rob Doyle wrote:
> On 10/1/2012 8:54 AM, Crandles wrote:
>> So I was wondering the other day why is it that recombining past input
>> samples (and output)with different weights ends up creating a digital
>> filter? I vaguely remember asking a prof this once and his response was
>> something along the lines of "constructive and destructive interference".
>> Could anyone elaborate on this for me (or provide me a link to read)?
>> Don't
>> be afraid to get too mathematical either. I figured this is kind of
>> important as it's the basis for a lot of work I do :)
...
>
> In high school we learned about "moving average filters". Later we
> re-learned them as FIR filters with N taps and filter coefficients
> of 1/N. AKA "boxcar" filters.
>
> Even a first order IIR filter is easy to understand in the time domain.

so a 2nd-order IIR can't be more than twice as hard as a 1st-order, no?

>
> For example, a first order exponential filter:
>
> Y[n] = (alpha)X[n] + (1-alpha)Y[n-1] ; where 0 < a < 1
>
> If alpha = 0.01, you could read the equation to say: "The new output
> is 99% the of the previous output plus (only) 1% of the new input".
>
> I think Digital Filters are easier to understand in the time domain.

okay so please explain to me, within the realm of the time domain, what 
this one is good for:


    y[n]  =  b0*x[n] + b1*x[n-1] + b2*x[n-2] - a1*y[n-1] - a2*y[n-2]

where

    b0 =  1.05957082442473
    b1 = -1.92519317924470
    b2 =  0.885329106308416
    a1 = -1.92519317924470
    a2 =  0.944899930733143


"The new output is 193% of the previous output minus 95% of the output 
previous to that plus 106% of the new input minus 193% of the previous 
input plus 89% of the input previous to that."


> Maybe less so in the frequency domain.

   ... oh-kaye ...


-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

On Mon, 01 Oct 2012 11:56:15 -0500, Tim Wescott <tim@seemywebsite.com>
wrote:

>On Mon, 01 Oct 2012 10:54:47 -0500, Crandles wrote:
>
>> So I was wondering the other day why is it that recombining past input
>> samples (and output)with different weights ends up creating a digital
>> filter? I vaguely remember asking a prof this once and his response was
>> something along the lines of "constructive and destructive
>> interference". Could anyone elaborate on this for me (or provide me a
>> link to read)? Don't be afraid to get too mathematical either. I figured
>> this is kind of important as it's the basis for a lot of work I do :)
>> 
>> Thanks,
>> Graham
>
>The classic "link" is "Signals and Systems", by Oppenheim and Wilsky 
>(sometimes others, too).  This is _not_ a web-page length subject: you 
>need a book.  I honestly couldn't tell you if you could learn by yourself 
>with Oppenheim -- but it's the one that the majority of US engineers of a 
>certain age learned out of in school (EE-250, last quarter of sophomore 
>year).
>
>"Understanding Digital Signal Processing" by Lyons may be helpful, but -- 
>at least in the second edition -- it's not obvious to me that he ever 
>backs off and says "this is what a filter does".  He kind of assumes that 
>you already know.  So I wouldn't tag it as a "start from nothing and get 
>all you need" resource.

          [Snipped by Lyons]

Hi Tim,
   Graham is asking a good question.
(Notice all the different replies 
in this thread.)  His professor's comment 
of "constructive and destructive interference" 
sounds fairly sensible to me.

I've always thought something like:
Transversal (tapped-delay line) FIR filters 
attenuate the signals we don't want.  And IIR 
filters amplify the signals we do want.


Regarding digital filters perhaps we might say:
"In the time domain we perform some sort of 
mathematical operation on the input sequence to
generate a filter output sequence. And that 
mathematical operation, by its very nature, has a 
"spectral effect". The spectrum of the output 
sequence will be different from the 
spectrum of the input sequence."

Or else we could say:
"Recombining past input samples (and output) with 
different weights ends up creating a digital filter 
because that's the way God wanted it to be.

[-Rick-]
 

>          [Snipped by Lyons]
>
>Hi Tim,
>   Graham is asking a good question.
>(Notice all the different replies 
>in this thread.)  His professor's comment 
>of "constructive and destructive interference" 
>sounds fairly sensible to me.
>
>I've always thought something like:
>Transversal (tapped-delay line) FIR filters 
>attenuate the signals we don't want.  And IIR 
>filters amplify the signals we do want.
>
>
>Regarding digital filters perhaps we might say:
>"In the time domain we perform some sort of 
>mathematical operation on the input sequence to
>generate a filter output sequence. And that 
>mathematical operation, by its very nature, has a 
>"spectral effect". The spectrum of the output 
>sequence will be different from the 
>spectrum of the input sequence."
>
>Or else we could say:
>"Recombining past input samples (and output) with 
>different weights ends up creating a digital filter 
>because that's the way God wanted it to be.
>
>[-Rick-]

The impulse response of a filter seen in frequency domain models its effect
correctly and I don't think I can visualise how it works in time domain
just like I don't need to visualise a signal decomposed into equivalent
multiple sinusoids. The best I can do is to look at averaging filter and
the concept of weights of previous samples reducing fluctuations or at a dc
filter which subtracts the average from signal. 

Indeed, everything around us has its rules from "God" and we just try to
model it to anticipate its behaviour. If our model works we get proud, if
it does not then we explain it. You can notice that in all aspects of our
daily life from weather forecasts to share prices, politics, elections,
wars and above all the economic meltdown. 

Kaz

On Tuesday, October 2, 2012 4:54:48 AM UTC+13, Crandles wrote:
> So I was wondering the other day why is it that recombining past input
> 
> samples (and output)with different weights ends up creating a digital
> 
> filter? I vaguely remember asking a prof this once and his response was
> 
> something along the lines of "constructive and destructive interference".
> 
> Could anyone elaborate on this for me (or provide me a link to read)? Don't
> 
> be afraid to get too mathematical either. I figured this is kind of
> 
> important as it's the basis for a lot of work I do :)
> 
> 
> 
> Thanks,
> 
> Graham

Elementary my dear Graham. Consider averaging a set of numbers. You add past N values and divide by N. You can get a recursive version of this which updates point by point.

Then why is averaging filtering? Because it is essentially integration which attenuates as you go up infrequency. This is probably one of the simplest of filters. Then you go about shaping the frequency response. In fact filtering hasn't changed much really, people make it sound so complicated. 

gyansorova@gmail.com wrote:
> On Tuesday, October 2, 2012 4:54:48 AM UTC+13, Crandles wrote:
>> So I was wondering the other day why is it that recombining past
> input samples (and output)with different weights ends up creating 
>> a digital filter? I vaguely remember asking a prof this once and 
>> his response was something along the lines of "constructive and 
>> destructive interference".

(snip)

> Elementary my dear Graham. Consider averaging a set of numbers. 
> You add past N values and divide by N. You can get a recursive 
> version of this which updates point by point.

But go back farther. Is it "obvious" how an RC filter works?

It isn't so hard to do the integrals and find out the response
for an RC filter for any input. Then a little harder to do the
RLC case.
 
> Then why is averaging filtering? Because it is essentially 
> integration which attenuates as you go up infrequency. 

In many cases, the continuous problem is easier to solve than
the discrete problem. Integrating a polynomial is easier 
than evaluating a sum over integer values of powers of x.

> This is probably one of the simplest of filters. 
> Then you go about shaping the frequency response. 
> In fact filtering hasn't changed much really, people 
> make it sound so complicated. 

Some time ago, I was working with a friend on a probability
problem that could be done in either a continuous case or
discrete case. (That is, N was large enough that you didn't
notice the difference.)  I found the continuous case easier,
she found the discrete case easier, and in the end both 
answers agreed!

It is a little different in filtering. You might design a filter
that depends on two capacitors having exactly the same value. 
With careful selection and trimming you might get to 0.1% or so.

A digital filter can easily depend on the exact subtraction of
two different values, no trimming required.

On the other hand, you can pretty much make resistors and capacitors
have any value through the available range. That is, variable 
resistors and capacitors work on a continuous scale. 

Digital filters don't work on a continuous scale, which can
complicate the design of some filters.

-- glen

>On 10/1/12 1:23 PM, Les Cargill wrote:
>> Crandles wrote:
>>> So I was wondering the other day why is it that recombining past input
>>> samples (and output)with different weights ends up creating a digital
>>> filter? I vaguely remember asking a prof this once and his response
was
>>> something along the lines of "constructive and destructive
interference".
>>> Could anyone elaborate on this for me (or provide me a link to read)?
>...
>>
>> Analog filters made of inductors and capacitors also delay (components
>> of ) signals as well. The "why" of a filter is "that is how they add
>> back together."
>
>here is my philosophical spin on this:
>
>LTI Analog filters are made up of 3 building blocks:
>
>    1.  Adders   (signals are added together)
>    2.  Scalers  (signals are multiplied by a constant value)
>    3.  Integrators   (by use of capacitors)
>
>
>LTI Digial filters are made up of these 3 building blocks:
>
>    1.  Adders   (signals are added together)
>    2.  Scalers  (signals are multiplied by a constant value)
>    3.  Delays
>
>
>you can see that 1. and 2. are the same in analog and digital filters. 
>you can also see that neither 1. nor 2. do anything different for 
>different frequencies.  they do not discriminate between frequencies. 
>this is because they are "memoryless".  so you cannot make something 
>that discriminates between frequencies with only memoryless components.
>
>if you view it on a time scale that is normalized to the period of a 
>sinusoid, when low frequency or high frequency signals are added or 
>scaled, there is no difference to their size nor phase whether they were 
>low or high frequency.  but this is not the case if they are integrated 
>(the high frequency sinusoids will come out smaller than the low) nor 
>when they are delayed (the high frequency sinusoid will experience a 
>greater phase shift than the low).
>
>so, in order to make a linear, time-invariant *filter* that 
>discriminates between sinusoids of different frequencies, you need a 
>component inside that filter that discriminates and in an analog filter 
>it's the integrator (usually made outa a capacitor) and in a digital 
>filter it's a delay element, even a one-sample delay.
>
>
>-- 
>
>r b-j                  rbj@audioimagination.com
>
>"Imagination is more important than knowledge."
>
>
>

Hey,

I just want to say thanks for all the responses the discussion definitely
helped improve my comprehension of the topic and I think I finally got it.
I felt that rbj's answer really helped clarify it for me though.

From my understanding of it digital filters (and analog ones follow very
similarly) work in the following manner: The delay component causes phase
shifts in the individual/component sinusoids that make up a signal. By
creating phase shifts the peaks and troughs of the sinusoids are realigned.
The peaks and troughs can also be enhanced by scaling components. Adders
then recombine all these modified sinusoids which causes the constructive
and deconstructive interference my professor had mentioned. By changing the
different amounts of delay and the magnitude of the scalers (the order and
filter coefficients) a digital filter can be adjusted to form a particular
frequency response.

I used the following piece of Matlab to convince myself of the frequency
dependent phase change aspect of delaying. In the code there are two waves
at the same sampling rate and one is twice the frequency of the other. A
one sample delay in the higher frequency wave results in a 180 degree phase
change while it results in a 90 degree phase shift for the lower frequency
signal.

wave2 = [1,0,-1,0,1,0,-1];
wave1 = [1,-1,1,-1,1,-1,1];

stem(wave1(2:6), 'p', 'r');
hold on;
stem(wave2(2:6), 's', 'b');

figure;
stem(wave1(1:5), 'p', 'r');
hold on;
stem(wave2(1:5), 's', 'b');

For analog filters one could replace the effects of a delay unit with that
of an integrator (or differentiator depending on the implementation).

 - Graham

Crandles <61563@dsprelated> wrote:
>>On 10/1/12 1:23 PM, Les Cargill wrote:
>>> Crandles wrote:
>>>> So I was wondering the other day why is it that recombining past input
>>>> samples (and output)with different weights ends up creating a digital
>>>> filter? I vaguely remember asking a prof this once and his response

(snip)
>>> Analog filters made of inductors and capacitors also delay (components
>>> of ) signals as well. The "why" of a filter is "that is how they add
>>> back together."

(snip)
> I just want to say thanks for all the responses the discussion definitely
> helped improve my comprehension of the topic and I think I finally got it.
> I felt that rbj's answer really helped clarify it for me though.
 
> From my understanding of it digital filters (and analog ones follow very
> similarly) work in the following manner: The delay component causes phase
> shifts in the individual/component sinusoids that make up a signal. By
> creating phase shifts the peaks and troughs of the sinusoids are realigned.
> The peaks and troughs can also be enhanced by scaling components. Adders
> then recombine all these modified sinusoids which causes the constructive
> and deconstructive interference my professor had mentioned. By changing the
> different amounts of delay and the magnitude of the scalers (the order and
> filter coefficients) a digital filter can be adjusted to form a particular
> frequency response.

Some years ago, I was working on a probability problem with someone
else. I did the problem considering the system as continuous (like a 
string), she did it as a discrete problem (like a pearl necklace).

Analog (RLC) filters work in the continuous domain (even though descrete
numbers of electrons go through them) and approximate the integral and
derivative operators. So, yes, they have a frequency dependent delay,
but the amplitude is also frequency dependent.

Digital filters are built with discrete delays. The math of finite
differences is different from that of differentials, and so are the
resulting filters.
 
> I used the following piece of Matlab to convince myself of the frequency
> dependent phase change aspect of delaying. In the code there are two waves
> at the same sampling rate and one is twice the frequency of the other. A
> one sample delay in the higher frequency wave results in a 180 degree phase
> change while it results in a 90 degree phase shift for the lower frequency
> signal.

Yes.

(snip)

> For analog filters one could replace the effects of a delay unit with that
> of an integrator (or differentiator depending on the implementation).

Yes. Remember the definition of derivative:

   lim (h-->0)  (f(x+h)-f(x))/h

The delay (h) goes to zero. 

In the digital filter case, only finite delays are allowed.

For low frequencies, you might not notice the difference, but as
the frequency approaches (half) the sampling rate, the difference
becomes more and more significant.

-- glen