What Nyquist Didn't Say

Jerry Avins wrote:
> miso@sushi.com wrote:
>
> > Ring before the signal arrives? That sound non-causal to me.
>
> Please read more carefully. The filter rings before the main part of the
> output step *emerges* but after the step arrives at the input. The
> filter's inherent delay makes that quite possible.
>
> Jerry
> --
>         "The rights of the best of men are secured only as the
>         rights of the vilest and most abhorrent are protected."
>             - Chief Justice Charles Evans Hughes, 1927
> =AF=AF=AF=AF=AF

=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF
Here is the line verbatim:
"Linear phase has a very undesirable side effect, it rings *before and
after*
the step, supposed to be more audible."

Nothing wrong with my reading. Now if you are somehow looking at the
output to interpret where the large transition occurred, that is a
different story.  However, any filter where the impulse response goes
negative will  have such ringing, be it linear phase or not. You need
to visualize the convolution.

Steve Underwood wrote:
> Hi Tim,
> 
> Tim Wescott wrote:
> 
>> I've seen a lot of posts over the last year or so that indicate a lack 
>> of understanding of the implications of the Nyquist theory, and just 
>> where the Nyquist rate fits into the design of sampled systems.
>>
>> So I decided to write a short little article to make it all clear.
>>
>> It's a little longer than 'short', and it took me way longer than I 
>> thought it would, but at least it's done and hopefully it's clear.
>>
>> You can see it at 
>> http://www.wescottdesign.com/articles/Sampling/sampling.html.
>>
>> If you're new to this stuff, I hope it helps.  If you're an expert and 
>> you have the time, please feel free to read it and send me comments or 
>> post them here.
>>
> 
> May I ask what software you used to render the maths on that page? It 
> looks clearer than the stuff I produce. MathML is getting into browsers 
> now, but the rendering of that looks so bad with anything I have tried, 
> that inserted images in HTML pages still seems the only practical approach.

SciLab
> 
> I'd still like to see a web page I can point people to when they say a 
> 10kHz sine wave on a CD will come out as a square wave/triangular 
> wave/some other weird notion.

Well, get writing!
> 
> Steve

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

miso@sushi.com wrote:
> Jerry Avins wrote:
>> miso@sushi.com wrote:
>>
>>> Ring before the signal arrives? That sound non-causal to me.
>>
>> Please read more carefully. The filter rings before the main part of
>> the output step *emerges* but after the step arrives at the input.
>> The filter's inherent delay makes that quite possible.
>>
>> Jerry
>> --
>>         "The rights of the best of men are secured only as the
>>         rights of the vilest and most abhorrent are protected."
>>             - Chief Justice Charles Evans Hughes, 1927
>> �����
>
> ����������������������������������������������������������������
> Here is the line verbatim:
> "Linear phase has a very undesirable side effect, it rings *before and
> after*
> the step, supposed to be more audible."
>
> Nothing wrong with my reading. Now if you are somehow looking at the
> output to interpret where the large transition occurred, that is a
> different story.  However, any filter where the impulse response goes
> negative will  have such ringing, be it linear phase or not. You need
> to visualize the convolution.

It's called *pre-ringing* and it appears because the chunks are processed 
forward and backward in a row, so a unity pulse will have an identical 
rising and falling edge. If the filter is of the ringing type, thus the 
ringing occurrs twice.
You are right in saying it's impossible, but only in an analog world. 
Digital filters do have a latency which will always be longer than the delay 
of the corresponding analog filter; with linear phase it will be twice the 
FIR size plus twice the conversion time and more than double than the analog 
counterpart.
Well done analog filters are of the *minimum phase* type, having just the 
lowest possible delay for that shape of output response. This is possible to 
realize digitally with IIR filters only.
And do not think that even a Gauss filter has only positive 
FIR-coefficients. This would be only true for a filter of infinite length, 
which apparently isn't that desirable at all. For practicable sizes the 
location of the poles and zeros has to be modified and one might get even 
negative coefficients, depending on the ratio of sampling- and filter 
frequency and filter length.

-- 
ciao Ban
Apricale, Italy 

Ban wrote:
> miso@sushi.com wrote:
> > Jerry Avins wrote:
> >> miso@sushi.com wrote:
> >>
> >>> Ring before the signal arrives? That sound non-causal to me.
> >>
> >> Please read more carefully. The filter rings before the main part of
> >> the output step *emerges* but after the step arrives at the input.
> >> The filter's inherent delay makes that quite possible.
> >>
> >> Jerry
> >> --
> >>         "The rights of the best of men are secured only as the
> >>         rights of the vilest and most abhorrent are protected."
> >>             - Chief Justice Charles Evans Hughes, 1927
> >> =AF=AF=AF=AF=AF
> >
> > =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF
> > Here is the line verbatim:
> > "Linear phase has a very undesirable side effect, it rings *before and
> > after*
> > the step, supposed to be more audible."
> >
> > Nothing wrong with my reading. Now if you are somehow looking at the
> > output to interpret where the large transition occurred, that is a
> > different story.  However, any filter where the impulse response goes
> > negative will  have such ringing, be it linear phase or not. You need
> > to visualize the convolution.
>
> It's called *pre-ringing* and it appears because the chunks are processed
> forward and backward in a row, so a unity pulse will have an identical
> rising and falling edge. If the filter is of the ringing type, thus the
> ringing occurrs twice.
> You are right in saying it's impossible, but only in an analog world.
> Digital filters do have a latency which will always be longer than the de=
lay
> of the corresponding analog filter; with linear phase it will be twice the
> FIR size plus twice the conversion time and more than double than the ana=
log
> counterpart.
> Well done analog filters are of the *minimum phase* type, having just the
> lowest possible delay for that shape of output response. This is possible=
 to
> realize digitally with IIR filters only.
> And do not think that even a Gauss filter has only positive
> FIR-coefficients. This would be only true for a filter of infinite length,
> which apparently isn't that desirable at all. For practicable sizes the
> location of the poles and zeros has to be modified and one might get even
> negative coefficients, depending on the ratio of sampling- and filter
> frequency and filter length.
>
> --
> ciao Ban
> Apricale, Italy

The Gaussian to which I refer is S domain. If you mapped it to Z
domain, it would have to be IIR, not FIR.

Ban wrote:
>>Here is the line verbatim:
>>"Linear phase has a very undesirable side effect, it rings *before and
>>after*
>>the step, supposed to be more audible."
>>
> 
> It's called *pre-ringing* and it appears because the chunks are processed 
> forward and backward in a row, so a unity pulse will have an identical 
> rising and falling edge. If the filter is of the ringing type, thus the 
> ringing occurrs twice.

Even steep linear phase analogue filters will exhibit pre-ringing.
If you were to linearise the phase response of, say, a Butterworth
filter, by adding one or more all-pass sections, its impulse
repsonse will ring before and after the main output. Of course,
the overall delay must go up for the filter to remain causal.

Jeroen Belleman

Joerg wrote:
(someone wrote)

>> It was actually Shannon (among others) that did the sampling theorem;
>> Nyquist made an observation. Your bibliography doesn't cite either of
>> them. It's probably correct to use "Nyquist rate" but not "Nyquist
>> theorem."

> Nyquist published his paper about the minimum required sample rate in 
> 1928. Shannon was a kid of 12 years back then. The paper wasn't about 
> ADCs or sampling in today's sense but about how many pulses per second 
> could be passed through a telegraph channel of a given bandwidth.
(and be distinguished on the other end).

The important point being that the math is the same even though the
goal is different.  I suppose, then, the sample rate should be
a lemma to Nyquist's telegraph channel theorem.

By the way, Gauss published the first paper on the FFT.

-- glen

Ban wrote:

   ...

> Well done analog filters are of the *minimum phase* type, having just the 
> lowest possible delay for that shape of output response. This is possible to 
> realize digitally with IIR filters only.

Minimum-phase (or nearly minimum) FIRs are possible, just not symmetric 
FIRs. You can make maximum-phase FIRs too. Then *all* the ringing is on 
the leading edge.

Jerry
-- 
        "The rights of the best of men are secured only as the
        rights of the vilest and most abhorrent are protected."
            - Chief Justice Charles Evans Hughes, 1927
���������������������������������������������������������������������

glen herrmannsfeldt wrote:
> By the way, Gauss published the first paper on the FFT.

(Actually, Gauss never published it.  It was only published
posthumously as part of his notes.)

Tim Wescott wrote:

> I've seen a lot of posts over the last year or so that indicate a lack 
> of understanding of the implications of the Nyquist theory, and just 
> where the Nyquist rate fits into the design of sampled systems.

> So I decided to write a short little article to make it all clear.

I like it.

As for section 1, for a periodic signal, or one that you only care
about over a finite time, you can (mathematically) sample perfectly in a 
finite time.  Realistically, quantum mechanics and the uncertainty
principle, in other words noise, will get to you.

The question of < or <= comes up often.  There is zero probability
(that is, zero width) so it will never come up in real signals.
(Or consider jitter in the time base.)

Other than that, I think it is pretty good.

-- glen

stevenj@alum.mit.edu wrote:

> glen herrmannsfeldt wrote:

>>By the way, Gauss published the first paper on the FFT.

> (Actually, Gauss never published it.  It was only published
> posthumously as part of his notes.)

Then, Gauss wrote the first published paper on FFT?

If you want to put it that way, very few people publish
papers, they just send them to someone else to publish.

But yes, I had forgotten that.

-- glen