Nyquist Didn't Say That

On Wed, 23 Aug 2006 12:13:33 -0400, Phil Hobbs
<pcdh@SpamMeSenseless.pergamos.net> wrote:

>Tim Wescott wrote:
>
>> Kinda off topic --
>> 
>> A month or two ago there was a spate of postings on these groups 
>> displaying a profound misunderstanding of how to apply Nyquist's theorem 
>> to problems of setting sampling or designing anti-alias filters.  I 
>> helped folks out as much as I could, but it really demands an article, 
>> which I am currently working on.
>> 
>> The misconceptions that I noticed pretty much boiled down to the 
>> following two:
>> 
>> One, "I need to monitor a signal that happens at X Hz, so I'm going to 
>> sample it at 2X Hz".
>> 
>> Two, "I can sample at X Hz, so I'm going to build an anti-alias filter 
>> with a cutoff of X/2 Hz".
>> 
>> I estimate that answering these misconceptions will only take 3-4k 
>> words, but I don't want to miss any other big ones.
>> 
>> Have you seen any other real howlers that relate to Nyquist, and what 
>> you should really be thinking about when you're pondering sampling 
>> rates, anti-aliasing filters and/or reconstruction filters?
>> 
>> Danke.
>
>The other one I run into is that N. really applies to the bandwidth, not 
>the highest frequency as is commonly thought.  Harmonic mixers make use 
>of this all the time, using the equivalence of the sampled interval to 
>the fundamental interval [-f_s/2, f_s/2), and alias down to some lower 
>frequency in the process.  If you really reconstruct with impulses, you 
>can use a bandpass filter to get back the original signal at the 
>original carrier frequency.
>
>People also routinely neglect the to account for the zero-order hold in 
>their DAC circuits--if you take a signal, run it through an A/D and a 
>D/A, you don't wind up with the original signal, but one with an 
>additional sinc function rolloff.

This last paragraph seems worth emphasizing, particularly on the
subject of sampling rates, as it points out a reason why rather more
than 2.00...01 X sampling may be important.  I'm not sure how a
practical reconstruction filter to compensate for ZOH could be
arranged, causal or acausal, otherwise.  You need some margin for the
skirts, don't you?

Jon

>
>Cheers,
>
>Phil Hobbs

Scott Seidman wrote:
> "steve" <bungalow_steve@yahoo.com> wrote in news:1156354799.705801.226400
> @b28g2000cwb.googlegroups.com:
>
> > There is no additional information obtained by sampling at a higher
> > rate.
>
> No additional information, but its certainly easier to look at your data
> when there's more than one point in each half cycle.
> 
> -- 
Yes, easier to reconstruct a signal with more samples

Tim Wescott wrote:
> Joel Kolstad wrote:
>>...
>> Perhaps you'd be willing to take your articles and post them on
>> Wikipedia as well as the places where your name is directly tied to it
>> (in a slightly modified form)?  That way you'd help the public at
>> large (it's a lot easier to find things on Wikipedia than trying to
>> search through a dozen technical journals), and anyone who actually
>> *has* money to pay will still find you.
>>
>> ---Joel
>>
>>
> I may do that.

Wikipedia articles often have external links, which people frequently
leave alone (IE don't vandalize). Just contribute *something* useful
to the wikipedia article, then link to your own area for in-depth
coverage. Win-win scenario.

I don't think Wikipedia's going away. I find myself using it more and
more as a first step to getting any info on some new subject -- even
before google actually.

Now here's a thought -- if Wikipedia can become financially
viable in its own right (currently it depends on donations) maybe a
business model can appear where based on number of "views" of
pages, the contributing authors can get some $$$ sent their
way.

Yes -- it's viable!

#1) Suggest the possibility
#2) ???
#3) Profit!

-Dave

David Ashley said the following on 23/08/2006 19:28:
> I don't think Wikipedia's going away. I find myself using it more and
> more as a first step to getting any info on some new subject -- even
> before google actually.

In some areas, Wikipedia is great, in others it's dire (no disrepect 
intended to anyone that contributes, myself included).  Articles about 
comms and signal processing (as relevant examples) are on the whole 
scant, badly written and error-prone.  However, I'm sure this will 
change over time.


> Now here's a thought -- if Wikipedia can become financially
> viable in its own right (currently it depends on donations) maybe a
> business model can appear where based on number of "views" of
> pages, the contributing authors can get some $$$ sent their
> way.
> 
> Yes -- it's viable!
> 
> #1) Suggest the possibility
> #2) ???
> #3) Profit!

Nice idea, but I don't think it's ever going to happen.  For one, the 
Wikipedia administrators are already working hard to reduce the 
systematic bias that exists in Wikipedia (see 
http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Countering_systemic_bias), 
introducing a financial incentive to writing good articles could only 
make this worse.


-- 
Oli

On Wed, 23 Aug 2006 12:14:07 +0800, rebel <me@privacy.net> wrote:

>On Tue, 22 Aug 2006 23:28:19 -0400, Pat Farrell <none@nospam.info> wrote:
>
>>rebel wrote:
>>> Consider anything *other than* a pure sine wave at x Hz.  Consider say a
>>> square  wave at x Hz, sampled at 2x Hz.  What do *you* envisage
>>> those sample will   let you reconstruct?
>>
>>And the frequency of a square wave is what?
>>Hint, read up on Fourier series.
>
>I'm fully aware of that, but thanks for passing the tip on for others.  That WAS
>why I posed the question that way.
>
>>Sigh.
>>A square wave has infinite frequency, so what sample rate
>>do you propose?
>>
>>All real signals are composites of sine waves in theory.
>>In practice, they usually don't have infinite numbers of composite
>>waves at infinite bandwidth.
>
>Of course they don't, but the fourier series illustrates the point - the need to
>sample at least twice per period of the highest frequency component present (in
>a significant enough amplitude to matter wrt the sampling step)
>
>>BTW, a square wave can usually be expressed in four or six bytes.
>>just encode "squarewave, 10hz, 2 volt" and you are done.
>
>For a sampling oscilloscope looking at an analog waveform, that isn't really
>much help.

Again, "it depends".   You really only need to sample 2x the
bandwidth, as Nyquist stated, and in the case of something like a
square wave one must determine the "bandwidth of interest", i.e., some
point above which you're not interested or it won't matter.

For sampled-IF (or super-Nyquist as some call it), one must pay
attention to folding frequencies, etc.

It's not hard to sort out, but I think an article like what is
proposed is always a good thing, if well written, to clarify things
and help folks avoid the pitfalls.
Eric Jacobsen
Minister of Algorithms, Intel Corp.
My opinions may not be Intel's opinions.
http://www.ericjacobsen.org

Jonathan Kirwan wrote:

> This last paragraph seems worth emphasizing, particularly on the
> subject of sampling rates, as it points out a reason why rather more
> than 2.00...01 X sampling may be important.  I'm not sure how a
> practical reconstruction filter to compensate for ZOH could be
> arranged, causal or acausal, otherwise.  You need some margin for the
> skirts, don't you?

I used to work in an FFT factory, and we typically sampled at 2.56 x
BW.

On Tue, 22 Aug 2006 15:46:32 -0700, Tim Wescott <tim@seemywebsite.com>
wrote:

>Oli Filth wrote:
>
>> Tim Wescott said the following on 22/08/2006 23:23:
>> 
>>> The misconceptions that I noticed pretty much boiled down to the 
>>> following two:
>>>
>>> One, "I need to monitor a signal that happens at X Hz, so I'm going to 
>>> sample it at 2X Hz".
>>>
>>> Two, "I can sample at X Hz, so I'm going to build an anti-alias filter 
>>> with a cutoff of X/2 Hz".
>> 
>> 
>> Are you referring to:
>> 
>> a) bandpass sampling,
>
>I doubt that I'm going to touch bandpass sampling, and if I do it'll be 
>using a 10 foot pole.

Oh, in that case I'm not clear on where there's so much confusion that
needs an entire article to clear up.   I was hoping you'd hit the idea
that Nyquist really said 2x the bandwidth of interest (as others have
already mentioned).   Clarifying that doesn't lose the context of
baseband sampling, does address where the most common pitfalls lie,
and provide a full treatment of the issue as well as covers what
Nyquist really said.

>> or
>> b) in baseband sampling, the notion that in practice, one needs to 
>> sample faster than 2X Hz to measure something at X Hz?
>> 
>Yes, (b).  As well as the notion that just because your signal has a 
>fundamental frequency of X that doesn't mean it doesn't have harmonics 
>up as far as the imagination can reach.

That's a fine notion to address, that all systems are essentially
bandwidth limited by nature or can be made so easily.   Tying that to
the sampling rate is a fundamental issue, but I'm not certain that it
can't be cleared up in a few well-written paragraphs with an
illustration or two.

But maybe I'm too optimistic...
Eric Jacobsen
Minister of Algorithms, Intel Corp.
My opinions may not be Intel's opinions.
http://www.ericjacobsen.org

steve wrote:
> Jerry Avins wrote:
>> steve wrote:
> 
>> No; it's more than that. It means (among other problems) that there's no
>> way to determine the component in phase with the sample clock (sine
>> component), so the amplitude remains unknown.
>>
> sampling at 2.000001X solves that problem, there are no frequencies in
> phase with the sample clock anymore, the point I was making
> 
> There is no additional information obtained by sampling at a higher
> rate.
> 
>> That's the least of the problems, though. To resolve a frequency of f
>> Hz, one must sample on the order of 1/f seconds.
> 
> doesn't make any sense to me, so to resolve a frequency of 10 hz one
> must sample on the order of 1/10 seconds? Is that what you are saying,
> or am I reading it wrong?

Right or wrong, that's what I meant.* What's more, to resolve Fs/2 - 10 
Hz, you also need to to sample for a time in the order of 1/10 second. 
Why does it seem strange?

>> So many misconceptions, so little time. Tim: are you tuned in?
>>
> Tim is making many assumptions (unfairly in my opinion) beforehand
> about the signal and anti-alias filter in his original post, and then
> saying this and that statement is not correct. Is he assuming
> frequencies higher then the desired signal exist, I think so, but I
> don't know, is he assuming a non-brick wall anti-alias filter? I think
> so but who knows. Nyquist assumes the ideals, you can't have a theorem
> otherwise.

It seems to me that Tim is assuming anti-alias filters that produce 
results sooner than next week, and signals that would have components 
above Fs/2 without them. I don't think those assumptions are unfair.

The Nyquist criterion does indeed assume ideal conditions. Tim will show 
that the assumption is rarely justified for real work.

Jerry
___________________________________________
* Shorter time serves if you know more about your signal. If you know 
frequency, phase, and amplitude, no sampling is needed at all. If noise 
and quantization are insignificant and you know that only a single 
frequency is present, three samples suffice. If you know what that 
frequency is, two samples suffice. With most real-world conditions, you 
need about Fs * 10 samples.
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

steve wrote:
 > Jerry Avins wrote:
 >> steve wrote:
 >
 >> No; it's more than that. It means (among other problems) that there's no
 >> way to determine the component in phase with the sample clock (sine
 >> component), so the amplitude remains unknown.
 >>
 > sampling at 2.000001X solves that problem, there are no frequencies in
 > phase with the sample clock anymore, the point I was making

In theory only. To resolve a signal at 2.00000X would require 1,000,000 
seconds. (OK: maybe only 150 hours.)

 > There is no additional information obtained by sampling at a higher
 > rate.

True, but you can get that information a lot faster.

 >> That's the least of the problems, though. To resolve a frequency of f
 >> Hz, one must sample on the order of 1/f seconds.
 >
 > doesn't make any sense to me, so to resolve a frequency of 10 hz one
 > must sample on the order of 1/10 seconds? Is that what you are saying,
 > or am I reading it wrong?

Right or wrong, that's what I meant.* What's more, to resolve Fs/2 - 10 
Hz, you also need to to sample for a time in the order of 1/10 second. 
Why does it seem strange?

 >> So many misconceptions, so little time. Tim: are you tuned in?
 >>
 > Tim is making many assumptions (unfairly in my opinion) beforehand
 > about the signal and anti-alias filter in his original post, and then
 > saying this and that statement is not correct. Is he assuming
 > frequencies higher then the desired signal exist, I think so, but I
 > don't know, is he assuming a non-brick wall anti-alias filter? I think
 > so but who knows. Nyquist assumes the ideals, you can't have a theorem
 > otherwise.

It seems to me that Tim is assuming anti-alias filters that produce 
results sooner than next week, and signals that would have components 
above Fs/2 without them. I don't think those assumptions are unfair.

The Nyquist criterion does indeed assume ideal conditions. Tim will show 
that the assumption is rarely justified for real work.

Jerry
___________________________________________
* Shorter time serves if you know more about your signal. If you know 
frequency, phase, and amplitude, no sampling is needed at all. If noise 
and quantization are insignificant and you know that only a single 
frequency is present, three samples suffice. If you know what that 
frequency is, two samples suffice. With most real-world conditions, you 
need about Fs * 10 samples.
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Tim Wescott wrote:

> Have you seen any other real howlers that relate to Nyquist, and what
> you should really be thinking about when you're pondering sampling
> rates, anti-aliasing filters and/or reconstruction filters?

it's not a howler, but the sampling frequency, Fs, must be strictly
greater than twice the highest frequency, B, at least if that highest
frequency is sinusoidal resulting in two dirac spikes at +/- B on the
spectrum.  the simplest way to say it is that Fs > 2*B

the other thing i was gonna say is that at Wikipedia we are stuggling
with some of this same stuff (what the Sampling Theorem, as commonly
depicted in textbooks really says, the historic sampling theorem from
Shannon is a bit different) and, perhaps, to avoid duplication of
effort, you might want to jump into that fray instead.  it's at:

http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem

i think there is some writing that craps up the article, but that is
the lot and legacy of Wikipedia.  an encyclopedia written by committee
(the biggest, most inclusive committee possible).  so "design by
committee" is a problem.

> Danke.

Bitte.

r b-j