# Nyquist theorem and Sampling rate of signals (Newbie)

Started by October 29, 2010
```Please I am a complete newbie in Signal analysis and modelling and I
am quite confused  about the use of  Nyquist theorem in sampling .Does
this Nyquist criterion affect the  rate at which data is transferred
that is  the data rate ? I have  read the wikiopedia saying that I
must sample at twice the bandwidth to be able to recontruct a signal
perfectly But I don't understand the concept . If I sample at 2B then
it is bigger than the wave itself .  I did like to go on to DSP stuffs
but I got to learn the basics first.

Incase it help I have a Physics Background .
```
```Deamon <persistence911@gmail.com> wrote:

> Please I am a complete newbie in Signal analysis and modelling and I
> am quite confused  about the use of  Nyquist theorem in sampling .Does
> this Nyquist criterion affect the  rate at which data is transferred
> that is  the data rate ? I have  read the wikiopedia saying that I
> must sample at twice the bandwidth to be able to recontruct a signal
> perfectly But I don't understand the concept . If I sample at 2B then
> it is bigger than the wave itself .  I did like to go on to DSP stuffs
> but I got to learn the basics first.

> Incase it help I have a Physics Background .

It should be obvious that it you sample much higher than twice
the highest frequency component, and appropriately filter the
reconstructed signal, that it would be close to the original.

What isn't so obvious is that with perfect (and non-quantized)
sampling that you mathematically get back the exact waveform.
Also, that it is exact, assuming perfect filters, down to just
(epsilon) above twice the sampling rate.

One way to see it that seems more obvious to physicists, and
less obvious to others, is as solutions to boundary value problems.

Consider a string fixed a both ends, such as a violin string,
and allow for modes only up to a specific maximum frequency.
Now, note how many modes are allowed, and that the amplitude
and phase of those modes is enough to uniquely determine the
vibrational state of the system.  Then, as you increase the
length of the string the number of normal modes increases
(approximately) linearly with the length.

Maybe it is more obvious to consider the problem that Nyquist
was actually working on.  He was trying to find out how fast
telegraph pulses could be sent through a cable with a given
frequency response.  (Cable shunt capacitance and source
impedance creating a low pass filter.)  At the point where
the pulses rate exceeds twice the bandwidth of the cable,
you can't distinguish the pusles anymore.  That is similar
to the problem of the resolution of an optical system with
limited spatial frequency transmission.

-- glen
```
```On Fri, 29 Oct 2010 00:26:21 -0700 (PDT), Deamon
<persistence911@gmail.com> wrote:

>Please I am a complete newbie in Signal analysis and modelling and I
>am quite confused  about the use of  Nyquist theorem in sampling .Does
>this Nyquist criterion affect the  rate at which data is transferred
>that is  the data rate ? I have  read the wikiopedia saying that I
>must sample at twice the bandwidth to be able to recontruct a signal
>perfectly But I don't understand the concept . If I sample at 2B then
>it is bigger than the wave itself .  I did like to go on to DSP stuffs
>but I got to learn the basics first.
>
>Incase it help I have a Physics Background .

Hello Deamon,
Have a look at:

http://www.dspguide.com/ch3/2.htm

Good Luck,
[-Rick-]

```
```
Deamon wrote:
>
> Please I am a complete newbie in Signal analysis and modelling and I
> am quite confused  about the use of  Nyquist theorem in sampling .Does
> this Nyquist criterion affect the  rate at which data is transferred
> that is  the data rate ? I have  read the wikiopedia saying that I
> must sample at twice the bandwidth to be able to recontruct a signal
> perfectly But I don't understand the concept . If I sample at 2B then
> it is bigger than the wave itself .

Yes that is a good way to view it. For the highest frequency in the
signal you need at least 2 samples for each full wave period.

Suppose you have a cosine wave form and your sample rate is such that
your samples land at the following points on the cosine wave in degrees:

0 179 358 537 .....

That has 2 samples per period and it satisfies Nyquist.

Now suppose your sample rate is a little faster such that the samples
land at these following points on the cosine wave in degrees:

0 181 362 543 .......

Now you have less than 2 samples per period. But notice that your
sampled data ends up looking exactly the same as it was for the first
sequence. That is what happens when you have less than 2 samples per
period. That effect of frequencies above the Nyquist rate appearing out
of place is what is known as aliasing.

-jim
```
```On 10/29/2010 12:26 AM, Deamon wrote:
> Please I am a complete newbie in Signal analysis and modelling and I
> am quite confused  about the use of  Nyquist theorem in sampling .Does
> this Nyquist criterion affect the  rate at which data is transferred
> that is  the data rate ?

No more than quantum theory affects how electrons behave in a
semiconductor -- it's rather the other way around.

> I have  read the wikiopedia saying that I
> must sample at twice the bandwidth to be able to recontruct a signal
> perfectly But I don't understand the concept . If I sample at 2B then
> it is bigger than the wave itself .

http://www.wescottdesign.com/articles/Sampling/sampling.html

I believe it shows pretty early on why you need to sample at over 2B to
completely capture a signal.

Keep in mind that as a speed limit, capturing signals at the Nyquist
rate is a task more akin to accelerating particles to the speed of light
than it is to leaning against a door: the closer to the Nyquist rate you
try to sample, the more difficult the job gets, and the more side

> I did like to go on to DSP stuffs
> but I got to learn the basics first.
>
> Incase it help I have a Physics Background .

Actually, the Nyquist rate is derived from more basic stuff.  It helps a
lot to have a thorough understanding of signal analysis in the Fourier
domain.  If you _thoroughly_ understand the Fourier transform and if you
have a good understanding (I won't say "full") of the application of the
Dirac delta functional to Fourier transform analysis, then the Nyquist
theorem and it's many ramifications just kind of jumps out and barks at
you, then curls up by your feet for a nap.

(Note that my analysis above avoids the Dirac delta -- it's a great
shortcut, once you spend a _lot_ of time understanding it.  So for
things like casual articles, I avoid it like the plague.  Were I writing
a signal processing book, I'd make you learn it, then I'd use it all over).

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html
```
```Tim Wescott <tim@seemywebsite.com> wrote:
> On 10/29/2010 12:26 AM, Deamon wrote:
>> Please I am a complete newbie in Signal analysis and modelling and I
>> am quite confused  about the use of  Nyquist theorem in sampling .Does
>> this Nyquist criterion affect the  rate at which data is transferred
>> that is  the data rate ?

> No more than quantum theory affects how electrons behave in a
> semiconductor -- it's rather the other way around.

Actually, it is right.

The actual Nyquist problem, as I wrote previously, was how fast
telegraph pulses could be sent through a bandwidth limited medium.

(I used to have a copy of the paper.  I don't know that there are
any copies on the web, though.)

The math for sending pulses through an analog channel, and for

I haven't tried to make an argument for or against the electron
behavior question, but pretty often QM has the same symmetry.

-- glen
```
```On 10/29/2010 11:47 AM, glen herrmannsfeldt wrote:
> Tim Wescott<tim@seemywebsite.com>  wrote:
>> On 10/29/2010 12:26 AM, Deamon wrote:
>>> Please I am a complete newbie in Signal analysis and modelling and I
>>> am quite confused  about the use of  Nyquist theorem in sampling .Does
>>> this Nyquist criterion affect the  rate at which data is transferred
>>> that is  the data rate ?
>
>> No more than quantum theory affects how electrons behave in a
>> semiconductor -- it's rather the other way around.
>
> Actually, it is right.
>
> The actual Nyquist problem, as I wrote previously, was how fast
> telegraph pulses could be sent through a bandwidth limited medium.
>
> (I used to have a copy of the paper.  I don't know that there are
> any copies on the web, though.)
>
> The math for sending pulses through an analog channel, and for
>
> I haven't tried to make an argument for or against the electron
> behavior question, but pretty often QM has the same symmetry.

I think I was being excessively pedantic and stating things poorly.  I
was reacting to his phrasing, which doesn't really capture the essence
of the Nyquist/Shannon/That Russian Guy theorem -- that you can quantify
some of the information content of a signal as "bandwidth", and make a
limit out of that which relates to sampling rate.

Then I chose a poor analogy...

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html
```
```On Oct 29, 12:26&#4294967295;am, Deamon <persistence...@gmail.com> wrote:
> Please I am a complete newbie in Signal analysis and modelling and I
> am quite confused &#4294967295;about the use of &#4294967295;Nyquist theorem in sampling .Does
> this Nyquist criterion affect the &#4294967295;rate at which data is transferred
> that is &#4294967295;the data rate ? I have &#4294967295;read the wikiopedia saying that I
> must sample at twice the bandwidth to be able to recontruct a signal
> perfectly But I don't understand the concept . If I sample at 2B then
> it is bigger than the wave itself . &#4294967295;I did like to go on to DSP stuffs
> but I got to learn the basics first.
>
> Incase it help I have a Physics Background .

There are quite a few ways to approach the Nyquist-Shannon sampling
theorem. It sounds like you want an intuitively satisfying answer.

I assume you're familiar with basic Fourier series and the Fourier
transform, and the notion that a signal can be decomposed (in most
cases) to a superposition of sinusoids with different amplitudes and
phases.

That said, you should be able to understand that the baseband
bandwidth is defined by the highest frequency sinusoid in this
superposition. Let's just focus on this one frequency, and pretend
that it is the only content of your signal. So let's sample it by
hand. You get to place one point every cycle if you sample at that
frequency (at 1B as you defined it above). So where do you place it?
Every positive peak? Now give that sequence of samples to someone and
tell them to recreate the signal. Always it appears like a flat line
(either zero, or a DC offset). Now let's say I give you 2 points per
cycle (sampling at 2B as it were). Then you can place a dot at every
positive peak, as well as every negative peak. Knowing that the ideal
kernel for reconstruction is the normalized sinc function (see
Whittaker-Shannon interpolation formula, this only applies for ideal
reconstruction), one can easily recreate the signal. More intuitively,
look up the concept of zero-order hold. Using this, we would end up
with a square wave of our original frequency which could be low pass
filtered (equivalent of being convolved with a normalized sinc
function so it approximates the Whittaker-Shannon interpolation
formula or perfectly implements it for ideal brick wall low pass
filters).

Obviously this is strictly an intuitive definition and should be taken
at face value. Another way would be to discretize the signal
mathematically using the sampling rate, and then attempt to make the
waveform continuous again mathematically, and you'll see if it wasn't
sampled at the Nyquist rate, you won't end up with your original
frequency. There are many other ways of looking at this, so let me
know if you have a preferred approach and I'll do my best to explain
it that way.
```
```On Oct 29, 9:23&#4294967295;am, Tim Wescott <t...@seemywebsite.com> wrote:
> On 10/29/2010 12:26 AM, Deamon wrote:
>
> > Please I am a complete newbie in Signal analysis and modelling and I
> > am quite confused &#4294967295;about the use of &#4294967295;Nyquist theorem in sampling .Does
> > this Nyquist criterion affect the &#4294967295;rate at which data is transferred
> > that is &#4294967295;the data rate ?
>
> No more than quantum theory affects how electrons behave in a
> semiconductor -- it's rather the other way around.
>
> > I have &#4294967295;read the wikiopedia saying that I
> > must sample at twice the bandwidth to be able to recontruct a signal
> > perfectly But I don't understand the concept . If I sample at 2B then
> > it is bigger than the wave itself .
>
> If the other links that you've gotten haven't helped, read this:http://www.wescottdesign.com/articles/Sampling/sampling.html
>
> I believe it shows pretty early on why you need to sample at over 2B to
> completely capture a signal.
>
> Keep in mind that as a speed limit, capturing signals at the Nyquist
> rate is a task more akin to accelerating particles to the speed of light
> than it is to leaning against a door: the closer to the Nyquist rate you
> try to sample, the more difficult the job gets, and the more side
> effects your signal will suffer.
>
> > I did like to go on to DSP stuffs
> > but I got to learn the basics first.
>
> > Incase it help I have a Physics Background .
>
> Actually, the Nyquist rate is derived from more basic stuff. &#4294967295;It helps a
> lot to have a thorough understanding of signal analysis in the Fourier
> domain. &#4294967295;If you _thoroughly_ understand the Fourier transform and if you
> have a good understanding (I won't say "full") of the application of the
> Dirac delta functional to Fourier transform analysis, then the Nyquist
> theorem and it's many ramifications just kind of jumps out and barks at
> you, then curls up by your feet for a nap.
>
> (Note that my analysis above avoids the Dirac delta -- it's a great
> shortcut, once you spend a _lot_ of time understanding it. &#4294967295;So for
> things like casual articles, I avoid it like the plague. &#4294967295;Were I writing
> a signal processing book, I'd make you learn it, then I'd use it all over).
>
> --
>
> Tim Wescott
> Wescott Design Serviceshttp://www.wescottdesign.com
>
> Do you need to implement control loops in software?
> "Applied Control Theory for Embedded Systems" was written for you.
> See details athttp://www.wescottdesign.com/actfes/actfes.html

Excellent write-up; thank you for sharing. I was going to mention the
pitfalls of sampling right at the Nyquist rate if you happen to sample
at just the right phase but I thought I might have already overwhelmed
the OP.

And I agree. It took probably 2 years to fully (if I can truly say
that; most likely I cannot) understand frequency domain analysis for
me, and it took viewing the problem from as many angles as possible.
Actually Lyon's book does a fantastic job in retrospect and would have
shortened the time of understanding. The 3-dimensional views of the
Fourier transform are an indispensable perspective on the topic in
regard to full understanding (to the OP: he actually was the 2nd reply
```
```2010-10-29 18:23, Tim Wescott wrote:
(...)
> If the other links that you've gotten haven't helped, read this:
> http://www.wescottdesign.com/articles/Sampling/sampling.html