Forums

Sampling Theorem history

Started by Tom August 10, 2003
It is generally credited that the Sampling Theorem is due to fistly
the Mathematician Whittaker and Shannon and the Russian Kotelnikov. I
have no doubt that Whittaker was first but was Shannon aware of
Whittakers work? Also where does the Russian engineer fit in? What
role did Nyquist play. We talk of the Nyquist frequency (half
sampling) but why name it this if the work is due to Shannon? Or was
the Bell labs work really due to Nyquist?


Tom
Tom,

If you read Shannon's classic paper, he describes the work of these other
gentlemen. He also cites in such a way that this was common knowledge to
those versed in the field. But Shannon was the first to formalize the
theory, and that is why his name is attached.

To find the historical bits, 1st read Shannon's paper. Then you can also
find a paper entitled "Sampling: 50 years after Shannon." by Michael Unser

Look here for Unser's paper:

http://bigwww.epfl.ch/publications/unser0001.pdf

And an introduction to the above paper:

http://bigwww.epfl.ch/publications/unser0001prolog.pdf

Shannon's paper should be easy to find.

Clay



"Tom" <aberdonian_2000@yahoo.com> wrote in message
news:e1b1658f.0308101351.ecaf1ca@posting.google.com...
> It is generally credited that the Sampling Theorem is due to fistly > the Mathematician Whittaker and Shannon and the Russian Kotelnikov. I > have no doubt that Whittaker was first but was Shannon aware of > Whittakers work? Also where does the Russian engineer fit in? What > role did Nyquist play. We talk of the Nyquist frequency (half > sampling) but why name it this if the work is due to Shannon? Or was > the Bell labs work really due to Nyquist? > > > Tom
aberdonian_2000@yahoo.com (Tom) wrote in message news:<e1b1658f.0308101351.ecaf1ca@posting.google.com>...
> It is generally credited that the Sampling Theorem is due to fistly > the Mathematician Whittaker and Shannon and the Russian Kotelnikov. I > have no doubt that Whittaker was first but was Shannon aware of > Whittakers work? Also where does the Russian engineer fit in? What > role did Nyquist play. We talk of the Nyquist frequency (half > sampling) but why name it this if the work is due to Shannon? Or was > the Bell labs work really due to Nyquist? > > > Tom
Nyquist found : You cannot send more than 2 symbols in 1 Hz of bandidth with no ISI. This is related to communication Shnnon said : you need at least 2*F independent values to reconstruct a signal with bandwidth F. This is related to signal interpolation. I have read in a paper that these are two independent results. Factor 2 is a coincidence. I am puzzled why nyquist limit is called so.

"Clay S. Turner" wrote:

> Tom, > > If you read Shannon's classic paper, he describes the work of these other > gentlemen. He also cites in such a way that this was common knowledge to > those versed in the field. But Shannon was the first to formalize the > theory, and that is why his name is attached. > > To find the historical bits, 1st read Shannon's paper. Then you can also > find a paper entitled "Sampling: 50 years after Shannon." by Michael Unser > > Look here for Unser's paper: > > http://bigwww.epfl.ch/publications/unser0001.pdf > > And an introduction to the above paper: > > http://bigwww.epfl.ch/publications/unser0001prolog.pdf > > Shannon's paper should be easy to find. > > Clay > > "Tom" <aberdonian_2000@yahoo.com> wrote in message > news:e1b1658f.0308101351.ecaf1ca@posting.google.com... > > It is generally credited that the Sampling Theorem is due to fistly > > the Mathematician Whittaker and Shannon and the Russian Kotelnikov. I > > have no doubt that Whittaker was first but was Shannon aware of > > Whittakers work? Also where does the Russian engineer fit in? What > > role did Nyquist play. We talk of the Nyquist frequency (half > > sampling) but why name it this if the work is due to Shannon? Or was > > the Bell labs work really due to Nyquist? > > > > > > Tom
That's not my understanding. I have read Whittakers paper and he definately pre-dates Shannon but Shannon in my book was the first person to realise the engineering implications of this. I too am unsure what Nyquist has to do with it! In maths circles they talk of the Whittaker-Shannon sampling theory. Tom
"Tom" <somebody@nOpam.com> wrote in message
news:3F385ED5.61B66580@nOpam.com...
> > That's not my understanding. I have read Whittakers paper and he
definately
> pre-dates Shannon but Shannon in my book was the first person to realise
the
> engineering implications of this. I too am unsure what Nyquist has to do
with
> it! > In maths circles they talk of the Whittaker-Shannon sampling theory.
Hello Tom, If you read Shannon's paper, you will find he gives a reference to Whittaker's work. Clearly this means he knows that Whittaker predated him. Shannon also never claimed this was his. Are you responding to my post or another one? Clay
> > Tom > >
Hello Tom, et. al.,

You may find the following link useful

http://imagescience.bigr.nl/meijering/research/chronology/

Clay



> Nyquist found : You cannot send more than 2 symbols in 1 Hz of > bandidth with no ISI. > > This is related to communication > > Shnnon said : you need at least 2*F independent values to reconstruct > a signal with bandwidth F. > > This is related to signal interpolation. > > I have read in a paper that these are two independent results. Factor 2 > is a coincidence. > > I am puzzled why nyquist limit is called so.
Sorry to join in on the discussion so late in the day. However, the truth is that Harry Nyquist discovered the sampling theorem, which is correctly ascribed to him (and Shannon states this quite openly). In Nyquist's 1928 paper "Certain Topics in Telegraph Transmission Theory" he first defines sampling 1) ... time is divided into equal units which will be called time units 2) There is a finite number of conditions and each time unit is characterized by a single one of these conditions so not only does Nyquist use discrete time, he realized that practical sampling will have quantized amplitudes as well. In the section entitled "Analysis of a DC wave" (what he calls DC is what we would now term "low-pass signal", i.e. a signal with components from DC up to a certain frequency) Nyquist uses Fourier techniques to show that if there are 2W time units per second, then the bandwidth must be limited to W. His reasoning is correct, if perhaps slightly less than a strict mathematical proof. "... a number of components equal to one-half the number of signal elements should be sufficient to determine the a's (the Fourier coefficients) completely". Furthermore Nyquist also understood the more general case of bandpass sampling. In his 1924 paper he defined "The calculated difference between the maximum rate of oscillation (or highest frequency) and the minimum rate of oscillation (or lowest frequency) of the sine waves that compose a complex signal is known as the bandwidth of the signal." Finally, the factor of two in both Nyquist and Shannon theorems is NOT coincidental. That's why Shannon needed Nyquist's theorem! Y(J)S
Does anyone know of a source for an electronic copy of Nyquist's 1924
paper?   I did a couple of searches on Google and couldn't sort
anything out other than lots of websites that summarize the idea.

Thanks.

Cheers,

Eric



On Tue, 12 Aug 2003 19:24:51 +0300, "Jonathan Y Stein"
<author@dspcsp.com> wrote:

>> Nyquist found : You cannot send more than 2 symbols in 1 Hz of >> bandidth with no ISI. >> >> This is related to communication >> >> Shnnon said : you need at least 2*F independent values to reconstruct >> a signal with bandwidth F. >> >> This is related to signal interpolation. >> >> I have read in a paper that these are two independent results. Factor 2 >> is a coincidence. >> >> I am puzzled why nyquist limit is called so. > >Sorry to join in on the discussion so late in the day. > >However, the truth is that Harry Nyquist discovered the sampling >theorem, which is correctly ascribed to him (and Shannon >states this quite openly). > >In Nyquist's 1928 paper "Certain Topics in Telegraph Transmission Theory" >he first defines sampling > 1) ... time is divided into equal units which will be called time units > 2) There is a finite number of conditions and each time unit is >characterized > by a single one of these conditions >so not only does Nyquist use discrete time, he realized that practical >sampling >will have quantized amplitudes as well. > >In the section entitled "Analysis of a DC wave" (what he calls DC >is what we would now term "low-pass signal", i.e. a signal with components >from DC up to a certain frequency) Nyquist uses Fourier techniques >to show that if there are 2W time units per second, then the bandwidth >must be limited to W. His reasoning is correct, if perhaps slightly less >than a strict mathematical proof. "... a number of components equal to >one-half the number of signal elements should be >sufficient to determine the a's (the Fourier coefficients) completely". > >Furthermore Nyquist also understood the more general case >of bandpass sampling. In his 1924 paper he defined >"The calculated difference between the maximum rate of oscillation >(or highest frequency) and the minimum rate of oscillation (or lowest >frequency) >of the sine waves that compose a complex signal >is known as the bandwidth of the signal." > >Finally, the factor of two in both Nyquist and Shannon theorems >is NOT coincidental. That's why Shannon needed Nyquist's theorem! > >Y(J)S > > >
Eric Jacobsen Minister of Algorithms, Intel Corp. My opinions may not be Intel's opinions. http://www.ericjacobsen.org
"Tom" <aberdonian_2000@yahoo.com> wrote in message
news:e1b1658f.0308101351.ecaf1ca@posting.google.com...
> It is generally credited that the Sampling Theorem is due to fistly > the Mathematician Whittaker and Shannon and the Russian Kotelnikov. I > have no doubt that Whittaker was first but was Shannon aware of > Whittakers work? Also where does the Russian engineer fit in? What > role did Nyquist play. We talk of the Nyquist frequency (half > sampling) but why name it this if the work is due to Shannon? Or was > the Bell labs work really due to Nyquist?
I used to have a copy of Nyquists paper. You should be able to find it in the engineering library of most universities. He was pretty much doing the opposite problem, how fast he could send telegraph pulses down a band limited transmission line. The math is the same. -- glen
Tom <somebody@nopam.com> wrote:
> That's not my understanding. I have read Whittakers paper and he definately > pre-dates Shannon but Shannon in my book was the first person to realise the > engineering implications of this. I too am unsure what Nyquist has to do with > it! > In maths circles they talk of the Whittaker-Shannon sampling theory.
Current (politically correct) IEEE circles call them Whittaker-Kotelnikov-Shannon(WKS) theorem :) In http://citeseer.nj.nec.com/schilling01some.html there are two Whittakers: What we normally think of as the sampling theorem is the theorem named after E. T. Whittaker, J. M. Whittaker, V. A. Kotelnikov or C. E. Shannon [Whi15, Whi29, Whi35] Kot33] [Sha49] Once I saw some referrences to Cauchy work and even ancient Midl East people.