On Fri, 29 Sep 2006 13:51:42 -0700, John Larkin <jjlarkin@highNOTlandTHIStechnologyPART.com> wrote:><snip> >The wiki article has links to Nyquist's 1928 paper and to Shannon's >1949 paper. Few EEs that I've met have ever read either of them, and >have absorbed "the Nyquist theorem" mostly by hearsay, which I guess >is your point.I've read Shannon's paper, thorougly. At least, these: The Bell System Technical Journal, Vol. 27, pp. 379�423, 623�656, July, October, 1948. Note that this is NOT 1949. But it was, in fact, what made me understand Boltzmann much better than before and allowed me to better access the underlying meaning of macro concepts such as temperature and entropy (which have no micro-meaning.) Hamming, Shannon, and Golay all worked together in the same place, if I recall, around that time. Marcel J. E. Golay, 1949, and a little (short) paper called, "Notes on Digital Coding" which came out in 1949. (I think he was pushed into it by Hamming and Shannon.) Jon
What Nyquist Didn't Say
Started by ●September 29, 2006
Reply by ●September 29, 20062006-09-29
Reply by ●September 29, 20062006-09-29
In the first sentence of your papere, you say that if the signal is band limited to fo or less, then a sample frequency of 2fo or more is adequate to contain completely all the information necessary to recreate the signal. My understanding from water cooler conservation is that the signal bandwidth must be strictly less than fo for a sample frequency of 2fo and that the signal must be infinite in extent to allow perfect reconstruction. IMHO. I could be wrong. I have never seen a formal proof just some handwaving. In article <UaOdnVhcfetF6IDYnZ2dnUVZ_oudnZ2d@web-ster.com>, Tim Wescott <tim@seemywebsite.com> wrote:>I've seen a lot of posts over the last year or so that indicate a lack >of understanding of the implications of the Nyquist theory, and just >where the Nyquist rate fits into the design of sampled systems. > >So I decided to write a short little article to make it all clear. > >It's a little longer than 'short', and it took me way longer than I >thought it would, but at least it's done and hopefully it's clear. > >You can see it at >http://www.wescottdesign.com/articles/Sampling/sampling.html. > >If you're new to this stuff, I hope it helps. If you're an expert and >you have the time, please feel free to read it and send me comments or >post them here. >
Reply by ●September 29, 20062006-09-29
"Tim Wescott" <tim@seemywebsite.com> wrote in message news:UaOdnVhcfetF6IDYnZ2dnUVZ_oudnZ2d@web-ster.com...> I've seen a lot of posts over the last year or so that indicate a lack > of understanding of the implications of the Nyquist theory, and just > where the Nyquist rate fits into the design of sampled systems. > > So I decided to write a short little article to make it all clear. > > It's a little longer than 'short', and it took me way longer than I > thought it would, but at least it's done and hopefully it's clear. > > You can see it at > http://www.wescottdesign.com/articles/Sampling/sampling.html. > > If you're new to this stuff, I hope it helps. If you're an expert and > you have the time, please feel free to read it and send me comments or > post them here. > > -- > > Tim Wescott > Wescott Design Services > http://www.wescottdesign.com > > Posting from Google? See http://cfaj.freeshell.org/google/ > > "Applied Control Theory for Embedded Systems" came out in April. > See details at http://www.wescottdesign.com/actfes/actfes.htmlExccept it was Shannon that said it... -- Posted via a free Usenet account from http://www.teranews.com
Reply by ●September 30, 20062006-09-30
On Fri, 29 Sep 2006 22:00:34 GMT, I wrote:>On Fri, 29 Sep 2006 13:51:42 -0700, John Larkin ><jjlarkin@highNOTlandTHIStechnologyPART.com> wrote: > >><snip> >>The wiki article has links to Nyquist's 1928 paper and to Shannon's >>1949 paper. Few EEs that I've met have ever read either of them, and >>have absorbed "the Nyquist theorem" mostly by hearsay, which I guess >>is your point. > >I've read Shannon's paper, thorougly. At least, these: The Bell >System Technical Journal, Vol. 27, pp. 379�423, 623�656, July, >October, 1948. Note that this is NOT 1949. But it was, in fact, what >made me understand Boltzmann much better than before and allowed me to >better access the underlying meaning of macro concepts such as >temperature and entropy (which have no micro-meaning.) > >Hamming, Shannon, and Golay all worked together in the same place, if >I recall, around that time. Marcel J. E. Golay, 1949, and a little >(short) paper called, "Notes on Digital Coding" which came out in >1949. (I think he was pushed into it by Hamming and Shannon.)Some more info and a correction. First, the correction. Golay didn't actually with Shannon at Bell labs. His first paper, the one I mentioned called "Notes on Digital Coding" was actually published in the Correspondence section of Proc. I.R.E., 37, 657 (1949) was written while he was at the Signal Corps Engineering Laboratories in Fort Monmoth, N.J. Now, the additional info. Golay's 1949 paper is supplemented by two more papers he wrote: "Binary Coding" I.R.E. Trans. Inform. Theory, PGIT-4, 23-28 (1954) -- written also from Fort Monmoth, NJ; and "Notes on the Penny-Weighing Problem, Lossless Symbol Coding with Nonprimes, etc.," I.R.E Trans. Inform. Theory, IT-4, 103-109 (1958) -- written from Philco Corporation when in Philadelphia, Pa. Twelve of Shannon's papers (1948 through 1967) are conveniently collected in the anthology, "Key Papers in the Development of Information Theory," edited by David Slepian (IEEE Press, 1974.) Shannon referenced Golay's 1949 paper in the book "The Mathematical Theory of Communication" (written with Warren Weaver, Univ. Illinois Press, 1949.) This book contains a slightly rewritten version of Shannon's first 1948 papers together with a popular-level paper by Weaver. Shannon describes the Hamming-7 code in his 1948 papers in section 17, attributed to Hamming there, but since there is no reference to a specific paper by Hamming I suspect this reference must have been via personal communication with Hamming. (Golay also refers to the Hamming-7 code in Shannon's first paper.) The first paper by Hamming is "Error Detecting and Error Correcting Codes" Bell System Tech. J., 29, 147-160 (1950.) Note this is actually _after_ Shannon's reference to Hamming's code. The anthology, "Algebraic Coding Theory: History and Development," edited by Ian F. Blake (Dowden, Hutchinson & Ross, 1973) includes this paper. Blake says in his introduction to the first 9 papers in his anthology: "The first nontrivial example of an error-correcting code appears, appropriately enough, in the classical paper of Shannon in 1948. This code would today be called the (7,4) Hamming code, containing 16 = 2^4 codewords of length 7, and its construction was credited to Hamming by Shannon. Golay gives a construction that generalizes this code over GF (p), p a prime number, of length (p^n -1)/(p - 1) for some positive integer n. Hamming also obtained the same generalization of his example of codes of length (2^n - 1) over GF(2) and investigates their structure and decoding in some depth. The codes of both Golay and Hamming are now designated as Hamming codes. The interest of Golay was in perfect codes, which have also been called lossless, or close-packed, codes. Since he mentions the binary repetition codes and gives explicit constructions for his remarkable (23,11) binary and (11,6) ternary codes, it is not stretching a point to say that in the first paper written specifically on error-correcting codes, a paper that occupied, in its entirety, only half a journal page, Golay found essentially all the linear perfect codes which are known today. ... The multiple error-correcting perfect codes of Golay, now called Golay codes, have inspired enough papers to fill a separate volume." By the way, the Hamming-7 code can be used to generate the E7 root lattice, which corresponds to the E7 Lie algebra. And the Hamming-8 code similarly generates the E8 root lattice, corresponding to the E8 Lie algebra. Heterotic superstring theory has an E8 x E8 symmetry which is needed for anomaly cancellation. It is nifty that the very first error-correcting codes of Golay and Hamming play such a profound role in modern superstring theory. An interesting supplemental work is from J. H. Conway and N. J. A. Sloane, "Sphere Packings, Lattices and Groups" from Springer-Verlag, 1988. But probably the best ever book on algebraic coding theory is: "The Theory of Error-correcting Codes" by F. J. MacWilliams and N. J. A. Sloane, North-Holland Publishing Co., 1977. Jon
Reply by ●September 30, 20062006-09-30
On 2006-09-29, Tim Wescott <tim@seemywebsite.com> wrote:> I've seen a lot of posts over the last year or so that indicate a lack > of understanding of the implications of the Nyquist theory, and just > where the Nyquist rate fits into the design of sampled systems. > > So I decided to write a short little article to make it all clear. > > It's a little longer than 'short', and it took me way longer than I > thought it would, but at least it's done and hopefully it's clear. > > You can see it at > http://www.wescottdesign.com/articles/Sampling/sampling.html. > > If you're new to this stuff, I hope it helps. If you're an expert and > you have the time, please feel free to read it and send me comments or > post them here.No expert, but figure 14 could possibly be improved by explicitly showing the rising edge of the pulse, maybe start the time axis at -0.5 or start the pulse at +0.5 The "contact us" link appears to be broken too. -- Bye. Jasen
Reply by ●September 30, 20062006-09-30
On Sat, 30 Sep 2006 04:05:32 GMT, Jonathan Kirwan <jkirwan@easystreet.com> wrote:>Some more info and a correction.You guys are good. I doubt I'll ever read all that myself, but I do appreciate the work and motivation it took to take the time to care and find it all. I'll only say that I hope all of you all keep your enthuiasm to dig, find the best answer, and share. It's why I hang out here, read, and snipe occasionally. As Martha said, even back before she became a capitalistic criminal, "It's a good thing."
Reply by ●September 30, 20062006-09-30
Tim Wescott skrev:> I've seen a lot of posts over the last year or so that indicate a lack > of understanding of the implications of the Nyquist theory, and just > where the Nyquist rate fits into the design of sampled systems. > > So I decided to write a short little article to make it all clear. > > It's a little longer than 'short', and it took me way longer than I > thought it would, but at least it's done and hopefully it's clear. > > You can see it at > http://www.wescottdesign.com/articles/Sampling/sampling.html. > > If you're new to this stuff, I hope it helps. If you're an expert and > you have the time, please feel free to read it and send me comments or > post them here.Tim, I really like your style of writing. Accessible, and yet with no hint at the "for dummies" craze. I hadn't really thought of the Signal-to-Aliased-Energy ratio as a metric for signal quality before. It will be a lot easier to discuss the quality of sampled systems with that as a tool. Very good presentation of the both the concept and the effect. I whish it was me who thought of that sequence of figures 6-9... oh well. A couple of very subjective comments: - There is something about the typography of the page that annoys me. I can't really see if there is an open line between paragraphs inside a section; if there is, you may want to make it clearer - The footnote indicators are WAY too small to be useful, or even seen. - I can't see why you need the maths paragraph ("To understand aliasing...", eqs. (2),(3)) at all in section 2.1. I think that what you say there is covered by the plain-text parts of the section. Lay-men might find that maths paragraph scary, throwing them off your article, while technicians know it already. - Similar comments apply for eq. (4). I can't see that it is strictly necessary, the explanation is in the text anyway. - Eqs. (5)-(7) seem to be necessary, but are the only big equations in the article. Hmm... that makes me wonder... As far as I can tell, you are 99.9% at the point where this is an article a non-engineer layman can read and understand. If you find a way to get rid of equations 1-7, maybe even eq. 8, you ought to be there. Impressive work! Rune
Reply by ●September 30, 20062006-09-30
Tim Wescott wrote:> I've seen a lot of posts over the last year or so that indicate a lack > of understanding of the implications of the Nyquist theory, and just > where the Nyquist rate fits into the design of sampled systems. > > So I decided to write a short little article to make it all clear. > > It's a little longer than 'short', and it took me way longer than I > thought it would, but at least it's done and hopefully it's clear. > > You can see it at > http://www.wescottdesign.com/articles/Sampling/sampling.html. > > If you're new to this stuff, I hope it helps. If you're an expert and > you have the time, please feel free to read it and send me comments or > post them here. > > -- > > Tim Wescott > Wescott Design Services > http://www.wescottdesign.com > > Posting from Google? See http://cfaj.freeshell.org/google/ > > "Applied Control Theory for Embedded Systems" came out in April. > See details at http://www.wescottdesign.com/actfes/actfes.htmlVery good. I would change this phrase: "This will allow you to use a more open anti-aliasing filter." By "more open", I gather you mean a larger transition ratio, which lowers the Q of the resonators. Note that the Bessel filter will ring at higher orders. I don't have my copy of Zverev handy, but I think the Bessel rings at 4th order and higher. The Gaussian filter doesn't ring at any order. The key is to look at the impulse response of the filter. If it ever goes negative, then filter will ring. You might want to go into the inverse sinc response requirements in the recontruction filter.
Reply by ●September 30, 20062006-09-30
On 29 Sep 2006 22:33:21 -0700, "Rune Allnor" <allnor@tele.ntnu.no> wrote:>> http://www.wescottdesign.com/articles/Sampling/sampling.html.>As far as I can tell, you are 99.9% at the point where this is an >article a non-engineer layman can read and understand. If you >find a way to get rid of equations 1-7, maybe even eq. 8, you >ought to be there.Since understanding the formula _notation_ is not essential for understanding most of the rest of the text, I would suggest moving the formulas into separate boxes and moving these boxes out of the direct text flow e.g. into a box in the right margin of the paper.>Impressive work!Definitively ! The problem with quite a few text dealing with sampling is that they are written by mathematicians for the mathematicians. However, these days, most people using various DSP algorithms are programmers, not mathematicians, thus the terse mathematical notation can be hard to understand to them, especially without too much numerical analysis background. Instead of using the terse mathematical notation, it might be more productive for most readers to publish e.g. a Fortran/Algol/C algorithm. Paul
Reply by ●September 30, 20062006-09-30
miso@sushi.com wrote:> Note that the Bessel filter will ring at higher orders. I don't have my > copy of Zverev handy, but I think the Bessel rings at 4th order and > higher.Are you sure about that? Here is a 9th order 1 MHz Bessel for LTspice. (Save as a CKT file) * UTS Mike Monett * Converted From Micro Cap Source file to LTspice * C1 0 1 248.3PF C2 0 2 1200.0PF C3 0 3 2007.3PF C4 0 4 2749.9PF C5 0 Vout 7209.4PF L1 1 2 1.8UH L2 2 3 4.1UH L3 3 4 5.9UH L4 4 Vout 8.6UH R1 Vin 1 50 R5 0 Vout 50 V1 Vin 0 DC 0 PULSE (0 1 0 0 0 2.5e-006 5e-006) .TRAN 1e-008 10u 0 1n UIC .PRINT TRAN V(VOUT) V(VIN) .PLOT TRAN V(VOUT) V(VIN) .PROBE .END ;$SpiceType=SPICE3 It doesn't ring. Regards, Mike Monett Antiviral, Antibacterial Silver Solution: http://silversol.freewebpage.org/index.htm SPICE Analysis of Crystal Oscillators: http://silversol.freewebpage.org/spice/xtal/clapp.htm Noise-Rejecting Wideband Sampler: http://www3.sympatico.ca/add.automation/sampler/intro.htm
