CAUTION! was "What is the advantage on high-sampling rate ?"

Ronald H. Nicholson Jr. wrote:

> In article <108m263idq1hr13@corp.supernews.com>,
> Tim Wescott  <tim@wescottnospamdesign.com> wrote:

>>The sampling process ends up being the mathematical equivalent of 
>>multiplying the signal by a train of dirac delta functions (impulses). 

> Didn't some UK ham radio type recently use this argument to annoy this
> newsgroup with some completely bizarre DSP theory?

> The conclusion seemed to be that there is no train of Dirac delta
> functions in the sampling process, only a train of numbers, each number
> of which could be considered as the result of the integral of a single
> impulse (approaching the Dirac delta as a limit) with some input signal
> or function.

At some point it is a philosophy question, not a science or engineering
question.  Mathematically, to see what part of the original signal
is removed by sampling, the dirac train is useful.  It is also the
result of doing a Fourier transform on a periodic function when you
forgot to notice that it was periodic (and use the Fourier series).

I do remember the discussion, though, and I have never been interested
in trying to show that things can be done without dirac functions when
they can be done easier with them.  That said, once you understand the
effect of sampling on the signal spectrum I don't see the need to go
through the step of conversion to delta functions.

-- glen

Rick Lyons wrote:
> On Sun, 25 Apr 2004 13:26:00 -0700, Tim Wescott
> <tim@wescottnospamdesign.com> wrote:
> 
> 
>    (snipped)
> 
> Hi Tim,
>   Ya' know, I read, often, in the literature of 
> DSP that "the DFT summes this about its input", 
> or "the DFT assumes that about its input".  
> This is all pure nonsense.  The DFT is not alive, 
> it has no brain with which to makes assumptions. 
>  The DFT has no eyes with which to "view" 
> anything.  The DFT is a math algorithm.  The 
> question is: "What does the DFT of a time-dmain 
> sequence represent?"

(snip)

> The DFT is a "sampled" version of the continuous 
> Fourier transform of a continuous periodic function 
> of impulses of varying amplitudes.

The way I tend to think of it, mostly because I learned Fourier
series before Fourier transform, and I believe historically
that is the order that they were done...

Well, I knew about Fourier series probably before I was in
high school, looking at waveforms on an oscilloscope with
my father, and also why triac based light dimmers cause RFI,
but it took me a while to figure out the difference between
Fourier series and transform.  When I finally did figure out
the difference it seemed so obvious, well, not that obvious
but that it shouldn't have taken so long.

Anyway, the Fourier series converts a periodic function to
a discrete sum of sines and cosines, and its inverse a
sum of sines and cosines to a periodic function.

 From that you can get the Fourier transform pairs

periodic function <---> discrete transform
non-periodic function <---> continuous transform

As they are transform pairs, you can combine them to

periodic discrete function <---> periodic discrete transform

If you take the Fourier transform on a periodic series of
delta functions you will get a periodic series of delta
functions in the transform, just like the DFT would give.

-- glen

robert bristow-johnson wrote:
> In article ih1jc.37211$IW1.1714703@attbi_s52, glen herrmannsfeldt at
> gah@ugcs.caltech.edu wrote on 04/26/2004 01:40:

>>While I do agree that the DFT is periodic (discussed in another thread),
>>I don't necessarily believe that sampling previously band limited
>>signals generates periodic signals in frequency space.

> sure it does!

>>The sampled signal is a representation of the continuous signal.

> it could.  but it wouldn't have to.  once it's sampled, that information is
> gone forever.  (at least it is not contained in the retained samples.)

It is not contained in the samples, but then neither is the information
that makes the samples more than a set of numbers.  When I buy an
audio CD from the store, it is usually implied, by the label, that
it is a sampled result of a continuous function.  It may even give
the name of the artist.

Now, I do in fact own an audio CD that is not the result of sampling
a continuous function.  It has audio test tones that were apparently
generated mathematically on a computer.

If the label gives the name of a singer I can be pretty sure that it
contains sampled data, and not extracted digitally from someone's brain.

>>It could, theoretically, be converted back to a continuous signal as a
>>sum of sinc's.

> yeah, and those sinc's all have a brickwall filter in the other (frequency)
> domain which kills off all of those repeated images.  applying those sinc's
> for reconstruction means that you are making an assumption (probably a legit
> assumption) about what the continuous signal was before sampling it.

How to best convert a sampled signal back to the appropriate
continuous form is an engineering problem.  Note that the sinc
requires t from -infinity to infinity, and so is not practical in
most cases.

>>  Other reconstruction methods generate extraneous
>>frequency components which need to be filtered out.

> that is if it is already known (or assumed) that the original continuous
> signal had no energy where those extraneous components ended up.  what if my
> original continuous signal was a piecewise linear with breakpoints exactly
> at the sampling instances.  then the correct reconstruction method would use
> triangular pulses instead of sinc's.  and those "extraneous" frequency
> components belong there.  but i am making an assumption about where those
> samples came from in the first place in doing that reconstruction.

Assuming we are talking about physical signals and not someone's
imagination I can be pretty sure the function is not piecewise linear.
Such a function requires infinite acceleration to reproduce.

-- glen

Ronald H. Nicholson Jr. wrote:

> Interesting.  The article appears to be about a point in the
> implementation of the sampling process that isn't often discussed.
> First a signal (S0) has to be low-pass filtered to removed frequency
> components with the potential to alias (S1).  

Unless one is lucky enough that it came that way.

> Then it seems for some
> implementations one then has to add a lot of high frequency content.
> This addition is due to the need to "flatten" the signal so that it
> stays sufficiently constant during a finite A/D measurement period
> (creating sample-and-hold "stair steps" for signal S2).  Of course
> flattening portions of a continuous signal implies steepening other
> portions which introduced high frequency content to this intermediate
> signal.  It's sort of counter-intuitive to consider that the more high
> frequency content introduced during this part of the process the less
> this high frequency content is represented in the final samples (as in
> the resulting sequence of numbers, S4).

For sufficiently slow signals and fast A/D converters that may not
be necessary.   For fast signals, it may be required to hold for
longer than the sampling interval.  There is one system I know of
that does high speed 8 bit conversion that goes through the process:

1) sample and hold, and do four bit A/D conversion
2) convert back to analog with a D/A converter
3) analog subtract from a copy of the original signal
    (so that step 1 can be applied to the next sample)
4) convert the difference with a four bit A/D converter.

Then there are flash A/D converters, pretty much 2**n
comparators and a priority encoder to convert to an n bit
binary number, which may be fast enough not to require a
sample and hold for most signals.

-- glen

In article %Gwjc.51070$w96.4576864@attbi_s54, glen herrmannsfeldt at
gah@ugcs.caltech.edu wrote on 04/27/2004 13:24:

> robert bristow-johnson wrote:
>> In article ih1jc.37211$IW1.1714703@attbi_s52, glen herrmannsfeldt at
>> gah@ugcs.caltech.edu wrote on 04/26/2004 01:40:
...
>>> It could, theoretically, be converted back to a continuous signal as a
>>> sum of sinc's.
> 
>> yeah, and those sinc's all have a brickwall filter in the other (frequency)
>> domain which kills off all of those repeated images.  applying those sinc's
>> for reconstruction means that you are making an assumption (probably a legit
>> assumption) about what the continuous signal was before sampling it.
> 
> How to best convert a sampled signal back to the appropriate
> continuous form is an engineering problem.  Note that the sinc
> requires t from -infinity to infinity, and so is not practical in
> most cases.
> 
>>> Other reconstruction methods generate extraneous
>>> frequency components which need to be filtered out.
> 
>> that is if it is already known (or assumed) that the original continuous
>> signal had no energy where those extraneous components ended up.  what if my
>> original continuous signal was a piecewise linear with breakpoints exactly
>> at the sampling instances.  then the correct reconstruction method would use
>> triangular pulses instead of sinc's.  and those "extraneous" frequency
>> components belong there.  but i am making an assumption about where those
>> samples came from in the first place in doing that reconstruction.
> 
> Assuming we are talking about physical signals and not someone's
> imagination I can be pretty sure the function is not piecewise linear.
> Such a function requires infinite acceleration to reproduce.

Glen,

our differences (if there are any) are mostly semantic and *theoretical*.  i
totally agree that, for the most part, the assumptions we make about
discrete-time data is that it was sampled, with equally spaced sample times,
from a continuous-time signal that was bandlimited so that to satisfy
Nyquist.  this is a most reasonable assumption about the discrete-time data
that operations such as the DFT or discrete convolution see.

but i'm usually *really* anal about semantics so that i don't make
assumptions that, if i'm doing anything weird, don't bite me later.
strictly speaking, the DFT transforms a set of N numbers, which represent a
single period of a discrete periodic sequence to a another set of N numbers,
which represent a single period of anaother discrete periodic sequence.
they *don't* necessarily have to be attached to impulses in the
continuous-time and continuous-frequency domains, but *if* they are or *if*
they are thought to be so attached, then all that you say is very true.

there are continuous-time to continuous-time or continuous-time to
continuous-frequency transformations like the convolution integral and the
Fourier Transform.

there are discrete-time to discrete-time or discrete-time to
discrete-frequency transformations like the convolution summation and the
Discrete Fourier Transform.

there are discrete-time to periodic continuous-frequency transformations
like the  Discrete Time Fourier Transform.

there are discrete-time to continuous-time transformations like the
Shannon/Nyquist/Wittiker Sampling and Reconstruction theorem.

whenever i would make a statement like you did a couple of posts ago, i
would go through more of these basic transformations than you did.  i would
not relate the DFT directly to what is happening in the continuous-time and
continuous-frequency domains, since the DFT all by itself has nothing to say
to those domains.

r b-j

robert bristow-johnson wrote:

> In article %Gwjc.51070$w96.4576864@attbi_s54, glen herrmannsfeldt at
> gah@ugcs.caltech.edu wrote on 04/27/2004 13:24:
> 
> 
>>robert bristow-johnson wrote:
>>
>>>In article ih1jc.37211$IW1.1714703@attbi_s52, glen herrmannsfeldt at
>>>gah@ugcs.caltech.edu wrote on 04/26/2004 01:40:
> 
> ...
> 
>>>>It could, theoretically, be converted back to a continuous signal as a
>>>>sum of sinc's.
>>
>>>yeah, and those sinc's all have a brickwall filter in the other (frequency)
>>>domain which kills off all of those repeated images.  applying those sinc's
>>>for reconstruction means that you are making an assumption (probably a legit
>>>assumption) about what the continuous signal was before sampling it.
>>
>>How to best convert a sampled signal back to the appropriate
>>continuous form is an engineering problem.  Note that the sinc
>>requires t from -infinity to infinity, and so is not practical in
>>most cases.
>>
>>
>>>>Other reconstruction methods generate extraneous
>>>>frequency components which need to be filtered out.
>>
>>>that is if it is already known (or assumed) that the original continuous
>>>signal had no energy where those extraneous components ended up.  what if my
>>>original continuous signal was a piecewise linear with breakpoints exactly
>>>at the sampling instances.  then the correct reconstruction method would use
>>>triangular pulses instead of sinc's.  and those "extraneous" frequency
>>>components belong there.  but i am making an assumption about where those
>>>samples came from in the first place in doing that reconstruction.
>>
>>Assuming we are talking about physical signals and not someone's
>>imagination I can be pretty sure the function is not piecewise linear.
>>Such a function requires infinite acceleration to reproduce.
> 
> 
> Glen,
> 
> our differences (if there are any) are mostly semantic and *theoretical*.  i
> totally agree that, for the most part, the assumptions we make about
> discrete-time data is that it was sampled, with equally spaced sample times,
> from a continuous-time signal that was bandlimited so that to satisfy
> Nyquist.  this is a most reasonable assumption about the discrete-time data
> that operations such as the DFT or discrete convolution see.
> 
> but i'm usually *really* anal about semantics so that i don't make
> assumptions that, if i'm doing anything weird, don't bite me later.
> strictly speaking, the DFT transforms a set of N numbers, which represent a
> single period of a discrete periodic sequence to a another set of N numbers,
> which represent a single period of anaother discrete periodic sequence.
> they *don't* necessarily have to be attached to impulses in the
> continuous-time and continuous-frequency domains, but *if* they are or *if*
> they are thought to be so attached, then all that you say is very true.
> 
> there are continuous-time to continuous-time or continuous-time to
> continuous-frequency transformations like the convolution integral and the
> Fourier Transform.
> 
> there are discrete-time to discrete-time or discrete-time to
> discrete-frequency transformations like the convolution summation and the
> Discrete Fourier Transform.
> 
> there are discrete-time to periodic continuous-frequency transformations
> like the  Discrete Time Fourier Transform.
> 
> there are discrete-time to continuous-time transformations like the
> Shannon/Nyquist/Wittiker Sampling and Reconstruction theorem.
> 
> whenever i would make a statement like you did a couple of posts ago, i
> would go through more of these basic transformations than you did.  i would
> not relate the DFT directly to what is happening in the continuous-time and
> continuous-frequency domains, since the DFT all by itself has nothing to say
> to those domains.

I really hate replying to a long post with just a few lines at the
end, but I think I agree with all the comments about my previous post.

Some of the points I was trying to make are that DFT represents
periodic functions, and that sampling doesn't require, but can
be described using, dirac functions.

-- glen

r.lyons@_BOGUS_ieee.org (Rick Lyons) wrote in message news:<408922bf.1261440968@news.sf.sbcglobal.net>...
> Hi Guys,
>   in a recent thread, mention was made of a "Sampling" 
> paper by Dan Lavry.  At the following web site
> 
> http://www.lavryengineering.com/pdfs/sample.pdf
> 
> you can see Dan's 1997 paper:
> "Sampling, Oversampling, Imaging and 
> Aliasing - a basic tutorial".
> 
> 
> I recommend caution if you decide to
> read that paper.  In the second paragraph Dan wrote:
> 
>   *** Sampling theory ***
>   Sampling a value, in the Nyquist sense amounts to 
>   "pinning down" signal values at equal time intervals. 
>   Let us "cover much of the audio band" by summing four 
>   equal amplitude tones (5,10,15 and 20KHz). The time 
>   domain plot shows the resulting continues (analog) 
>   waveform and it's sampled counterpart (44.1KHz sampling). 
>   The frequency domain plot shows the energy concentration 
>   of the sampled signal across 0 to 110KHz (tones exist 
>   beyond 110KHz). The frequency domain plot shows the 
>   energy concentration of the sampled signal across 0 to 
>   110KHz (tones exist beyond 110KHz). The sampled signal 
>   contains the four tones bellow 22KHz and undesirable 
>   energy at frequencies above 22KHz. Recovering the analog 
>   signal requires no more then removal of all energy 
>   above 22KHz.
> 
>           [The misspelled words are Dan's, not mine.]
> 

Hi Rick,

I was just curious so I ran the text through a spell-checker and only
found 'analog' as misspelled. Are there others?
Or is it possibly that my spell-checker is set up for South African
English and an American English checker would pick up other errors?

Regards
Robert

Robert Gush wrote:

   ...

>>  *** Sampling theory ***
>>  Sampling a value, in the Nyquist sense amounts to 
>>  "pinning down" signal values at equal time intervals. 
>>  Let us "cover much of the audio band" by summing four 
>>  equal amplitude tones (5,10,15 and 20KHz). The time 
>>  domain plot shows the resulting continues (analog) 
>>  waveform and it's sampled counterpart (44.1KHz sampling). 
>>  The frequency domain plot shows the energy concentration 
>>  of the sampled signal across 0 to 110KHz (tones exist 
>>  beyond 110KHz). The frequency domain plot shows the 
>>  energy concentration of the sampled signal across 0 to 
>>  110KHz (tones exist beyond 110KHz). The sampled signal 
>>  contains the four tones bellow 22KHz and undesirable 
>>  energy at frequencies above 22KHz. Recovering the analog 
>>  signal requires no more then removal of all energy 
>>  above 22KHz.
>>
>>          [The misspelled words are Dan's, not mine.]
>>
> 
> 
> Hi Rick,
> 
> I was just curious so I ran the text through a spell-checker and only
> found 'analog' as misspelled. Are there others?
> Or is it possibly that my spell-checker is set up for South African
> English and an American English checker would pick up other errors?

"Analog" is proper U.S. spelling. The first time through, I thought I 
saw one spelling error, but I see none now. I do, however, deplore the 
lack of hyphens. What are amplitude tones, and how does one know if four 
of them are equal? (I can decipher the meaning: equal-amplitude tones. A 
reader shouldn't need to decipher.) Likewise, what is a domain plot?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

robert bristow-johnson wrote:
>>Robert, you will absolutely not lure me back into saying
>>that the DFT merely compiles the correlations of an interval
>>of a signal with all the sinusoids and cosinusoids that have
>>an integral number of cycles in the same length interval (up
>>to the Nyquist one) and that it says nothing whatsoever
>>about anything outside that interval.  I absolutely refuse
>>to restate my position.  :-)
> 
> 
> okay in the same spirit, i absolutely refuse to decode it either.
> 
> 
> ???

You really didn't understand that?  Granted it's concise but 
I'm not sure how else to not state it.


Bob
-- 

"Things should be described as simply as possible, but no 
simpler."

                                              A. Einstein

In article c6okrl02rmi@enews3.newsguy.com, Bob Cain at
arcane@arcanemethods.com wrote on 04/28/2004 12:07:

> 
> 
> robert bristow-johnson wrote:
>>> Robert, you will absolutely not lure me back into saying
>>> that the DFT merely compiles the correlations of an interval
>>> of a signal with all the sinusoids and cosinusoids that have
>>> an integral number of cycles in the same length interval (up
>>> to the Nyquist one) and that it says nothing whatsoever
>>> about anything outside that interval.  I absolutely refuse
>>> to restate my position.  :-)
>> 
>> 
>> okay in the same spirit, i absolutely refuse to decode it either.
>> 
>> 
>> ???
> 
> You really didn't understand that?  Granted it's concise but
> I'm not sure how else to not state it.

okay, so you make me parse it:

>>> ... you will absolutely not lure me back into saying

the follow you will not say:

>>> that the DFT merely compiles the correlations of an interval of a signal

an "interval of a signal" is going to be a finite length segment of a
signal, i presume by this time it has been sampled so that finite length is
a finite number of samples, no?  we'll call that number, N.

>>> with all the sinusoids and cosinusoids that have an integral number of
>>> cycles in the same length interval (up to the Nyquist one)

okay, i got that one (i presume that you're using the Nyquist one since that
appears in bin N/2 of the DFT if N is even).  but you're not going over the
Nyquist bin.

>>> and that it says nothing whatsoever about anything outside that interval.

so you're refuse to say "that it says nothing whatsoever..." or that, if you
would say it, "that it says nothing whatsoever about anything outside that
interval."  *that's* a little ambiguous.  from the past, i think you believe
that it *does* say something about something outside that interval.  no?

if that *is* the case, we have a disagreement whereas i would say that
neither the DFT nor those N samples tell you what is going on outside that
interval.  you are welcome to say that it's, say, zero outside that interval
(and then the consequence of that *assumption* is that the DFT is the
sampling of N samples of the DTFT around the unit circle).  i insist that
it's an assumption that may be reasonable in some contexts, but neither
those N samples nor the DFT of them give you that information.

>>>  I absolutely refuse to restate my position.  :-)

let's see where this gets us, Bob.  i promise to be nice (i.e. respectful),
but, as you might guess, i take no prisoners regarding the content of the
discussion.