John Monro wrote:
> On 24/12/2010 12:08 PM, Phil Hobbs wrote:
>> John Monro wrote:
>>> On 24/12/2010 3:26 AM, Phil Hobbs wrote:
>>>> John Monro wrote:
>>>>> On 23/12/2010 4:56 PM, Phil Hobbs wrote:
>>>>>> John Monro wrote:
>>>>>>> On 23/12/2010 12:40 PM, Phil Hobbs wrote:
>>>>>>>> John Monro wrote:
>>>>>>>>> On 23/12/2010 8:07 AM, Phil Hobbs wrote:
>>>>>>>>>> Tim Wescott wrote:
>>>>>>>>>>> On Wed, 22 Dec 2010 20:24:41 +1100, John Monro wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On 22/12/2010 1:00 PM, Tim Wescott wrote:
>>>>>>>>>>>>> On 12/21/2010 04:10 PM, John Monro wrote:
>>>>>>>>>>>>>> On 21/12/2010 8:15 AM, Tim Wescott wrote:
>>>>>>>>>>>>>>> I'm starting a new thread (from "Sampling: What
>>>>>>>>>>>>>>> Nyquist Didn't Say,
>>>>>>>>>>>>>>> and What to Do About It"), because the old one
>>>>>>>>>>>>>>> rapidly
>>>>>>>>>>>>>>> filled up with
>>>>>>>>>>>>>>> all sorts of interesting stuff that I didn't
>>>>>>>>>>>>>>> want to
>>>>>>>>>>>>>>> detract from.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've posted a new version. It uses Bitstream
>>>>>>>>>>>>>>> fonts
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> Roman -- used
>>>>>>>>>>>>>>> because it was mentioned, and because it was
>>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>> It's still a
>>>>>>>>>>>>>>> serif font which isn't optimum for monitor
>>>>>>>>>>>>>>> viewing,
>>>>>>>>>>>>>>> but I want the
>>>>>>>>>>>>>>> thing to look good when it's printed (and I'm
>>>>>>>>>>>>>>> lazy
>>>>>>>>>>>>>>> about figuring out
>>>>>>>>>>>>>>> how to tell Lyx/LaTeX how to use sans!).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's 12-point, so you won't have to squint to
>>>>>>>>>>>>>>> see it,
>>>>>>>>>>>>>>> or have as much
>>>>>>>>>>>>>>> trouble scanning across the line. It certainly
>>>>>>>>>>>>>>> looks
>>>>>>>>>>>>>>> better in
>>>>>>>>>>>>>>> Evince, and I'm about to find out how it looks in
>>>>>>>>>>>>>>> Adobe, on my wife's
>>>>>>>>>>>>>>> computer upstairs.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> And Randy, I've changed the discussion of
>>>>>>>>>>>>>>> subsampling
>>>>>>>>>>>>>>> to make it more
>>>>>>>>>>>>>>> clear -- I hope that if it doesn't fully answer
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>> difficulties (I
>>>>>>>>>>>>>>> think you thought I was claiming to sample at an
>>>>>>>>>>>>>>> effectively infinite
>>>>>>>>>>>>>>> rate) it does explain what I'm thinking more
>>>>>>>>>>>>>>> fully.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> THANK YOU ALL who responded to the previous
>>>>>>>>>>>>>>> thread,
>>>>>>>>>>>>>>> and please don't
>>>>>>>>>>>>>>> feel shy if you see something that I still
>>>>>>>>>>>>>>> haven't
>>>>>>>>>>>>>>> caught! I need to
>>>>>>>>>>>>>>> add an acknowledgements section for all the kind
>>>>>>>>>>>>>>> folks
>>>>>>>>>>>>>>> on USENET who
>>>>>>>>>>>>>>> critique my work.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Tim,
>>>>>>>>>>>>>> Thanks for making available a clear and
>>>>>>>>>>>>>> informative
>>>>>>>>>>>>>> paper. I would
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>> to make a comment at this late stage about the
>>>>>>>>>>>>>> section
>>>>>>>>>>>>>> on signal
>>>>>>>>>>>>>> reconstruction, mainly about terminology.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You explain that reconstruction is done by
>>>>>>>>>>>>>> interpolating the output
>>>>>>>>>>>>>> signal but unfortunately I feel that you have not
>>>>>>>>>>>>>> clearly identified
>>>>>>>>>>>>>> where the interpolaton occurs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You say that the first step in interpolation is
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> generation of a
>>>>>>>>>>>>>> stepped waveform by the zero-hold D/A and you
>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>> the stepped
>>>>>>>>>>>>>> waveform of Figure 5 as being a picture of an
>>>>>>>>>>>>>> interpolated signal. In
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>> view this should be changed to say that
>>>>>>>>>>>>>> generating the
>>>>>>>>>>>>>> stepped
>>>>>>>>>>>>>> waveform
>>>>>>>>>>>>>> is the first step in the reconstruction process,
>>>>>>>>>>>>>> interpolaton being
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> second step.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Interpolation, the generation of intermediate
>>>>>>>>>>>>>> values
>>>>>>>>>>>>>> between known
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>> points, occurs when the stepped analog signal
>>>>>>>>>>>>>> reaches
>>>>>>>>>>>>>> the analog
>>>>>>>>>>>>>> output
>>>>>>>>>>>>>> reconstruction filter. It is the analog components
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> generate the
>>>>>>>>>>>>>> intermediate data values, on a continuous
>>>>>>>>>>>>>> basis. You
>>>>>>>>>>>>>> have mentioned
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> output filter, and I think it would be useful to
>>>>>>>>>>>>>> identify this filter
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> the spot where interpolation occurs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I thought about that before I used the term, and I
>>>>>>>>>>>>> like
>>>>>>>>>>>>> my way. The
>>>>>>>>>>>>> reason I used it is because, in the context of that
>>>>>>>>>>>>> paper (and as I
>>>>>>>>>>>>> generally think about sampling), the sampled time
>>>>>>>>>>>>> signal
>>>>>>>>>>>>> _does not
>>>>>>>>>>>>> exist_ in between the sampling instants. That's a
>>>>>>>>>>>>> rather
>>>>>>>>>>>>> strict way to
>>>>>>>>>>>>> consider things, but I find it keeps me from
>>>>>>>>>>>>> miss-applying
>>>>>>>>>>>>> continuous-time wisdom to sampled time systems.
>>>>>>>>>>>>> Since
>>>>>>>>>>>>> the signal
>>>>>>>>>>>>> _doesn't_ exist, the action of a ZOH is, indeed, to
>>>>>>>>>>>>> interpolate.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The usage is different from what some people would
>>>>>>>>>>>>> use,
>>>>>>>>>>>>> but I (a) don't
>>>>>>>>>>>>> think it's all that far off, (b) think it's more
>>>>>>>>>>>>> accurate, and (c)
>>>>>>>>>>>>> think that if someone can't handle Author A using
>>>>>>>>>>>>> different terminology
>>>>>>>>>>>>> from Author B, then they can't handle engineering.
>>>>>>>>>>>>>
>>>>>>>>>>>> Tim,
>>>>>>>>>>>> Using a single sample to decide the values
>>>>>>>>>>>> existing at
>>>>>>>>>>>> later instants
>>>>>>>>>>>> (up to 1/fs later in this case) is extrapolation.
>>>>>>>>>>>> It is
>>>>>>>>>>>> interpolation
>>>>>>>>>>>> only when you use two or more samples to decide the
>>>>>>>>>>>> values existing
>>>>>>>>>>>> between samples. Regards,
>>>>>>>>>>>> John
>>>>>>>>>>>
>>>>>>>>>>> Ahhh. Right. Dangit.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Depends on where you put you time origin. If it's
>>>>>>>>>> in the
>>>>>>>>>> middle of the sample, a ZOH is an interpolator.
>>>>>>>>>>
>>>>>>>>>> Cheers
>>>>>>>>>>
>>>>>>>>>> Phil Hobbs
>>>>>>>>>
>>>>>>>>> Phil,
>>>>>>>>> Not he way I look at it, which is as follows:
>>>>>>>>>
>>>>>>>>> If a process uses known samples to determine signal
>>>>>>>>> values
>>>>>>>>> at times
>>>>>>>>> OUTSIDE the interval covered by the known samples then
>>>>>>>>> the
>>>>>>>>> process is
>>>>>>>>> extrapolation. The prefix 'extra,' meaning 'outside,'
>>>>>>>>> indicates this.
>>>>>>>>>
>>>>>>>>> If the process uses known samples to determine the
>>>>>>>>> signal
>>>>>>>>> values INSIDE
>>>>>>>>> the interval covered by the known values then the
>>>>>>>>> process is
>>>>>>>>> interpolation. Again, note the prefix.
>>>>>>>>>
>>>>>>>>> The Zero Hold circuit uses the last known sample to
>>>>>>>>> determine that the
>>>>>>>>> signal will have this sample value at any later time,
>>>>>>>>> potentially for
>>>>>>>>> all of eternity, but in practice up until the instant
>>>>>>>>> that
>>>>>>>>> the next
>>>>>>>>> sample arrives.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As far as I can see, none of this depends on my
>>>>>>>>> choice of
>>>>>>>>> a t=0 point.
>>>>>>>>> If we choose t=0 to be half-way between samples as you
>>>>>>>>> suggest then
>>>>>>>>> extrapolation starts at t = -T/2 and continues up to
>>>>>>>>> t =
>>>>>>>>> +T/2. (T being
>>>>>>>>> the sampling interval.)
>>>>>>>>> For all of this time the Zero Hold circuit is producing
>>>>>>>>> values which are
>>>>>>>>> outside the interval covered by the known values,
>>>>>>>>> and so
>>>>>>>>> extrapolation
>>>>>>>>> is taking pleace.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> John
>>>>>>>>
>>>>>>>> Interpolation vs extrapolation depends on whether you
>>>>>>>> are or
>>>>>>>> are not outside the range of your actual data points.
>>>>>>>> Estimating the data values before your first sample or
>>>>>>>> after
>>>>>>>> your last one is extrapolation, otherwise it's
>>>>>>>> interpolation. Given that the samples are taken after
>>>>>>>> the
>>>>>>>> antialiasing filter, the value of each sample depends to
>>>>>>>> some degree on the value of the input function at
>>>>>>>> 'future'
>>>>>>>> times--you can't very well make an antialiasing filter
>>>>>>>> without at least one sample's worth of group delay!
>>>>>>>>
>>>>>>>> Zero-order interpolation is all over the place, e.g. in
>>>>>>>> the
>>>>>>>> rectangle rule for integration, which is accurate up to
>>>>>>>> terms of second order as long as the rectangles are
>>>>>>>> centred
>>>>>>>> on the data points.
>>>>>>>>
>>>>>>>> It's not something I'm prepared to die defending,
>>>>>>>> however.
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>> Phil Hobbs
>>>>>>>>
>>>>>>> I wouldn't want to defend a claim that realisable
>>>>>>> anti-alias filters can
>>>>>>> be non-causal.
>>>>>>>
>>>>>>> Regards,
>>>>>>> John
>>>>>>
>>>>>> Neither would I. My claim was exactly the
>>>>>> opposite--that due
>>>>>> to the delay of the antialiasing filter, the signal for
>>>>>> sample N+1 must already exist by the time we're processing
>>>>>> sample N at the output. What's acausal about that?
>>>>>>
>>>>>> And most DSP applications aren't real-time anyway.
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> Phil Hobbs
>>>>>>
>>>>>
>>>>> On re-reading your post I see that you were referring to
>>>>> the input
>>>>> anti-alias filter. As the disussion was all about what
>>>>> occurs at the
>>>>> system output I assumed that you were referring to the
>>>>> output
>>>>> reconstruction filter, and to samples that had not yet
>>>>> arrived at this
>>>>> filter.
>>>>>
>>>>> I don't think the history of the system's input samples is
>>>>> relevant to
>>>>> the issue of whether the Zero-Hold circuit performs
>>>>> extrapolation or
>>>>> interpolation. By it's nature the Zero-Hold cicuit
>>>>> 'remembers' only the
>>>>> last sample presented to it and performs the simplest
>>>>> possible
>>>>> extrapolation of this value. As it uses only one input
>>>>> value it can not
>>>>> possibly be regarded as an interpolator.
>>>>>
>>>>> Regards,
>>>>> John
>>>>
>>>> Okay, we agree that sample N+1 already exists at time N,
>>>> then.
>>>>
>>>> You seem to be arguing that the fact that the value of
>>>> sample N+1 doesn't appear in the mathematical form of the
>>>> ZOH means that it's not an interpolator, but that isn't so.
>>>> An Nth order interpolator needs data from at least N+1
>>>> sample points to determine all its coefficients, and since
>>>> the ZOH is a zero-order interpolator, it only needs one data
>>>> point.
>>>>
>>>> The issue of interpolation vs eztrapolation is not the
>>>> functional form of the interpolator, but whether or not
>>>> you're doing it between samples (which we are, since we've
>>>> both now decided that sample N+1 already exists at time N).
>>>> This isn't semantics, it's a question of the reliability of
>>>> the procedure.
>>>>
>>>> Both interpolation and extrapolation introduce errors, and
>>>> by making assumptions e.g. about the boundedness of some
>>>> derivative of the function, you can put a bound on those
>>>> errors.(*) The bound is commonly expressed in terms of some
>>>> power of h (the time step) times the corresponding
>>>> derivative bound, times a constant. This limit grows without
>>>> bound as you get further and further from the last sample.
>>>>
>>>> The operational distinction between extrapolation and
>>>> interpolation is whether the errors are bounded, because you
>>>> get another sample at time N+1 (interpolation), or whether
>>>> they keep growing because you keep getting further from the
>>>> last data point (extrapolation). That's a really important
>>>> difference, one that goes a long way beyond sample
>>>> numbering.
>>>>
>>>> Cheers
>>>>
>>>> Phil Hobbs
>>>>
>>>> (*) In the special case of sinc interpolation of a genuinely
>>>> band-limited function, the error is zero, but band-limiting
>>>> requires infinite temporal extension, so we're never really
>>>> in the zero-error case.
>>>>
>>>
>>> Sample N+! may exist, but since the Zero Order Hold
>>> Circuit does not
>>> have access to it, its existence is irrelevant to any
>>> discussion of the
>>> operation of the ZOH circuit.
>>>
>>> Your argument is logically flawed because it assumes from
>>> the start that
>>> interpolation is possible with only one data point.
>>>
>>> Interestingly, if we substitute 'extrapolator' where you
>>> have used
>>> 'interpolator' in your argument, we get the following
>>> statement which,
>>> while it is still not valid as an argument, it is at least
>>> a correct
>>> statement of fact:
>>> "An Nth order EXTRApolator needs data from at least N+1
>>> sample points to determine all its coefficients, and since
>>> the ZOH is a
>>> zero-order EXTRApolator, it only needs one data point."
>>>
>>>
>>> Your overall argument, that the ZOK is an interpolator,
>>> relies on your
>>> establishing that extrapolation is a special case of
>>> interpolation. I
>>> have never seen this view either in textbooks or in
>>> general useage, and
>>> I am sure it is not widely held.
>>>
>>> Even if you think that extrapolation is a special case of
>>> interpolation
>>> I can not think of any good reason for you to query my use
>>> of the term
>>> 'extrapolation' in describing the action of the ZOH.
>>>
>>>
>>> Regards,
>>> John
>>>
>>>
>>
>> You haven't read what I've written, at least not very
>> carefully.
>>
>> Cheers
>>
>> Phil Hobbs
>>
> I did, very carefully, before coming to the conclusion that while
> interesting, none of it is relevant to the issue of whether a ZOH
> performs extrapolation.
>
> You might care to consider this: if the ZOH is NOT performing
> extrapolation, what would be the effect of putting an extrapolating
> circuit there instead?
>
> Regards,
> John
You're using a private vocabulary. The fact that it's being done
between adjacent samples makes it interpolation, not extrapolation. See
for example the following Google Books searches for "zero order
interpolation" and "zeroth order interpolation":
http://preview.tinyurl.com/236ectl
http://preview.tinyurl.com/24u4ezf
One or two of them use nearest neighbour averaging as the zero-order
interpolator, but most just use the nearest-neighbour value.
Cheers
Phil Hobbs
--
Dr Philip C D Hobbs
Principal
ElectroOptical Innovations
55 Orchard Rd
Briarcliff Manor NY 10510
845-480-2058
email: hobbs (atsign) electrooptical (period) net
http://electrooptical.net