IIR not fully causal?

Andor wrote:

>> [...]
>>True, but he unequivocally states that he has to do all this
>>withing two - 2 - samples worth of time. Is it possible to obtain
>>this in general, with a 2-tap FIR filter or a 2nd order IIR filter?
> 
> 
> I understand his requirement as meaning that the overall processing is
> allowed to have 2 samples (at the original sampling rate) of processing
> latency. 

YES!  That's exactly what I meant!  :-)

Importantly:  two samples at the original rate -- which means that
if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x
by low-pass digital filter interpolation), then the filter can have
a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an
example -- I'm still not 100% sure that for the particular application
I can afford two samples of lattency)

I'm glad that a couple of persons could read through my misleading
terminology -- I think I'm misled by the fact that I first learned
that this trick existed by looking at CD players :-)  Then, several
years later, I learned it as part of my DSP courses, so the bad
terminology stuck  :-)

I see that some of the answers address my original question (what
my original question was, in my mind), but there has been some
doubt about the terminology -- does this clarification change
anything about the answers so far?

BTW, I'm still reluctant to post exactly the application I need
this for -- go figure, if with a simplified version of the problem
I was able to cause such level of confusion, you don't want to
imagine what would happen if I attempt to explain what I need
this for!!!  :-)

Thanks,

Carlos
--

Carlos Moreno wrote:

> Andor wrote:
> 
>>> [...]
>>> True, but he unequivocally states that he has to do all this
>>> withing two - 2 - samples worth of time. Is it possible to obtain
>>> this in general, with a 2-tap FIR filter or a 2nd order IIR filter?
>>
>>
>>
>> I understand his requirement as meaning that the overall processing is
>> allowed to have 2 samples (at the original sampling rate) of processing
>> latency. 
> 
> 
> YES!  That's exactly what I meant!  :-)
> 
> Importantly:  two samples at the original rate -- which means that
> if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x
> by low-pass digital filter interpolation), then the filter can have
> a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an
> example -- I'm still not 100% sure that for the particular application
> I can afford two samples of lattency)

You _really_ threw us by not mentioning the different clock domains. 
You have a requirement for one or two tick delay in the original clock 
domain (call it A), but you can have a 32-tick delay in the oversampled 
domain (call it B)?  32 is way, way different from 2.
> 
> I'm glad that a couple of persons could read through my misleading
> terminology -- I think I'm misled by the fact that I first learned
> that this trick existed by looking at CD players :-)  Then, several
> years later, I learned it as part of my DSP courses, so the bad
> terminology stuck  :-)
> 
> I see that some of the answers address my original question (what
> my original question was, in my mind), but there has been some
> doubt about the terminology -- does this clarification change
> anything about the answers so far?
> 
> BTW, I'm still reluctant to post exactly the application I need
> this for -- go figure, if with a simplified version of the problem
> I was able to cause such level of confusion, you don't want to
> imagine what would happen if I attempt to explain what I need
> this for!!!  :-)
> 
Try to reduce it down to a technical description of the one bitty little 
problem you're trying to solve.  We don't (well, we shouldn't) care if 
you're making a plorbnitz-enhanced frisbee flanger, but we _do_ care 
about how much processing power you have available, what kind of delay 
you have, how many samples in and how many samples out you can support, etc.

Not only will you help us with such a technical reduction, when you're 
done spelling it out you may find the answer jumping out saying hello.

> Thanks,
> 
> Carlos
> -- 


-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Carlos Moreno wrote:
> Andor wrote:
>
> >> [...]
> >>True, but he unequivocally states that he has to do all this
> >>withing two - 2 - samples worth of time. Is it possible to obtain
> >>this in general, with a 2-tap FIR filter or a 2nd order IIR filter?
> >
> >
> > I understand his requirement as meaning that the overall processing
is
> > allowed to have 2 samples (at the original sampling rate) of
processing
> > latency.
>
> YES!  That's exactly what I meant!  :-)
>
> Importantly:  two samples at the original rate -- which means that
> if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x
> by low-pass digital filter interpolation), then the filter can have
> a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an
> example -- I'm still not 100% sure that for the particular
application
> I can afford two samples of lattency)

Carlos,

I think you are in trouble. Let's consider the case where you want to
upsample by a factor of 16. If the original signal was full bandwidth
(ie. wasn't already oversampled), the new signal consists of 16
alternately mirrored replicas of the spectrum of the old signal. To
filter out these replicas, you need to apply a filter with cutoff
frequency at 1/32 f_s (where f_s is 16 times the old sampling rate).

As there exists no real filter with infinitely short bandwidth, you
have two choices:

1. Accept that you reduce the bandwidth of the original signal
somewhat. In that case, you need a lowpass filter with cutoff frequency
at (1/32 - epsilon) f_s, and a stopand frequency of 1/32 f_s. epsilon
is the bandwidth.

2. Reduce the bandwidth of the original signal a little less at the
cost of additionial aliasing. Design a lowpass filter with cutoff
frequency (1/32 - epsilon/2) and stopband frequency of (1/32 +
epsilon/2). If you design this with equiripple criterion (or with
windowing), every 16th coefficient of the FIR is zero, which saves you
a bit of processing.

You can, of course, trade off bandwidth reduction vs. aliasing in a
continuous manner.

When you have the filter specificiations, plug them into a filter
design program to see how much stopband attenuation a linear-phase FIR
of order 64 can buy you at the specified bandwidth (here you trade-off
bandwidth with stopband attenuation). You will see that this isn't very
much. If it is enough depends on you.

You might want to experiment with an additional IIR cascaded after the
FIR filter (also running at 16 times the original sampling rate). Such
a filter might help you with tighter bandwidth / more stopband
attenuation at the cost of linear phase.

Regards,
Andor

"Carlos Moreno" <moreno_at_mochima_dot_com@xx.xxx> wrote in message 
news:4qIfe.40131$2g1.332520@weber.videotron.net...
> Andor wrote:
>
>>> [...]
>>>True, but he unequivocally states that he has to do all this
>>>withing two - 2 - samples worth of time. Is it possible to obtain
>>>this in general, with a 2-tap FIR filter or a 2nd order IIR filter?
>>
>>
>> I understand his requirement as meaning that the overall processing is
>> allowed to have 2 samples (at the original sampling rate) of processing
>> latency.
>
> YES!  That's exactly what I meant!  :-)
>
> Importantly:  two samples at the original rate -- which means that
> if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x
> by low-pass digital filter interpolation), then the filter can have
> a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an
> example -- I'm still not 100% sure that for the particular application
> I can afford two samples of lattency)
>

Carlos,

I think the term you're looking for is "upsampling" rather than 
"oversampling".  That's the common term for what it seems you're doing.  If 
I understand the problem:

1) a sampled data stream at fs0.
2) an upsampler at 16fs0 = fs1
The results of upsampling are 15 zeros interspersed between each of the 
original samples.

Andor describes the resulting spectrum - 16 repetitions of the original 
spectrum over fs1.
You need to filter (acceptably, not perfectly) down to fs0/32.
You need to provide an output at the end of two sample periods (or is it 
one?) 1/fs0.

If we think outside the box a bit I wonder what would happen if you had 
infinite compute resources (just for thinking purposes).  Then one could 
envision a process that looks like this:

When the second sample interval 1/fs0 is complete, there are three samples 
x0, x1 and x2.
Instantaneously (using infinite resources) compute interpolated values 
between them.  I believe the best you can do is linear interpolation so a 
filter:
               0
   0.06666666666667
   0.13333333333333
   0.20000000000000
   0.26666666666600
   0.33333333333333
   0.40000000000000
   0.46666666666667
   0.53333333333333
   0.60000000000000
   0.66666666666667
   0.73333333333333
   0.80000000000000
   0.86666666666667
   0.93333333333333
   1.00000000000000
   0.93333333333333
   0.86666666666667
   0.80000000000000
   0.73333333333333
   0.66666666666667
   0.60000000000000
   0.53333333333333
   0.46666666666667
   0.40000000000000
   0.33333333333333
   0.26666666666600
   0.20000000000000
   0.13333333333333
   0.06666666666667
                  0
...will do the linear interpolation and yield a rather decent looking 
spectrum - if the spectrum it yields is acceptable.  And, it doesn't require 
3 samples, only 2.  I believe that linear interpolation is the best you can 
do if all you have are 3 samples - because you don't know how to deal with 
the derivatives at the bracketing samples any better than that.  Maybe I'm 
wrong on this and there is a better filter length 31 for the situation you 
have - I just can't think of one right now and don't know better otherwise.

There is no reason in the world (except your compute resuources) to hold the 
rate of computation of the outputs of the linear interpolation to the output 
sample rate.  You can calculate the first 17 output  samples when the second 
input sample occurs and you can calculate the next 16 output samples when 
the next input sample occurs ad infinitum.  This puts you behind 1 input 
sample.

Fred 

Fred Marshall wrote:

> If we think outside the box a bit ...

http://www.cartoonbank.com/product_details.asp?mscssid=0EAGDNUXMNX48K4MAEJ02SJNG39GCSTA&sitetype=1&did=4&sid=40533&whichpage=1&sortBy=popular&keyword=box&section=cartoons

or, if that breaks:

http://makeashorterlink.com/?C41A12B0B

:-)

Ciao,

Peter K.

Tim Wescott wrote:

>>> I understand his requirement as meaning that the overall processing is
>>> allowed to have 2 samples (at the original sampling rate) of processing
>>> latency. 
>>
>> YES!  That's exactly what I meant!  :-)
>>
>> Importantly:  two samples at the original rate -- which means that
>> if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x
>> by low-pass digital filter interpolation), then the filter can have
>> a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an
>> example -- I'm still not 100% sure that for the particular application
>> I can afford two samples of lattency)
> 
> You _really_ threw us by not mentioning the different clock domains. 

Does "oops" get me off the hook this time?  (I know, I have to promise
that I won't make a habit out of it!  :-))

This is the kind of thing that one would very easily explain with a
graphic -- if I were asking you guys in person, with a board at hand,
it surely would have been much easier...  Oh well, I'm sure we all
had fun "debugging" my specs (or, from my point of view, "debugging"
your replies  :-))

>> BTW, I'm still reluctant to post exactly the application I need
>> this for -- go figure, if with a simplified version of the problem
>> I was able to cause such level of confusion, you don't want to
>> imagine what would happen if I attempt to explain what I need
>> this for!!!  :-)
>>
> Try to reduce it down to a technical description of the one bitty little 
> problem you're trying to solve.  We don't (well, we shouldn't) care if 
> you're making a plorbnitz-enhanced frisbee flanger, but we _do_ care 
> about how much processing power you have available, what kind of delay 
> you have, how many samples in and how many samples out you can support, 
> etc.

Roughly speaking, I need to interpolate parameters obtained for
individual frames of speech.

Say that a frame of speech is 256 samples;  I'm getting a set of
parameters in an LPC-like encoding, and then, for the decoding,
I'm using linear interpolation.

I am investigating whether or not the quality of the reconstruction
may be increased by doing an interpolation better than linear; in
fact, better than 3rd-order polynomial interpolation (which is what
I could do with the same amount of information -- corresponding to
the same amount of processing lattency).

The obvious idea that comes to mind as a better interpolation than
3rd-order polynomial is a "digital low-pass filter" interpolation
(basically, the process that I described:  upsample by placing
one sample of the signal, then N-1 zeros, then apply a digital
filter as close to perfect reconstruction as possible).

Now, you probably saw the tricky part:  the "original" sampling
rate is the frame rate! (in the order of 10ms, or 100Hz).  Now,
if I use a linear phase 32-taps FIR filter (the equivalent of
32-tap in the original sampling rate domain), then I'm in big
trouble, since my processing lattency would be 16 *frames*.
(that's assuming a 32-tap filter would do -- I'm not even sure
if 32-tap is sufficient)

With LPC-like encoding, I definitely can not afford to have to
wait for 16 or 32 frames (a non-negligible fraction of a second).

That's why an IIR came to mind -- but then, I thought that I would
need to allow for some processing lattency.  The idea being that
if I'm computing now the interpolated (missing) values between
samples x[n] and x[n+1] (in the original sampling domain) and I
do know the value of x[n+1], then I can do a much better job than
a "lattencyless" IIR, in which by the time that I'm computing the
interpolated values between x[n] and x[n+1] I still haven't seen
the value of x[n+1], and have to rely only on x[n], x[n-1], x[n-2],
etc.

(that's why I originally phrased it as being equivalent to seeing
one sample ahead)

One problem that comes to mind is that even if I indeed find a
way to use an IIR-like structure with the ambitious characteristics
I have in mind, the impulse response will be sooooooo long that
I might end up having serious accuracy/stability problems.  Not
having much practical experience with this, I'm not sure if my
guesstimate is off)


I'm hoping that the discussion will continue, even though I'm
quite happy and quite grateful for the replies so far!

Perhaps with a closer description of where I'm trying to apply
this, you'll be able to determine if what I'm thinking has any
merit, or alternative/better approaches.

Thanks!

Carlos
--

Andor wrote:
> ... as I understand, the OP wants to design a lowpass filter to
> remove spectral images created during an integer upsampling process
> (which he calls "oversampling", a term which seems to throw everybody
> off track here for some reason). 

   ...

Allow me to supply the reason (for me, anyway). Sampling is the process 
that digitizes analog signals. The minimum sample rate that will allow 
the original to be reproduced is the standard rate. (By "standard", I 
mean not "usual", but that rate against which other rates are measured.) 
It is necessary to sample at a frequency somewhat higher than twice the 
highest frequency in the signal. If the chosen sampling frequency is 
half again the minimum, we say that the signal is oversampled by 1.5 
times. If, as may be necessary in closed-loop servos, the sampling is 
ten times the highest signal frequency, that system is oversampled by 
5x. We use "upsampling" to mean the construction of a signal stream that 
has more frequent specified values than the original. consider the 
difference in meaning between "upgraded" and "overgraded". 'Nuff said?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Jerry Avins wrote:

(snip)

> If the chosen sampling frequency is 
> half again the minimum, we say that the signal is oversampled by 1.5 
> times. If, as may be necessary in closed-loop servos, the sampling is 
> ten times the highest signal frequency, that system is oversampled by 
> 5x. We use "upsampling" to mean the construction of a signal stream that 
> has more frequent specified values than the original. consider the 
> difference in meaning between "upgraded" and "overgraded". 'Nuff said?

It might be the first time I heard the term oversampling was in the
design of CD players using digital filters.  In the early days when
16 bit DACs were expensive there was a system to use 14bit DACs at
4x the sampling rate.  What should have been, as you say, upsampling
was called oversampling, maybe by the marketing department.

I don't know if the usage has stuck or not.  I do agree that upsampling
makes more sense, but popular usage doesn't always make sense.

-- glen

The OP did say that he saw this trick for the first time in a CD
player. That's where he got his vocabulary from.

The worst definnition of "oversampling" I have seen is in the
webopedia:

http://www.webopedia.com/TERM/o/oversampling.html

Talk about popular usage that doesn't make sense ... :(

Regards,
Andor

glen herrmannsfeldt wrote:
> Jerry Avins wrote:
> 
> (snip)
> 
>> If the chosen sampling frequency is half again the minimum, we say 
>> that the signal is oversampled by 1.5 times. If, as may be necessary 
>> in closed-loop servos, the sampling is ten times the highest signal 
>> frequency, that system is oversampled by 5x. We use "upsampling" to 
>> mean the construction of a signal stream that has more frequent 
>> specified values than the original. consider the difference in meaning 
>> between "upgraded" and "overgraded". 'Nuff said?
> 
> 
> It might be the first time I heard the term oversampling was in the
> design of CD players using digital filters.  In the early days when
> 16 bit DACs were expensive there was a system to use 14bit DACs at
> 4x the sampling rate.  What should have been, as you say, upsampling
> was called oversampling, maybe by the marketing department.
> 
> I don't know if the usage has stuck or not.  I do agree that upsampling
> makes more sense, but popular usage doesn't always make sense.

Glen,

What you describe was true oversampling. (But not enough oversampling. 
To gain 4 bits -- a factor of 4, you need to average 16 samples.) 
Oversampling is sampling faster than 2Fmax. Upsampling may achieve the 
same end result, but it gets there by a different route.

Jerry

-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������