Forums

IIR not fully causal?

Started by Carlos Moreno May 7, 2005
Andor wrote:

>> [...] >>True, but he unequivocally states that he has to do all this >>withing two - 2 - samples worth of time. Is it possible to obtain >>this in general, with a 2-tap FIR filter or a 2nd order IIR filter? > > > I understand his requirement as meaning that the overall processing is > allowed to have 2 samples (at the original sampling rate) of processing > latency.
YES! That's exactly what I meant! :-) Importantly: two samples at the original rate -- which means that if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x by low-pass digital filter interpolation), then the filter can have a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an example -- I'm still not 100% sure that for the particular application I can afford two samples of lattency) I'm glad that a couple of persons could read through my misleading terminology -- I think I'm misled by the fact that I first learned that this trick existed by looking at CD players :-) Then, several years later, I learned it as part of my DSP courses, so the bad terminology stuck :-) I see that some of the answers address my original question (what my original question was, in my mind), but there has been some doubt about the terminology -- does this clarification change anything about the answers so far? BTW, I'm still reluctant to post exactly the application I need this for -- go figure, if with a simplified version of the problem I was able to cause such level of confusion, you don't want to imagine what would happen if I attempt to explain what I need this for!!! :-) Thanks, Carlos --
Carlos Moreno wrote:

> Andor wrote: > >>> [...] >>> True, but he unequivocally states that he has to do all this >>> withing two - 2 - samples worth of time. Is it possible to obtain >>> this in general, with a 2-tap FIR filter or a 2nd order IIR filter? >> >> >> >> I understand his requirement as meaning that the overall processing is >> allowed to have 2 samples (at the original sampling rate) of processing >> latency. > > > YES! That's exactly what I meant! :-) > > Importantly: two samples at the original rate -- which means that > if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x > by low-pass digital filter interpolation), then the filter can have > a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an > example -- I'm still not 100% sure that for the particular application > I can afford two samples of lattency)
You _really_ threw us by not mentioning the different clock domains. You have a requirement for one or two tick delay in the original clock domain (call it A), but you can have a 32-tick delay in the oversampled domain (call it B)? 32 is way, way different from 2.
> > I'm glad that a couple of persons could read through my misleading > terminology -- I think I'm misled by the fact that I first learned > that this trick existed by looking at CD players :-) Then, several > years later, I learned it as part of my DSP courses, so the bad > terminology stuck :-) > > I see that some of the answers address my original question (what > my original question was, in my mind), but there has been some > doubt about the terminology -- does this clarification change > anything about the answers so far? > > BTW, I'm still reluctant to post exactly the application I need > this for -- go figure, if with a simplified version of the problem > I was able to cause such level of confusion, you don't want to > imagine what would happen if I attempt to explain what I need > this for!!! :-) >
Try to reduce it down to a technical description of the one bitty little problem you're trying to solve. We don't (well, we shouldn't) care if you're making a plorbnitz-enhanced frisbee flanger, but we _do_ care about how much processing power you have available, what kind of delay you have, how many samples in and how many samples out you can support, etc. Not only will you help us with such a technical reduction, when you're done spelling it out you may find the answer jumping out saying hello.
> Thanks, > > Carlos > --
-- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Carlos Moreno wrote:
> Andor wrote: > > >> [...] > >>True, but he unequivocally states that he has to do all this > >>withing two - 2 - samples worth of time. Is it possible to obtain > >>this in general, with a 2-tap FIR filter or a 2nd order IIR filter? > > > > > > I understand his requirement as meaning that the overall processing
is
> > allowed to have 2 samples (at the original sampling rate) of
processing
> > latency. > > YES! That's exactly what I meant! :-) > > Importantly: two samples at the original rate -- which means that > if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x > by low-pass digital filter interpolation), then the filter can have > a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an > example -- I'm still not 100% sure that for the particular
application
> I can afford two samples of lattency)
Carlos, I think you are in trouble. Let's consider the case where you want to upsample by a factor of 16. If the original signal was full bandwidth (ie. wasn't already oversampled), the new signal consists of 16 alternately mirrored replicas of the spectrum of the old signal. To filter out these replicas, you need to apply a filter with cutoff frequency at 1/32 f_s (where f_s is 16 times the old sampling rate). As there exists no real filter with infinitely short bandwidth, you have two choices: 1. Accept that you reduce the bandwidth of the original signal somewhat. In that case, you need a lowpass filter with cutoff frequency at (1/32 - epsilon) f_s, and a stopand frequency of 1/32 f_s. epsilon is the bandwidth. 2. Reduce the bandwidth of the original signal a little less at the cost of additionial aliasing. Design a lowpass filter with cutoff frequency (1/32 - epsilon/2) and stopband frequency of (1/32 + epsilon/2). If you design this with equiripple criterion (or with windowing), every 16th coefficient of the FIR is zero, which saves you a bit of processing. You can, of course, trade off bandwidth reduction vs. aliasing in a continuous manner. When you have the filter specificiations, plug them into a filter design program to see how much stopband attenuation a linear-phase FIR of order 64 can buy you at the specified bandwidth (here you trade-off bandwidth with stopband attenuation). You will see that this isn't very much. If it is enough depends on you. You might want to experiment with an additional IIR cascaded after the FIR filter (also running at 16 times the original sampling rate). Such a filter might help you with tighter bandwidth / more stopband attenuation at the cost of linear phase. Regards, Andor
"Carlos Moreno" <moreno_at_mochima_dot_com@xx.xxx> wrote in message 
news:4qIfe.40131$2g1.332520@weber.videotron.net...
> Andor wrote: > >>> [...] >>>True, but he unequivocally states that he has to do all this >>>withing two - 2 - samples worth of time. Is it possible to obtain >>>this in general, with a 2-tap FIR filter or a 2nd order IIR filter? >> >> >> I understand his requirement as meaning that the overall processing is >> allowed to have 2 samples (at the original sampling rate) of processing >> latency. > > YES! That's exactly what I meant! :-) > > Importantly: two samples at the original rate -- which means that > if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x > by low-pass digital filter interpolation), then the filter can have > a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an > example -- I'm still not 100% sure that for the particular application > I can afford two samples of lattency) >
Carlos, I think the term you're looking for is "upsampling" rather than "oversampling". That's the common term for what it seems you're doing. If I understand the problem: 1) a sampled data stream at fs0. 2) an upsampler at 16fs0 = fs1 The results of upsampling are 15 zeros interspersed between each of the original samples. Andor describes the resulting spectrum - 16 repetitions of the original spectrum over fs1. You need to filter (acceptably, not perfectly) down to fs0/32. You need to provide an output at the end of two sample periods (or is it one?) 1/fs0. If we think outside the box a bit I wonder what would happen if you had infinite compute resources (just for thinking purposes). Then one could envision a process that looks like this: When the second sample interval 1/fs0 is complete, there are three samples x0, x1 and x2. Instantaneously (using infinite resources) compute interpolated values between them. I believe the best you can do is linear interpolation so a filter: 0 0.06666666666667 0.13333333333333 0.20000000000000 0.26666666666600 0.33333333333333 0.40000000000000 0.46666666666667 0.53333333333333 0.60000000000000 0.66666666666667 0.73333333333333 0.80000000000000 0.86666666666667 0.93333333333333 1.00000000000000 0.93333333333333 0.86666666666667 0.80000000000000 0.73333333333333 0.66666666666667 0.60000000000000 0.53333333333333 0.46666666666667 0.40000000000000 0.33333333333333 0.26666666666600 0.20000000000000 0.13333333333333 0.06666666666667 0 ...will do the linear interpolation and yield a rather decent looking spectrum - if the spectrum it yields is acceptable. And, it doesn't require 3 samples, only 2. I believe that linear interpolation is the best you can do if all you have are 3 samples - because you don't know how to deal with the derivatives at the bracketing samples any better than that. Maybe I'm wrong on this and there is a better filter length 31 for the situation you have - I just can't think of one right now and don't know better otherwise. There is no reason in the world (except your compute resuources) to hold the rate of computation of the outputs of the linear interpolation to the output sample rate. You can calculate the first 17 output samples when the second input sample occurs and you can calculate the next 16 output samples when the next input sample occurs ad infinitum. This puts you behind 1 input sample. Fred
Fred Marshall wrote:

> If we think outside the box a bit ...
http://www.cartoonbank.com/product_details.asp?mscssid=0EAGDNUXMNX48K4MAEJ02SJNG39GCSTA&sitetype=1&did=4&sid=40533&whichpage=1&sortBy=popular&keyword=box&section=cartoons or, if that breaks: http://makeashorterlink.com/?C41A12B0B :-) Ciao, Peter K.
Tim Wescott wrote:

>>> I understand his requirement as meaning that the overall processing is >>> allowed to have 2 samples (at the original sampling rate) of processing >>> latency. >> >> YES! That's exactly what I meant! :-) >> >> Importantly: two samples at the original rate -- which means that >> if I'm doing 16x interpolation (i.e., upsampling at a rate of 16x >> by low-pass digital filter interpolation), then the filter can have >> a 32 samples "delay" (well, or 16 -- I said 1 or 2 samples as an >> example -- I'm still not 100% sure that for the particular application >> I can afford two samples of lattency) > > You _really_ threw us by not mentioning the different clock domains.
Does "oops" get me off the hook this time? (I know, I have to promise that I won't make a habit out of it! :-)) This is the kind of thing that one would very easily explain with a graphic -- if I were asking you guys in person, with a board at hand, it surely would have been much easier... Oh well, I'm sure we all had fun "debugging" my specs (or, from my point of view, "debugging" your replies :-))
>> BTW, I'm still reluctant to post exactly the application I need >> this for -- go figure, if with a simplified version of the problem >> I was able to cause such level of confusion, you don't want to >> imagine what would happen if I attempt to explain what I need >> this for!!! :-) >> > Try to reduce it down to a technical description of the one bitty little > problem you're trying to solve. We don't (well, we shouldn't) care if > you're making a plorbnitz-enhanced frisbee flanger, but we _do_ care > about how much processing power you have available, what kind of delay > you have, how many samples in and how many samples out you can support, > etc.
Roughly speaking, I need to interpolate parameters obtained for individual frames of speech. Say that a frame of speech is 256 samples; I'm getting a set of parameters in an LPC-like encoding, and then, for the decoding, I'm using linear interpolation. I am investigating whether or not the quality of the reconstruction may be increased by doing an interpolation better than linear; in fact, better than 3rd-order polynomial interpolation (which is what I could do with the same amount of information -- corresponding to the same amount of processing lattency). The obvious idea that comes to mind as a better interpolation than 3rd-order polynomial is a "digital low-pass filter" interpolation (basically, the process that I described: upsample by placing one sample of the signal, then N-1 zeros, then apply a digital filter as close to perfect reconstruction as possible). Now, you probably saw the tricky part: the "original" sampling rate is the frame rate! (in the order of 10ms, or 100Hz). Now, if I use a linear phase 32-taps FIR filter (the equivalent of 32-tap in the original sampling rate domain), then I'm in big trouble, since my processing lattency would be 16 *frames*. (that's assuming a 32-tap filter would do -- I'm not even sure if 32-tap is sufficient) With LPC-like encoding, I definitely can not afford to have to wait for 16 or 32 frames (a non-negligible fraction of a second). That's why an IIR came to mind -- but then, I thought that I would need to allow for some processing lattency. The idea being that if I'm computing now the interpolated (missing) values between samples x[n] and x[n+1] (in the original sampling domain) and I do know the value of x[n+1], then I can do a much better job than a "lattencyless" IIR, in which by the time that I'm computing the interpolated values between x[n] and x[n+1] I still haven't seen the value of x[n+1], and have to rely only on x[n], x[n-1], x[n-2], etc. (that's why I originally phrased it as being equivalent to seeing one sample ahead) One problem that comes to mind is that even if I indeed find a way to use an IIR-like structure with the ambitious characteristics I have in mind, the impulse response will be sooooooo long that I might end up having serious accuracy/stability problems. Not having much practical experience with this, I'm not sure if my guesstimate is off) I'm hoping that the discussion will continue, even though I'm quite happy and quite grateful for the replies so far! Perhaps with a closer description of where I'm trying to apply this, you'll be able to determine if what I'm thinking has any merit, or alternative/better approaches. Thanks! Carlos --
Andor wrote:
> ... as I understand, the OP wants to design a lowpass filter to > remove spectral images created during an integer upsampling process > (which he calls "oversampling", a term which seems to throw everybody > off track here for some reason).
... Allow me to supply the reason (for me, anyway). Sampling is the process that digitizes analog signals. The minimum sample rate that will allow the original to be reproduced is the standard rate. (By "standard", I mean not "usual", but that rate against which other rates are measured.) It is necessary to sample at a frequency somewhat higher than twice the highest frequency in the signal. If the chosen sampling frequency is half again the minimum, we say that the signal is oversampled by 1.5 times. If, as may be necessary in closed-loop servos, the sampling is ten times the highest signal frequency, that system is oversampled by 5x. We use "upsampling" to mean the construction of a signal stream that has more frequent specified values than the original. consider the difference in meaning between "upgraded" and "overgraded". 'Nuff said? Jerry -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;
Jerry Avins wrote:

(snip)

> If the chosen sampling frequency is > half again the minimum, we say that the signal is oversampled by 1.5 > times. If, as may be necessary in closed-loop servos, the sampling is > ten times the highest signal frequency, that system is oversampled by > 5x. We use "upsampling" to mean the construction of a signal stream that > has more frequent specified values than the original. consider the > difference in meaning between "upgraded" and "overgraded". 'Nuff said?
It might be the first time I heard the term oversampling was in the design of CD players using digital filters. In the early days when 16 bit DACs were expensive there was a system to use 14bit DACs at 4x the sampling rate. What should have been, as you say, upsampling was called oversampling, maybe by the marketing department. I don't know if the usage has stuck or not. I do agree that upsampling makes more sense, but popular usage doesn't always make sense. -- glen
The OP did say that he saw this trick for the first time in a CD
player. That's where he got his vocabulary from.

The worst definnition of "oversampling" I have seen is in the
webopedia:

http://www.webopedia.com/TERM/o/oversampling.html

Talk about popular usage that doesn't make sense ... :(

Regards,
Andor

glen herrmannsfeldt wrote:
> Jerry Avins wrote: > > (snip) > >> If the chosen sampling frequency is half again the minimum, we say >> that the signal is oversampled by 1.5 times. If, as may be necessary >> in closed-loop servos, the sampling is ten times the highest signal >> frequency, that system is oversampled by 5x. We use "upsampling" to >> mean the construction of a signal stream that has more frequent >> specified values than the original. consider the difference in meaning >> between "upgraded" and "overgraded". 'Nuff said? > > > It might be the first time I heard the term oversampling was in the > design of CD players using digital filters. In the early days when > 16 bit DACs were expensive there was a system to use 14bit DACs at > 4x the sampling rate. What should have been, as you say, upsampling > was called oversampling, maybe by the marketing department. > > I don't know if the usage has stuck or not. I do agree that upsampling > makes more sense, but popular usage doesn't always make sense.
Glen, What you describe was true oversampling. (But not enough oversampling. To gain 4 bits -- a factor of 4, you need to average 16 samples.) Oversampling is sampling faster than 2Fmax. Upsampling may achieve the same end result, but it gets there by a different route. Jerry -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;