IIR not fully causal?

Started by Carlos Moreno May 7, 2005
I've been thinking about this crazy idea that I assume has
to be something that someone did already.

Say that I'm trying to do oversampling;  let's take, as an
example, oversampling by a factor of 4.  So, what we do is,
just put a sample of the signal, then three samples with
zero, then apply a low-pass filter (with cutoff pi/4).

This low-pass filter should, in principle, be a 0-phase FIR
(in reality, it is a linear-phase FIR, and we determine the
oversampled signal with some delay).

Now, the above technique is not acceptable for real-time
applications with critical/tough delay requirements (i.e.,
if we can not afford the delay introduced by the length
required for the FIR, then the above is not an option).

Let's say that we have a situation where non-linear phase
is also out of the question (i.e., we must reconstruct the
signal such that it is very close to the "true" signal *in
the time domain* -- i.e., quasi-zero-phase is a requirement).

An IIR would have a very hard time with that task, right?

I'm thinking, what if we do an IIR in which the output of
the filter is the previous sample (or N samples before, in
general).

The idea is that, intuitively, a casual IIR has a hard time
reconstructing with quasi-zero-phase because it can not know
where the signal is going.  But then, if we allow it to
know one sample in the future (or, in general, N samples in
the future, where N is much smaller than the N we would need
for a linear phase FIR), it could do a much better job.

Is this a standard technique?  Are there existing algorithms
to design IIR filters as close as zero-phase as possible
given these conditions?  Or is it completely impossible with
this approach?  (i.e., would I have to do a regular IIR and
then use a non-linear estimator?  Or simply use a non-linear
estimator with the input signal alone?)

I'll be grateful for any ideas/pointers.

Thanks,

Carlos
--
Carlos Moreno wrote:
> > I've been thinking about this crazy idea that I assume has > to be something that someone did already. > > Say that I'm trying to do oversampling; let's take, as an > example, oversampling by a factor of 4. So, what we do is, > just put a sample of the signal, then three samples with > zero, then apply a low-pass filter (with cutoff pi/4). > > This low-pass filter should, in principle, be a 0-phase FIR > (in reality, it is a linear-phase FIR, and we determine the > oversampled signal with some delay). > > Now, the above technique is not acceptable for real-time > applications with critical/tough delay requirements (i.e., > if we can not afford the delay introduced by the length > required for the FIR, then the above is not an option). > > Let's say that we have a situation where non-linear phase > is also out of the question (i.e., we must reconstruct the > signal such that it is very close to the "true" signal *in > the time domain* -- i.e., quasi-zero-phase is a requirement). > > An IIR would have a very hard time with that task, right? > > I'm thinking, what if we do an IIR in which the output of > the filter is the previous sample (or N samples before, in > general). > > The idea is that, intuitively, a casual IIR has a hard time > reconstructing with quasi-zero-phase because it can not know > where the signal is going. But then, if we allow it to > know one sample in the future (or, in general, N samples in > the future, where N is much smaller than the N we would need > for a linear phase FIR), it could do a much better job. > > Is this a standard technique? Are there existing algorithms > to design IIR filters as close as zero-phase as possible > given these conditions? Or is it completely impossible with > this approach? (i.e., would I have to do a regular IIR and > then use a non-linear estimator? Or simply use a non-linear > estimator with the input signal alone?) > > I'll be grateful for any ideas/pointers. >
If I could predict the future I'd have better things to do than building IIR filters -- like trading on the stock market. Yes, you should be able to make a dynamite zero-phase-shift IIR filter by the simple expedient of taking a filter prototype H(z) and mirroring all of the poles and zeros around the unit circle, i.e. H_nc(z) = H(z) * H(1/z). But then your filter is either non-causal or unstable, neither of which is a good thing. Also, since the impulse response is not only infinite, but is now infinite in the _future_, you can't just add delay and make a linear phase filter out of it. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Tim Wescott wrote:

>> [...] >> >> The idea is that, intuitively, a casual IIR has a hard time >> reconstructing with quasi-zero-phase because it can not know >> where the signal is going. But then, if we allow it to >> know one sample in the future (or, in general, N samples in >> the future, where N is much smaller than the N we would need >> for a linear phase FIR), it could do a much better job. >> >> Is this a standard technique? Are there existing algorithms >> to design IIR filters as close as zero-phase as possible >> given these conditions? Or is it completely impossible with >> this approach? (i.e., would I have to do a regular IIR and >> then use a non-linear estimator? Or simply use a non-linear >> estimator with the input signal alone?) > > If I could predict the future I'd have better things to do than building > IIR filters -- like trading on the stock market. > > Yes, you should be able to make a dynamite zero-phase-shift IIR filter > by the simple expedient of taking a filter prototype H(z) and mirroring > all of the poles and zeros around the unit circle, i.e. H_nc(z) = H(z) * > H(1/z). But then your filter is either non-causal or unstable, neither > of which is a good thing. Also, since the impulse response is not only > infinite, but is now infinite in the _future_, you can't just add delay > and make a linear phase filter out of it.
Wait... I think you didn't understand what I was trying to say. An FIR is linear phase, right? It is perfectly causal (as any real system), and it can be seen as the output of a zero-phase filter, delayed by N/2 samples. Now, this "fictitious" zero-phase filter would be non-causal; if would require access to N/2 samples from the future of the input signal. Of course, that is possible because what we're doing is that the output of that fictitious zero-phase filter is being computed N/2 samples later -- and thus, N/2 samples later we can see what for that fictitious filter was the future N/2 samples. What I'm trying to get at is: what if we do an IIR where the output of a quasi-zero-phase filter is computed one sample later -- thus, doing the equivalent of having access to one sample from the future (then, of course, that quasi-zero-phase output is now being delayed by one sample). I hope I'm explaining better this time. Thanks, Carlos --
Carlos Moreno wrote:

> Tim Wescott wrote: > >>> [...] >>> >>> The idea is that, intuitively, a casual IIR has a hard time >>> reconstructing with quasi-zero-phase because it can not know >>> where the signal is going. But then, if we allow it to >>> know one sample in the future (or, in general, N samples in >>> the future, where N is much smaller than the N we would need >>> for a linear phase FIR), it could do a much better job. >>> >>> Is this a standard technique? Are there existing algorithms >>> to design IIR filters as close as zero-phase as possible >>> given these conditions? Or is it completely impossible with >>> this approach? (i.e., would I have to do a regular IIR and >>> then use a non-linear estimator? Or simply use a non-linear >>> estimator with the input signal alone?) >> >> >> If I could predict the future I'd have better things to do than >> building IIR filters -- like trading on the stock market. >> >> Yes, you should be able to make a dynamite zero-phase-shift IIR filter >> by the simple expedient of taking a filter prototype H(z) and >> mirroring all of the poles and zeros around the unit circle, i.e. >> H_nc(z) = H(z) * H(1/z). But then your filter is either non-causal or >> unstable, neither of which is a good thing. Also, since the impulse >> response is not only infinite, but is now infinite in the _future_, >> you can't just add delay and make a linear phase filter out of it. > > > Wait... I think you didn't understand what I was trying to say. > > An FIR is linear phase, right? It is perfectly causal (as any > real system), and it can be seen as the output of a zero-phase > filter, delayed by N/2 samples. Now, this "fictitious" zero-phase > filter would be non-causal; if would require access to N/2 samples > from the future of the input signal. > > Of course, that is possible because what we're doing is that the > output of that fictitious zero-phase filter is being computed > N/2 samples later -- and thus, N/2 samples later we can see > what for that fictitious filter was the future N/2 samples. > > What I'm trying to get at is: what if we do an IIR where the > output of a quasi-zero-phase filter is computed one sample > later -- thus, doing the equivalent of having access to one > sample from the future (then, of course, that quasi-zero-phase > output is now being delayed by one sample). > > I hope I'm explaining better this time. >
I believe that you can only get a zero-phase filter if the filter's response is symmetrical around t = 0. For a FIR filter you can get a linear phase filter because 1/2 the filter length is finite. For an IIR filter you could only get a zero-phase filter if it's response extended to infinity in both directions. You could _improve_ the phase linearity with your extra sample, but you couldn't get _perfect_ phase linearity that way. With analog filters you can implement a Gaussian filter, which has fairly linear phase in the passband at the expense of having the corners of the response being more shallowly sloped than with a Butterworth. I suspect that (a) you could do the same thing with a sampled-time filter and (b) you probably wouldn't gain that much over an FIR filter. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
"Carlos Moreno" <moreno_at_mochima_dot_com@xx.xxx> wrote in message 
news:Co4fe.60629$pe1.955226@wagner.videotron.net...
> > I've been thinking about this crazy idea that I assume has > to be something that someone did already. > > Say that I'm trying to do oversampling; let's take, as an > example, oversampling by a factor of 4. So, what we do is, > just put a sample of the signal, then three samples with > zero, then apply a low-pass filter (with cutoff pi/4). > > This low-pass filter should, in principle, be a 0-phase FIR > (in reality, it is a linear-phase FIR, and we determine the > oversampled signal with some delay). > > Now, the above technique is not acceptable for real-time > applications with critical/tough delay requirements (i.e., > if we can not afford the delay introduced by the length > required for the FIR, then the above is not an option). > > Let's say that we have a situation where non-linear phase > is also out of the question (i.e., we must reconstruct the > signal such that it is very close to the "true" signal *in > the time domain* -- i.e., quasi-zero-phase is a requirement). > > An IIR would have a very hard time with that task, right? > > I'm thinking, what if we do an IIR in which the output of > the filter is the previous sample (or N samples before, in > general). > > The idea is that, intuitively, a casual IIR has a hard time > reconstructing with quasi-zero-phase because it can not know > where the signal is going. But then, if we allow it to > know one sample in the future (or, in general, N samples in > the future, where N is much smaller than the N we would need > for a linear phase FIR), it could do a much better job. > > Is this a standard technique? Are there existing algorithms > to design IIR filters as close as zero-phase as possible > given these conditions? Or is it completely impossible with > this approach? (i.e., would I have to do a regular IIR and > then use a non-linear estimator? Or simply use a non-linear > estimator with the input signal alone?) > > I'll be grateful for any ideas/pointers.
Carlos, The first thing I would do is to stop thinking "phase" and start thinking "delay". Then "zero phase" becomes "zero delay" and "linear phase" becomes "fixed delay". To have zero delay, a filter becomes non-causal. This is fine in batch processing but not in real-time processing. You ask: "are there existing algorithms to design IIR filters as close as zero-phase as possible?" which is the same as asking "are there existing algorithms to design IIR filters as close as zero-delay as possible?" Think of delay as being the centroid of the impulse response. You want the centroid to be as close to zero as possible. That implies non-causal or it implies a very short impulse response indeed! It doesn't matter if it's FIR or IIR in structure because filters with IIR structure can be FIR. I would leave that distinction out of the discussion until you have dealt with *delay*. Fred PS: FIR does not imply linear phase. *Symmetric* FIR implies linear phase. Maybe you want minimum phase which implies minimum delay.
Ok...  Either I'm being excessively dense and am not understanding
that the replies so far are indeed answering my question, or no-one
has really understood what I'm asking...

So, let's try to re-phrase it...

I need to digitally oversample a signal.  With high presicion in
the time domain.  In real-time (with low delay).

One obvious solution that fails the last requirement is to use an
FIR filter.  A two-million taps low-pass filter at the appropriate
frequency  (the "two-million taps" is a figure of speedh -- what I
mean is that if I require lots of precision, I could increase the
length of the FIR response, and accomplish the task...  At the
expense of longer and longer delays).

What if I told you that I can afford a delay of two samples?

Is there a way to reconstruct the value of x[n-1] at time n
with high precision in the time-domain?  (in the context of a
low-pass oversampling filter)

Perhaps what I'm asking is equivalent to an IIR filter with
quasi-linear phase for frequencies below some point?  (a quasi-
linear phase corresponding to a physical delay of 1 sample?)

Thanks,

Carlos
--
Carlos Moreno wrote:

> Ok... Either I'm being excessively dense and am not understanding > that the replies so far are indeed answering my question, or no-one > has really understood what I'm asking... > > So, let's try to re-phrase it...
Keep going, either you or this forum (or both!) will get it eventually. :-)
> I need to digitally oversample a signal. With high presicion in > the time domain. In real-time (with low delay). > > One obvious solution that fails the last requirement is to use an > FIR filter.
OK. You need to "oversample a signal". To me that means you're sampling at something significantly greater than twice the highest frequency it contains (if it's baseband sampling). "With high precision in the time domain" just means (again, to me) that you need to not have too much jitter on the sampling instants. "In real-time (with a low delay)" seems, to me, to be a tautology because if you're sampling a signal you HAVE to do it in real-time, no? The thing that throws me is the next sentence: "One obvious solution that fails the last requirement is to use an FIR filter." People implement FIR filters in real-time every day. What I'm struggling to understand is what you mean by "low delay". FIR filters can be implemented so that with every sample in, you can output a filtered sample... regardless of the length of the FIR filter. If the FIR filter is linear phase, and you were listening to the input in one ear and the output in another ear... and the FIR filter length was 1 second, then you would hear a 0.5 second delay between one ear and t'other.
> A two-million taps low-pass filter at the appropriate > frequency (the "two-million taps" is a figure of speedh -- what I > mean is that if I require lots of precision, I could increase the > length of the FIR response, and accomplish the task... At the > expense of longer and longer delays).
OK.
> What if I told you that I can afford a delay of two samples? > > Is there a way to reconstruct the value of x[n-1] at time n > with high precision in the time-domain? (in the context of a > low-pass oversampling filter)
I'm not sure what you mean by x[n-1]... if it's the input signal, then I already know x[n-1] at time n.... because it was the last sample.
> Perhaps what I'm asking is equivalent to an IIR filter with > quasi-linear phase for frequencies below some point? (a quasi- > linear phase corresponding to a physical delay of 1 sample?)
It sounds like that's what you're asking for... but is it really what you want? If the system you're after needs a _really_ high order FIR filter and a _really_ low delay, then your specifications sound way too tight. So tight, that even an IIR filter will probably not help much. Your earlier message seemed to say that you wanted to reconstruct a signal at 4 times the original sampling rate... in real-time with the original signal. The only way I can think of doing what you're asking is by using some sort of adaptive filter... but you'd sacrifice the "high precision" part of what you're asking... I think. Ciao, Peter K.
Carlos Moreno wrote:
> > Ok... Either I'm being excessively dense and am not understanding > that the replies so far are indeed answering my question, or no-one > has really understood what I'm asking... > > So, let's try to re-phrase it... > > I need to digitally oversample a signal. With high presicion in > the time domain. In real-time (with low delay).
Why? Any sample rate above the minimum adds no information.
> One obvious solution that fails the last requirement is to use an > FIR filter. A two-million taps low-pass filter at the appropriate > frequency (the "two-million taps" is a figure of speedh -- what I > mean is that if I require lots of precision, I could increase the > length of the FIR response, and accomplish the task... At the > expense of longer and longer delays).
Not so. The longer filter is run at a higher rate, so it's delay is the same.
> What if I told you that I can afford a delay of two samples?
I would tell you that you could do very little filtering. Increasing the time between samples helps, though.
> Is there a way to reconstruct the value of x[n-1] at time n > with high precision in the time-domain? (in the context of a > low-pass oversampling filter)
The faster you sample, the closer x[n-1] is to x[n] is in time, and the easier it is to approximate. If you sample at 1000 times the Nyquist rate, assuming that x[n-1] equals x[n] will probably meet your accuracy requirement. If it doesn't, add another zero. Filters don't do sampling, they process samples. What do you mean by "oversampling filter"?
> Perhaps what I'm asking is equivalent to an IIR filter with > quasi-linear phase for frequencies below some point? (a quasi- > linear phase corresponding to a physical delay of 1 sample?)
You're making some assumptions that aren't shared by your audience, but they aren't stated. That makes it easy to get into the kind of exchange you've been happening. -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;
"Carlos Moreno" <moreno_at_mochima_dot_com@xx.xxx> wrote in message 
news:PNufe.9898$vN2.139056@wagner.videotron.net...
> > Ok... Either I'm being excessively dense and am not understanding > that the replies so far are indeed answering my question, or no-one > has really understood what I'm asking... > > So, let's try to re-phrase it... > > I need to digitally oversample a signal. With high presicion in > the time domain. In real-time (with low delay). > > One obvious solution that fails the last requirement is to use an > FIR filter. A two-million taps low-pass filter at the appropriate > frequency (the "two-million taps" is a figure of speedh -- what I > mean is that if I require lots of precision, I could increase the > length of the FIR response, and accomplish the task... At the > expense of longer and longer delays). > > What if I told you that I can afford a delay of two samples? > > Is there a way to reconstruct the value of x[n-1] at time n > with high precision in the time-domain? (in the context of a > low-pass oversampling filter) > > Perhaps what I'm asking is equivalent to an IIR filter with > quasi-linear phase for frequencies below some point? (a quasi- > linear phase corresponding to a physical delay of 1 sample?) > > Thanks, > > Carlos
Carlos, I think it would help (a little) if you would define in more detail exactly what you mean by oversampling. Then, what is wrong with all the answers so far? That's unclear to me. I still think the questions have been addressed. Fred
> --
Jerry Avins ha escrito:
> Carlos Moreno wrote: > > I need to digitally oversample a signal. With high presicion in > > the time domain. In real-time (with low delay). > > Why? Any sample rate above the minimum adds no information. >
That is *almost* true (it can't be applied to time-limited signals, except in approximation). However, oversampling *can* have some uses (not that I know many, to be honest). It would help a lot if we knew why there is a need to oversample and exacly why there must be a low delay.
> > One obvious solution that fails the last requirement is to use an > > FIR filter. A two-million taps low-pass filter at the appropriate > > frequency (the "two-million taps" is a figure of speedh -- what I > > mean is that if I require lots of precision, I could increase the > > length of the FIR response, and accomplish the task... At the > > expense of longer and longer delays). > > Not so. The longer filter is run at a higher rate, so it's delay is
the
> same. > > > What if I told you that I can afford a delay of two samples? > > I would tell you that you could do very little filtering. Increasing
the
> time between samples helps, though.
I agree. A delay of only two samples is of no use if you want any quality filtering. If instead you found out the maximum time delay that you could afford, then you could possibly translate that into a decent number of samples, enough for some filtering.
> > Is there a way to reconstruct the value of x[n-1] at time n > > with high precision in the time-domain? (in the context of a > > low-pass oversampling filter) > > The faster you sample, the closer x[n-1] is to x[n] is in time, and
the
> easier it is to approximate. If you sample at 1000 times the Nyquist > rate, assuming that x[n-1] equals x[n] will probably meet your
accuracy
> requirement.
Very true. Plus, if you only sample at 4 or 5 times the Nyquist rate, cubic extrapolation would get a decent approximation at the cost of some extra processing (which is not of much relevance if your processor can apply long FIR filters in real time). But truth to tell, I don't see why you would want to do that. Could you tell us?