IIR not fully causal?

I've been thinking about this crazy idea that I assume has
to be something that someone did already.

Say that I'm trying to do oversampling;  let's take, as an
example, oversampling by a factor of 4.  So, what we do is,
just put a sample of the signal, then three samples with
zero, then apply a low-pass filter (with cutoff pi/4).

This low-pass filter should, in principle, be a 0-phase FIR
(in reality, it is a linear-phase FIR, and we determine the
oversampled signal with some delay).

Now, the above technique is not acceptable for real-time
applications with critical/tough delay requirements (i.e.,
if we can not afford the delay introduced by the length
required for the FIR, then the above is not an option).

Let's say that we have a situation where non-linear phase
is also out of the question (i.e., we must reconstruct the
signal such that it is very close to the "true" signal *in
the time domain* -- i.e., quasi-zero-phase is a requirement).

An IIR would have a very hard time with that task, right?

I'm thinking, what if we do an IIR in which the output of
the filter is the previous sample (or N samples before, in
general).

The idea is that, intuitively, a casual IIR has a hard time
reconstructing with quasi-zero-phase because it can not know
where the signal is going.  But then, if we allow it to
know one sample in the future (or, in general, N samples in
the future, where N is much smaller than the N we would need
for a linear phase FIR), it could do a much better job.

Is this a standard technique?  Are there existing algorithms
to design IIR filters as close as zero-phase as possible
given these conditions?  Or is it completely impossible with
this approach?  (i.e., would I have to do a regular IIR and
then use a non-linear estimator?  Or simply use a non-linear
estimator with the input signal alone?)

I'll be grateful for any ideas/pointers.

Thanks,

Carlos
--

Carlos Moreno wrote:
> 
> I've been thinking about this crazy idea that I assume has
> to be something that someone did already.
> 
> Say that I'm trying to do oversampling;  let's take, as an
> example, oversampling by a factor of 4.  So, what we do is,
> just put a sample of the signal, then three samples with
> zero, then apply a low-pass filter (with cutoff pi/4).
> 
> This low-pass filter should, in principle, be a 0-phase FIR
> (in reality, it is a linear-phase FIR, and we determine the
> oversampled signal with some delay).
> 
> Now, the above technique is not acceptable for real-time
> applications with critical/tough delay requirements (i.e.,
> if we can not afford the delay introduced by the length
> required for the FIR, then the above is not an option).
> 
> Let's say that we have a situation where non-linear phase
> is also out of the question (i.e., we must reconstruct the
> signal such that it is very close to the "true" signal *in
> the time domain* -- i.e., quasi-zero-phase is a requirement).
> 
> An IIR would have a very hard time with that task, right?
> 
> I'm thinking, what if we do an IIR in which the output of
> the filter is the previous sample (or N samples before, in
> general).
> 
> The idea is that, intuitively, a casual IIR has a hard time
> reconstructing with quasi-zero-phase because it can not know
> where the signal is going.  But then, if we allow it to
> know one sample in the future (or, in general, N samples in
> the future, where N is much smaller than the N we would need
> for a linear phase FIR), it could do a much better job.
> 
> Is this a standard technique?  Are there existing algorithms
> to design IIR filters as close as zero-phase as possible
> given these conditions?  Or is it completely impossible with
> this approach?  (i.e., would I have to do a regular IIR and
> then use a non-linear estimator?  Or simply use a non-linear
> estimator with the input signal alone?)
> 
> I'll be grateful for any ideas/pointers.
> 
If I could predict the future I'd have better things to do than building 
IIR filters -- like trading on the stock market.

Yes, you should be able to make a dynamite zero-phase-shift IIR filter 
by the simple expedient of taking a filter prototype H(z) and mirroring 
all of the poles and zeros around the unit circle, i.e. H_nc(z) = H(z) * 
H(1/z).  But then your filter is either non-causal or unstable, neither 
of which is a good thing.  Also, since the impulse response is not only 
infinite, but is now infinite in the _future_, you can't just add delay 
and make a linear phase filter out of it.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Tim Wescott wrote:

>> [...]
>>
>> The idea is that, intuitively, a casual IIR has a hard time
>> reconstructing with quasi-zero-phase because it can not know
>> where the signal is going.  But then, if we allow it to
>> know one sample in the future (or, in general, N samples in
>> the future, where N is much smaller than the N we would need
>> for a linear phase FIR), it could do a much better job.
>>
>> Is this a standard technique?  Are there existing algorithms
>> to design IIR filters as close as zero-phase as possible
>> given these conditions?  Or is it completely impossible with
>> this approach?  (i.e., would I have to do a regular IIR and
>> then use a non-linear estimator?  Or simply use a non-linear
>> estimator with the input signal alone?)
>
> If I could predict the future I'd have better things to do than building 
> IIR filters -- like trading on the stock market.
> 
> Yes, you should be able to make a dynamite zero-phase-shift IIR filter 
> by the simple expedient of taking a filter prototype H(z) and mirroring 
> all of the poles and zeros around the unit circle, i.e. H_nc(z) = H(z) * 
> H(1/z).  But then your filter is either non-causal or unstable, neither 
> of which is a good thing.  Also, since the impulse response is not only 
> infinite, but is now infinite in the _future_, you can't just add delay 
> and make a linear phase filter out of it.

Wait...  I think you didn't understand what I was trying to say.

An FIR is linear phase, right?  It is perfectly causal (as any
real system), and it can be seen as the output of a zero-phase
filter, delayed by N/2 samples.  Now, this "fictitious" zero-phase
filter would be non-causal;  if would require access to N/2 samples
from the future of the input signal.

Of course, that is possible because what we're doing is that the
output of that fictitious zero-phase filter is being computed
N/2 samples later -- and thus, N/2 samples later we can see
what for that fictitious filter was the future N/2 samples.

What I'm trying to get at is:  what if we do an IIR where the
output of a quasi-zero-phase filter is computed one sample
later -- thus, doing the equivalent of having access to one
sample from the future (then, of course, that quasi-zero-phase
output is now being delayed by one sample).

I hope I'm explaining better this time.

Thanks,

Carlos
--

Carlos Moreno wrote:

> Tim Wescott wrote:
> 
>>> [...]
>>>
>>> The idea is that, intuitively, a casual IIR has a hard time
>>> reconstructing with quasi-zero-phase because it can not know
>>> where the signal is going.  But then, if we allow it to
>>> know one sample in the future (or, in general, N samples in
>>> the future, where N is much smaller than the N we would need
>>> for a linear phase FIR), it could do a much better job.
>>>
>>> Is this a standard technique?  Are there existing algorithms
>>> to design IIR filters as close as zero-phase as possible
>>> given these conditions?  Or is it completely impossible with
>>> this approach?  (i.e., would I have to do a regular IIR and
>>> then use a non-linear estimator?  Or simply use a non-linear
>>> estimator with the input signal alone?)
>>
>>
>> If I could predict the future I'd have better things to do than 
>> building IIR filters -- like trading on the stock market.
>>
>> Yes, you should be able to make a dynamite zero-phase-shift IIR filter 
>> by the simple expedient of taking a filter prototype H(z) and 
>> mirroring all of the poles and zeros around the unit circle, i.e. 
>> H_nc(z) = H(z) * H(1/z).  But then your filter is either non-causal or 
>> unstable, neither of which is a good thing.  Also, since the impulse 
>> response is not only infinite, but is now infinite in the _future_, 
>> you can't just add delay and make a linear phase filter out of it.
> 
> 
> Wait...  I think you didn't understand what I was trying to say.
> 
> An FIR is linear phase, right?  It is perfectly causal (as any
> real system), and it can be seen as the output of a zero-phase
> filter, delayed by N/2 samples.  Now, this "fictitious" zero-phase
> filter would be non-causal;  if would require access to N/2 samples
> from the future of the input signal.
> 
> Of course, that is possible because what we're doing is that the
> output of that fictitious zero-phase filter is being computed
> N/2 samples later -- and thus, N/2 samples later we can see
> what for that fictitious filter was the future N/2 samples.
> 
> What I'm trying to get at is:  what if we do an IIR where the
> output of a quasi-zero-phase filter is computed one sample
> later -- thus, doing the equivalent of having access to one
> sample from the future (then, of course, that quasi-zero-phase
> output is now being delayed by one sample).
> 
> I hope I'm explaining better this time.
> 
I believe that you can only get a zero-phase filter if the filter's 
response is symmetrical around t = 0.  For a FIR filter you can get a 
linear phase filter because 1/2 the filter length is finite.  For an IIR 
filter you could only get a zero-phase filter if it's response extended 
to infinity in both directions.  You could _improve_ the phase linearity 
with your extra sample, but you couldn't get _perfect_ phase linearity 
that way.

With analog filters you can implement a Gaussian filter, which has 
fairly linear phase in the passband at the expense of having the corners 
of the response being more shallowly sloped than with a Butterworth.  I 
suspect that (a) you could do the same thing with a sampled-time filter 
and (b) you probably wouldn't gain that much over an FIR filter.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

"Carlos Moreno" <moreno_at_mochima_dot_com@xx.xxx> wrote in message 
news:Co4fe.60629$pe1.955226@wagner.videotron.net...
>
> I've been thinking about this crazy idea that I assume has
> to be something that someone did already.
>
> Say that I'm trying to do oversampling;  let's take, as an
> example, oversampling by a factor of 4.  So, what we do is,
> just put a sample of the signal, then three samples with
> zero, then apply a low-pass filter (with cutoff pi/4).
>
> This low-pass filter should, in principle, be a 0-phase FIR
> (in reality, it is a linear-phase FIR, and we determine the
> oversampled signal with some delay).
>
> Now, the above technique is not acceptable for real-time
> applications with critical/tough delay requirements (i.e.,
> if we can not afford the delay introduced by the length
> required for the FIR, then the above is not an option).
>
> Let's say that we have a situation where non-linear phase
> is also out of the question (i.e., we must reconstruct the
> signal such that it is very close to the "true" signal *in
> the time domain* -- i.e., quasi-zero-phase is a requirement).
>
> An IIR would have a very hard time with that task, right?
>
> I'm thinking, what if we do an IIR in which the output of
> the filter is the previous sample (or N samples before, in
> general).
>
> The idea is that, intuitively, a casual IIR has a hard time
> reconstructing with quasi-zero-phase because it can not know
> where the signal is going.  But then, if we allow it to
> know one sample in the future (or, in general, N samples in
> the future, where N is much smaller than the N we would need
> for a linear phase FIR), it could do a much better job.
>
> Is this a standard technique?  Are there existing algorithms
> to design IIR filters as close as zero-phase as possible
> given these conditions?  Or is it completely impossible with
> this approach?  (i.e., would I have to do a regular IIR and
> then use a non-linear estimator?  Or simply use a non-linear
> estimator with the input signal alone?)
>
> I'll be grateful for any ideas/pointers.

Carlos,

The first thing I would do is to stop thinking "phase" and start thinking 
"delay".
Then "zero phase" becomes "zero delay" and "linear phase" becomes "fixed 
delay".

To have zero delay, a filter becomes non-causal.  This is fine in batch 
processing but not in real-time processing.  You ask: "are there existing 
algorithms to design IIR filters as close as zero-phase as possible?"  which 
is the same as asking "are there existing algorithms to design IIR filters 
as close as zero-delay as possible?"

Think of delay as being the centroid of the impulse response.  You want the 
centroid to be as close to zero as possible.  That implies non-causal or it 
implies a very short impulse response indeed!  It doesn't matter if it's FIR 
or IIR in structure because filters with IIR structure can be FIR.  I would 
leave that distinction out of the discussion until you have dealt with 
*delay*.

Fred

PS: FIR does not imply linear phase.  *Symmetric* FIR implies linear phase. 
Maybe you want minimum phase which implies minimum delay.

Ok...  Either I'm being excessively dense and am not understanding
that the replies so far are indeed answering my question, or no-one
has really understood what I'm asking...

So, let's try to re-phrase it...

I need to digitally oversample a signal.  With high presicion in
the time domain.  In real-time (with low delay).

One obvious solution that fails the last requirement is to use an
FIR filter.  A two-million taps low-pass filter at the appropriate
frequency  (the "two-million taps" is a figure of speedh -- what I
mean is that if I require lots of precision, I could increase the
length of the FIR response, and accomplish the task...  At the
expense of longer and longer delays).

What if I told you that I can afford a delay of two samples?

Is there a way to reconstruct the value of x[n-1] at time n
with high precision in the time-domain?  (in the context of a
low-pass oversampling filter)

Perhaps what I'm asking is equivalent to an IIR filter with
quasi-linear phase for frequencies below some point?  (a quasi-
linear phase corresponding to a physical delay of 1 sample?)

Thanks,

Carlos
--

Carlos Moreno wrote:

> Ok...  Either I'm being excessively dense and am not understanding
> that the replies so far are indeed answering my question, or no-one
> has really understood what I'm asking...
>
> So, let's try to re-phrase it...

Keep going, either you or this forum (or both!) will get it eventually.
:-)

> I need to digitally oversample a signal.  With high presicion in
> the time domain.  In real-time (with low delay).
>
> One obvious solution that fails the last requirement is to use an
> FIR filter.

OK. You need to "oversample a signal".  To me that means you're
sampling at something significantly greater than twice the highest
frequency it contains (if it's baseband sampling).

"With high precision in the time domain" just means (again, to me) that
you need to not have too much jitter on the sampling instants.

"In real-time (with a low delay)" seems, to me, to be a tautology
because if you're sampling a signal you HAVE to do it in real-time, no?

The thing that throws me is the next sentence: "One obvious solution
that fails the last requirement is to use an FIR filter."

People implement FIR filters in real-time every day.  What I'm
struggling to understand is what you mean by "low delay".

FIR filters can be implemented so that with every sample in, you can
output a filtered sample... regardless of the length of the FIR filter.

If the FIR filter is linear phase, and you were listening to the input
in one ear and the output in another ear... and the FIR filter length
was 1 second, then you would hear a 0.5 second delay between one ear
and t'other.

> A two-million taps low-pass filter at the appropriate
> frequency  (the "two-million taps" is a figure of speedh -- what I
> mean is that if I require lots of precision, I could increase the
> length of the FIR response, and accomplish the task...  At the
> expense of longer and longer delays).

OK.

> What if I told you that I can afford a delay of two samples?
>
> Is there a way to reconstruct the value of x[n-1] at time n
> with high precision in the time-domain?  (in the context of a
> low-pass oversampling filter)

I'm not sure what you mean by x[n-1]... if it's the input signal, then
I already know x[n-1] at time n.... because it was the last sample.

> Perhaps what I'm asking is equivalent to an IIR filter with
> quasi-linear phase for frequencies below some point?  (a quasi-
> linear phase corresponding to a physical delay of 1 sample?)

It sounds like that's what you're asking for... but is it really what
you want?

If the system you're after needs a _really_ high order FIR filter and a
_really_ low delay, then your specifications sound way too tight.  So
tight, that even an IIR filter will probably not help much.

Your earlier message seemed to say that you wanted to reconstruct a
signal at 4 times the original sampling rate... in real-time with the
original signal.

The only way I can think of doing what you're asking is by using some
sort of adaptive filter... but you'd sacrifice the "high precision"
part of what you're asking... I think.

Ciao,

Peter K.

Carlos Moreno wrote:
> 
> Ok...  Either I'm being excessively dense and am not understanding
> that the replies so far are indeed answering my question, or no-one
> has really understood what I'm asking...
> 
> So, let's try to re-phrase it...
> 
> I need to digitally oversample a signal.  With high presicion in
> the time domain.  In real-time (with low delay).

Why? Any sample rate above the minimum adds no information.

> One obvious solution that fails the last requirement is to use an
> FIR filter.  A two-million taps low-pass filter at the appropriate
> frequency  (the "two-million taps" is a figure of speedh -- what I
> mean is that if I require lots of precision, I could increase the
> length of the FIR response, and accomplish the task...  At the
> expense of longer and longer delays).

Not so. The longer filter is run at a higher rate, so it's delay is the 
same.

> What if I told you that I can afford a delay of two samples?

I would tell you that you could do very little filtering. Increasing the 
time between samples helps, though.

> Is there a way to reconstruct the value of x[n-1] at time n
> with high precision in the time-domain?  (in the context of a
> low-pass oversampling filter)

The faster you sample, the closer x[n-1] is to x[n] is in time, and the 
easier it is to approximate. If you sample at 1000 times the Nyquist 
rate, assuming that x[n-1] equals x[n] will probably meet your accuracy 
requirement. If it doesn't, add another zero.

Filters don't do sampling, they process samples. What do you mean by 
"oversampling filter"?

> Perhaps what I'm asking is equivalent to an IIR filter with
> quasi-linear phase for frequencies below some point?  (a quasi-
> linear phase corresponding to a physical delay of 1 sample?)

You're making some assumptions that aren't shared by your audience, but 
they aren't stated. That makes it easy to get into the kind of exchange 
you've been happening.

-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

"Carlos Moreno" <moreno_at_mochima_dot_com@xx.xxx> wrote in message 
news:PNufe.9898$vN2.139056@wagner.videotron.net...
>
> Ok...  Either I'm being excessively dense and am not understanding
> that the replies so far are indeed answering my question, or no-one
> has really understood what I'm asking...
>
> So, let's try to re-phrase it...
>
> I need to digitally oversample a signal.  With high presicion in
> the time domain.  In real-time (with low delay).
>
> One obvious solution that fails the last requirement is to use an
> FIR filter.  A two-million taps low-pass filter at the appropriate
> frequency  (the "two-million taps" is a figure of speedh -- what I
> mean is that if I require lots of precision, I could increase the
> length of the FIR response, and accomplish the task...  At the
> expense of longer and longer delays).
>
> What if I told you that I can afford a delay of two samples?
>
> Is there a way to reconstruct the value of x[n-1] at time n
> with high precision in the time-domain?  (in the context of a
> low-pass oversampling filter)
>
> Perhaps what I'm asking is equivalent to an IIR filter with
> quasi-linear phase for frequencies below some point?  (a quasi-
> linear phase corresponding to a physical delay of 1 sample?)
>
> Thanks,
>
> Carlos

Carlos,

I think it would help (a little) if you would define in more detail exactly 
what you mean by oversampling.

Then, what is wrong with all the answers so far?  That's unclear to me.  I 
still think the questions have been addressed.

Fred
> -- 

Jerry Avins ha escrito:
> Carlos Moreno wrote:
> > I need to digitally oversample a signal.  With high presicion in
> > the time domain.  In real-time (with low delay).
>
> Why? Any sample rate above the minimum adds no information.
>

That is *almost* true (it can't be applied to time-limited signals,
except in approximation). However, oversampling *can* have some
uses (not that I know many, to be honest).

It would help a lot if we knew why there is a need to oversample
and exacly why there must be a low delay.

> > One obvious solution that fails the last requirement is to use an
> > FIR filter.  A two-million taps low-pass filter at the appropriate
> > frequency  (the "two-million taps" is a figure of speedh -- what I
> > mean is that if I require lots of precision, I could increase the
> > length of the FIR response, and accomplish the task...  At the
> > expense of longer and longer delays).
>
> Not so. The longer filter is run at a higher rate, so it's delay is
the
> same.
>
> > What if I told you that I can afford a delay of two samples?
>
> I would tell you that you could do very little filtering. Increasing
the
> time between samples helps, though.

I agree. A delay of only two samples is of no use if you want any
quality filtering. If instead you found out the maximum time delay
that you could afford, then you could possibly translate that into
a decent number of samples, enough for some filtering.

> > Is there a way to reconstruct the value of x[n-1] at time n
> > with high precision in the time-domain?  (in the context of a
> > low-pass oversampling filter)
>
> The faster you sample, the closer x[n-1] is to x[n] is in time, and
the
> easier it is to approximate. If you sample at 1000 times the Nyquist
> rate, assuming that x[n-1] equals x[n] will probably meet your
accuracy
> requirement.

Very true. Plus, if you only sample at 4 or 5 times the Nyquist
rate, cubic extrapolation would get a decent approximation at the cost
of some extra processing (which is not of much relevance if your
processor can apply long FIR filters in real time).

But truth to tell, I don't see why you would want to do that. Could
you tell us?