Has anyone come across any alternatives to matlabs filtfilt function as it pertains to zero phase filtering with an FIR? It seems that Mathworks did some work in eliminating some of the transient response but the results arent 'symetric' which is kinda bothersome and in this case i really wish to minimize the transient response of a FIR output without chopping off samples. I have heard of a partial fractions method and a IEEE 'Determining Initial States in forward-backward Filtering' method. (the notation on that article looks a little complx, but ill dig into it if i have to). I just wanted to bounce the question off anyone out there before i mess around with that IEEE article. Has anyone done zerophase filtering with PFE methods? Any help would be appreciated.
alternative to Matlab's filtfilt function
Started by ●January 5, 2012
Reply by ●January 5, 20122012-01-05
On 5 Jan, 22:37, "westocl" <cweston_@n_o_s_p_a_m.hotmail.com> wrote:> Has anyone come across any alternatives to matlabs filtfilt function as it > pertains to zero phase filtering with an FIR? > > It seems that Mathworks did some work in eliminating some of the transient > response but the results arent 'symetric' which is kinda bothersome and in > this case i really wish to minimize the transient response of a FIR output > without chopping off samples.Forget about TMW. Do this yourself - it's not even trivial if you use a FIR filter. Rune
Reply by ●January 6, 20122012-01-06
On 1/5/12 4:37 PM, westocl wrote:> > I have heard of a partial fractions method and a IEEE 'Determining Initial > States in forward-backward Filtering' method. (the notation on that article > looks a little complx, but ill dig into it if i have to). >i dunno why the initial states would ever be anything other than zero. if you want to do zero-phase filtering, you want to use an FIR with zero-initialized states. if the FIR length is L, this increases the file length by L-1 samples. then take the resulting file and pass that through the very same FIR again, but starting from the last sample and working backwards. and the FIR should be zero-initialized this second time.> Has anyone done zerophase filtering with PFE methods?what'sa PFE method? -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
Reply by ●January 6, 20122012-01-06
Hi, I had a look at the Oppenheim-Schafer reference from the Matlab help page. It's an exercise (and an interesting one), but maybe not the most practical way to construct a linear-phase filter. You'll be probably better off with a FIR filter to start with. If I trivially compensate the group delay, it's essentially a non-causal filter, which circumvents the whole problem. In general terms, time domain ringing is related to the transition bandwidth of your filter. For example, if I build a 10kHz lowpass with a stopband at 10.001 kHz, the impulse response will ring for seconds. Using a wider and smoother transition bandwidth helps. BTW, here is an interesting prototype for a FIR filter that apparently manages with the shortest possible truncated impulse response: http://www.dsprelated.com/showcode/236.php What I'd try is to design a FIR filter with the function gPulseCoefs(), and compare the "time dispersion" of a relevant input signal with a conventional FIR design, such as these: http://www.dsprelated.com/showcode/210.php If you can take the FFT of your input signal (let's say up to 100M samples it's no big deal), the lazy way is to simply FFT, scale with your desired frequency response and IFFT. But, the time-bandwidth product can't be cheated: If the frequency response is very abrupt, there will be excessive ringing.
Reply by ●January 6, 20122012-01-06
On Thu, 05 Jan 2012 15:37:25 -0600, westocl wrote:> Has anyone come across any alternatives to matlabs filtfilt function as > it pertains to zero phase filtering with an FIR?Actually understanding what you're doing instead of blindly applying Matlab functions is a good start.> It seems that Mathworks did some work in eliminating some of the > transient response but the results arent 'symetric' which is kinda > bothersome and in this case i really wish to minimize the transient > response of a FIR output without chopping off samples.Which is an odd thing to say with respect to 'filtfilt' because it is not, at it's core, an FIR filter -- it's a function that runs a generic (IIR or FIR) filter both backwards and forwards on a set of data; running it both ways does force the response to be symmetrical, and hence have zero phase shift (when you ignore causality), but there are much better ways to do this (starting with gaining an understanding of the basics).> I have heard of a partial fractions method and a IEEE 'Determining > Initial States in forward-backward Filtering' method. (the notation on > that article looks a little complx, but ill dig into it if i have to).That remark makes no sense in connection with FIR filtering, but it makes oodles of sense in connection with IIR filtering.> I just wanted to bounce the question off anyone out there before i mess > around with that IEEE article. > > Has anyone done zerophase filtering with PFE methods? Any help would be > appreciated.Your feet seem to be on the wrong path. Instead of asking us for ever- fancier ways to blunder around in the dark, how about you ask us how to get onto a path that's well-lit? What are you trying to _do_? Not how do you propose to filter your data -- no, tell us what data you have, what you want to get out of it, and what your constraints are. The simple answer to your question is probably to tell you to use a _real_ FIR filter, unless you have a tightly-windowed data set. In that latter case then your best bet is some sort of double-ended Kalman filter (I'm sure there's a name for it, I just don't know what it might be) that comes up with the optimal estimate of the information that you want, at each point, from the data that you have. This would have to take into account the fact that the data beyond outside of what you have is completely unknown rather than zero (note that a FIR filter makes the implicit 'it's all zero' assumption, which leads to transients). -- My liberal friends think I'm a conservative kook. My conservative friends think I'm a liberal kook. Why am I not happy that they have found common ground? Tim Wescott, Communications, Control, Circuits & Software http://www.wescottdesign.com
Reply by ●January 6, 20122012-01-06
On 6 Jan, 18:48, Tim Wescott <t...@seemywebsite.com> wrote:> �In that > latter case then your best bet is some sort of double-ended Kalman filter > (I'm sure there's a name for it, I just don't know what it might be)You might be thinking of a Kalman smoother. As I understand it (which might be wrong), the Kalman filter operates on the data as they become available (real time), while the Kalman smoother operates off-line, by first doing a forward pass which is equivalent to the Kalman filter, and then a subsequent backward pass to 'improve' the estimate, taking into account 'future' data points. Rune
Reply by ●January 6, 20122012-01-06
>On Thu, 05 Jan 2012 15:37:25 -0600, westocl wrote: > >> Has anyone come across any alternatives to matlabs filtfilt function as >> it pertains to zero phase filtering with an FIR? > >Actually understanding what you're doing instead of blindly applying >Matlab functions is a good start. > >> It seems that Mathworks did some work in eliminating some of the >> transient response but the results arent 'symetric' which is kinda >> bothersome and in this case i really wish to minimize the transient >> response of a FIR output without chopping off samples. > >Which is an odd thing to say with respect to 'filtfilt' because it is >not, at it's core, an FIR filter -- it's a function that runs a generic >(IIR or FIR) filter both backwards and forwards on a set of data; running>it both ways does force the response to be symmetrical, and hence have >zero phase shift (when you ignore causality), but there are much better >ways to do this (starting with gaining an understanding of the basics). > >> I have heard of a partial fractions method and a IEEE 'Determining >> Initial States in forward-backward Filtering' method. (the notation on >> that article looks a little complx, but ill dig into it if i have to). > >That remark makes no sense in connection with FIR filtering, but it makes>oodles of sense in connection with IIR filtering. > >> I just wanted to bounce the question off anyone out there before i mess >> around with that IEEE article. >> >> Has anyone done zerophase filtering with PFE methods? Any help would be >> appreciated. > >Your feet seem to be on the wrong path. Instead of asking us for ever- >fancier ways to blunder around in the dark, how about you ask us how to >get onto a path that's well-lit? > >What are you trying to _do_? Not how do you propose to filter your data >-- no, tell us what data you have, what you want to get out of it, and >what your constraints are. > >The simple answer to your question is probably to tell you to use a >_real_ FIR filter, unless you have a tightly-windowed data set. In that >latter case then your best bet is some sort of double-ended Kalman filter>(I'm sure there's a name for it, I just don't know what it might be) that>comes up with the optimal estimate of the information that you want, at >each point, from the data that you have. This would have to take into >account the fact that the data beyond outside of what you have is >completely unknown rather than zero (note that a FIR filter makes the >implicit 'it's all zero' assumption, which leads to transients). > >-- >My liberal friends think I'm a conservative kook. >My conservative friends think I'm a liberal kook. >Why am I not happy that they have found common ground? > >Tim Wescott, Communications, Control, Circuits & Software >http://www.wescottdesign.comThank all of you for your responses. Tim, this 'sort of double ended Kalman' is what i seek after. And the 'implicit 'it's all zero' assumption' is the problem im seeing. Using matlabs filtfilt function was my first attempt and some prepackaged algorithm trying to give me an estimate of the initial states of the fir (which didnt work btw). So its not the zero phase effect that interested me about the filtfilt, it was the work in minimizing transients. In general the startup and ending transients dont bother us, but in my application it does.>This would have to take into >account the fact that the data beyond outside of what you have is >completely unknown rather than zeroHmm.. it seems that we can do somewhat better than completely unkown if we assume the signal is band limited to some reigion. very interesting thought about the Kalman. So i restate the question as does anyone have any experiences with algorithms that minimize startup and ending trainsients?.. possibly in the lease squares since? If you guys get the chance to google, 'Determining the initial states in forward and backward filtering' by Frederik Gustafsson (coulnt paste the link).
Reply by ●January 6, 20122012-01-06
On 6 Jan, 20:51, "westocl" <cweston_@n_o_s_p_a_m.hotmail.com> wrote:> So i restate the question as does anyone have any experiences with > algorithms that minimize startup and ending trainsients?.. possibly in the > lease squares since?No personal experience, but I am doing repeated attempts to get through the book by Durbin and Koopman, http://www.amazon.com/Analysis-Methods-Oxford-Statistical-Science/dp/0198523548/ref=sr_1_sc_1?ie=UTF8&qid=1325880218&sr=8-1-spell which addresses the initialization of Kalman filters in the case of short data series. Not very easy to read, though. The authors have published a C package for doing the computations described in the book, but I have not tested it. Rune
Reply by ●January 6, 20122012-01-06
On Fri, 06 Jan 2012 10:28:59 -0800, Rune Allnor wrote:> On 6 Jan, 18:48, Tim Wescott <t...@seemywebsite.com> wrote: > >> In that >> latter case then your best bet is some sort of double-ended Kalman >> filter (I'm sure there's a name for it, I just don't know what it might >> be) > > You might be thinking of a Kalman smoother. As I understand it (which > might be wrong), the Kalman filter operates on the data as they become > available (real time), while the Kalman smoother operates off-line, by > first doing a forward pass which is equivalent to the Kalman filter, and > then a subsequent backward pass to 'improve' the estimate, taking into > account 'future' data points.The stuff on Kalman filters that I've read is very much bound to the idea that you're processing the data sequentially: at step 1 you know the 1st measurement, at step 2 you know the second, etc. I haven't seen a Kalmanesque filter that takes in a vector of measurements as a whole and does it's magic thereon. Running a Kalman filter from one end to the other, then doing a Kalman smoothing for each point individually would work -- at least within the constraints of the Kalman filter. Running the filter "backwards" would be unnecessary. -- My liberal friends think I'm a conservative kook. My conservative friends think I'm a liberal kook. Why am I not happy that they have found common ground? Tim Wescott, Communications, Control, Circuits & Software http://www.wescottdesign.com
Reply by ●January 6, 20122012-01-06
On 1/6/2012 3:09 AM, robert bristow-johnson wrote:> On 1/5/12 4:37 PM, westocl wrote: >> >> I have heard of a partial fractions method and a IEEE 'Determining >> Initial >> States in forward-backward Filtering' method. (the notation on that >> article >> looks a little complx, but ill dig into it if i have to). >> > > i dunno why the initial states would ever be anything other than zero. > if you want to do zero-phase filtering, you want to use an FIR with > zero-initialized states. if the FIR length is L, this increases the file > length by L-1 samples. then take the resulting file and pass that > through the very same FIR again, but starting from the last sample and > working backwards. and the FIR should be zero-initialized this second time.Why run it backwards? Just make the FIR symmetric, and you get linear phase on the first pass. But you know that.>> Has anyone done zerophase filtering with PFE methods? > > what'sa PFE method? >Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
