Newbie Matlab Digital Filter / Feedback Question

On 01/13/2011 01:24 PM, j26 wrote:
>>
>>
>> j26 wrote:
>>
>>> By the way, would the non-linear filter have variable gain or would it
> have
>>> variable bandwidth?
>>
>
> Sorry for the multiple posts...I don't like the DSP Related forum interface
> and the duplicates were unintentional and caused by hitting the refresh
> button to get back to this thread...mods, please delete the extra posts.
>
> But Vladdy, why do you feel that my post was a stupid question?  I
> mentioned in the subject line that I'm a DSP newbie and I'm just here to
> learn.  I don't understand why you have to be so offensive.

Apparently, "Stupident" is Russian for "I'm feeling cranky and want to 
lash out today".

He's best if you ignore him when he gets like this.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On 01/13/2011 11:31 AM, j26 wrote:
>> On Thu, 13 Jan 2011 07:44:24 -0600, j26 wrote:
>>
>>> I am trying to use a filter to smooth time-series data.  Although it's
>>> been a good 5 years since my last signal and systems / control systems
>>> course, I have recently started experimenting with the butter and
> cheby2
>>> filter design functions in Octave.  I have a filter that smooths the
>>> input data well (low-order / low cutoff frequency), but sometimes when
>>> the input changes abruptly, the output cannot keep up (I know I could
>>> raise the order of the filter / raise cutoff frequency to improve
>>> response, but that sacrifices smoothness).  How do I apply feedback to
>>> my filter such that the output follows the input closer when there are
>>> large changes in input?  I basically want to take my filter and add on
> a
>>> factor equal to the difference between the input and output.  I
> remember
>>> that there was a function called feedback for the continuous time case,
>>> but don't recall how to apply feedback to a discrete time filter.
>>
>> As long as you have a linear filter, using feedback is going to leave you
>
>> with the same tradeoff between smoothness and transient response that you
>
>> have now.
>>
>> It sounds like you're using IIR filters -- have you considered using FIR
>> filters?  A good FIR filter will still have delay, but you may like the
>> shape of the transient response better.
>>
>> If that doesn't make you happy, then you'd need some sort of a nonlinear
>> filter, one that looks for a large difference between input and output
>> (or for large jumps in the input) and changes the filter gains until the
>> output has caught up.
>>
>> --
>> http://www.wescottdesign.com
>>
>
> I'll try the FIR filter, there's design functions for that in Octave.  But
> how would I go about designing a non-linear filter?  Are there any guides,
> functions, or resources that you can recommend?  I appreciate all the
> responses.

Generally nonlinear filter design involves the use of a lot more 
intuition and cut-and-try experimentation than linear filter design.  So 
in general you go about it by asking how it may be done (see below), 
then by trying it out on your data, and by seeing if it works.

I would start by making a low-order IIR filter, no more than 3rd order, 
and after I tried that I may retreat to 2nd order.  Then I'd figure out 
how I could switch the values of the coefficients of the filter without 
having a jump in the filter output and without having the filter output 
'go crazy'.

Then I'd monitor the difference between the input and output, and 
whenever it was above some threshold I'd switch the filter to the "tight 
following" mode; when it got below that threshold I'd switch the filter 
to "slow following" mode, or I'd let it transition there gracefully.

One method you could use to design this would be to design a Kalman 
filter to follow the signal in the absence of jumps.  Then when you 
detect a jump, artificially add some ginormous amount to the Kalman's 
variance estimate.  That'll force it to let in more signal (and noise), 
while gracefully settling back to a "slow following" mode in a 
controlled and smooth manner.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On 01/13/2011 02:27 PM, Tim Wescott wrote:
> On 01/13/2011 11:31 AM, j26 wrote:
>>> On Thu, 13 Jan 2011 07:44:24 -0600, j26 wrote:
>>>
>>>> I am trying to use a filter to smooth time-series data. Although it's
>>>> been a good 5 years since my last signal and systems / control systems
>>>> course, I have recently started experimenting with the butter and
>> cheby2
>>>> filter design functions in Octave. I have a filter that smooths the
>>>> input data well (low-order / low cutoff frequency), but sometimes when
>>>> the input changes abruptly, the output cannot keep up (I know I could
>>>> raise the order of the filter / raise cutoff frequency to improve
>>>> response, but that sacrifices smoothness). How do I apply feedback to
>>>> my filter such that the output follows the input closer when there are
>>>> large changes in input? I basically want to take my filter and add on
>> a
>>>> factor equal to the difference between the input and output. I
>> remember
>>>> that there was a function called feedback for the continuous time case,
>>>> but don't recall how to apply feedback to a discrete time filter.
>>>
>>> As long as you have a linear filter, using feedback is going to leave
>>> you
>>
>>> with the same tradeoff between smoothness and transient response that
>>> you
>>
>>> have now.
>>>
>>> It sounds like you're using IIR filters -- have you considered using FIR
>>> filters? A good FIR filter will still have delay, but you may like the
>>> shape of the transient response better.
>>>
>>> If that doesn't make you happy, then you'd need some sort of a nonlinear
>>> filter, one that looks for a large difference between input and output
>>> (or for large jumps in the input) and changes the filter gains until the
>>> output has caught up.
>>>
>>> --
>>> http://www.wescottdesign.com
>>>
>>
>> I'll try the FIR filter, there's design functions for that in Octave. But
>> how would I go about designing a non-linear filter? Are there any guides,
>> functions, or resources that you can recommend? I appreciate all the
>> responses.
>
> Generally nonlinear filter design involves the use of a lot more
> intuition and cut-and-try experimentation than linear filter design. So
> in general you go about it by asking how it may be done (see below),
> then by trying it out on your data, and by seeing if it works.
>
> I would start by making a low-order IIR filter, no more than 3rd order,
> and after I tried that I may retreat to 2nd order. Then I'd figure out
> how I could switch the values of the coefficients of the filter without
> having a jump in the filter output and without having the filter output
> 'go crazy'.
>
> Then I'd monitor the difference between the input and output, and
> whenever it was above some threshold I'd switch the filter to the "tight
> following" mode; when it got below that threshold I'd switch the filter
> to "slow following" mode, or I'd let it transition there gracefully.
>
> One method you could use to design this would be to design a Kalman
> filter to follow the signal in the absence of jumps. Then when you
> detect a jump, artificially add some ginormous amount to the Kalman's
> variance estimate. That'll force it to let in more signal (and noise),
> while gracefully settling back to a "slow following" mode in a
> controlled and smooth manner.
>
I should mention that the easiest nonlinear filter I know of is one that 
goes something like this:

Let x be the filter input.
Let y be the filter state, and filter output.

Compute ye = x - y.

Now compute the new filter state:

     {  y = ye * a1 if ye < threshold
y = {
     {  y = ye * a2 if ye >= threshold

As long as you choose 0 < a1 <= a2 < 1, your filter will be stable and 
well behaved.  It'll respond quickly to transients, and filter a lot for 
smaller inputs -- but it's a pretty simplistic filter, so it'll have a 
lot of drawbacks, too.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

Tom, I really appreciate your responses.  When I asked the question earlier
about a non-linear filter having variable bandwidth vs. variable gain, I
was thinking that the essential characteristic of a non-linear filter is
the variable bandwidth -- it's frequency response changes based upon the
input signal.  Although, after thinking about it some more, that's the same
thing as variable gain (the gain applied to each frequency changes based on
input signal).  Earlier, I was thinking that gain was more analogous to the
order of the filter.

Limiting / lowering the cutoff frequency of a low-pass filter will produce
a smoother output signal although it will not track the output signal as
closely.  In my original post, I was thinking about compensating for this
by having a correction factor proportional to the difference (essentially
applying feedback to the filter).  But upon thinking about this some more,
that can't be exactly what I need either because every linear feedback
system composed of multiple transfer functions can be described with a
single transfer function.  But non-linear rather than linear feedback
sounds like what I'm after.  

I'm not surprised that non-linear filter design is more of a
trial-and-error art rather than a science.  The tuning or optimizing of
coefficients sounds like a good use for genetic algorithms.

On 1/13/2011 4:34 PM, j26 wrote:
> Tom, I really appreciate your responses.  When I asked the question earlier
> about a non-linear filter having variable bandwidth vs. variable gain, I
> was thinking that the essential characteristic of a non-linear filter is
> the variable bandwidth -- it's frequency response changes based upon the
> input signal.  Although, after thinking about it some more, that's the same
> thing as variable gain (the gain applied to each frequency changes based on
> input signal).  Earlier, I was thinking that gain was more analogous to the
> order of the filter.
>
> Limiting / lowering the cutoff frequency of a low-pass filter will produce
> a smoother output signal although it will not track the output signal as
> closely.  In my original post, I was thinking about compensating for this
> by having a correction factor proportional to the difference (essentially
> applying feedback to the filter).  But upon thinking about this some more,
> that can't be exactly what I need either because every linear feedback
> system composed of multiple transfer functions can be described with a
> single transfer function.  But non-linear rather than linear feedback
> sounds like what I'm after.
>
> I'm not surprised that non-linear filter design is more of a
> trial-and-error art rather than a science.  The tuning or optimizing of
> coefficients sounds like a good use for genetic algorithms.

That's a fairly limited definition of non-linear filtering.  There are 
also things like median filtering that really can't be understood as a 
variation on the standard FIR/IIR concepts.

-- 
Rob Gaddi, Highland Technology
Email address is currently out of order

On 01/13/2011 04:38 PM, Rob Gaddi wrote:
> On 1/13/2011 4:34 PM, j26 wrote:
>> Tom, I really appreciate your responses. When I asked the question
>> earlier
>> about a non-linear filter having variable bandwidth vs. variable gain, I
>> was thinking that the essential characteristic of a non-linear filter is
>> the variable bandwidth -- it's frequency response changes based upon the
>> input signal. Although, after thinking about it some more, that's the
>> same
>> thing as variable gain (the gain applied to each frequency changes
>> based on
>> input signal). Earlier, I was thinking that gain was more analogous to
>> the
>> order of the filter.
>>
>> Limiting / lowering the cutoff frequency of a low-pass filter will
>> produce
>> a smoother output signal although it will not track the output signal as
>> closely. In my original post, I was thinking about compensating for this
>> by having a correction factor proportional to the difference (essentially
>> applying feedback to the filter). But upon thinking about this some more,
>> that can't be exactly what I need either because every linear feedback
>> system composed of multiple transfer functions can be described with a
>> single transfer function. But non-linear rather than linear feedback
>> sounds like what I'm after.
>>
>> I'm not surprised that non-linear filter design is more of a
>> trial-and-error art rather than a science. The tuning or optimizing of
>> coefficients sounds like a good use for genetic algorithms.
>
> That's a fairly limited definition of non-linear filtering. There are
> also things like median filtering that really can't be understood as a
> variation on the standard FIR/IIR concepts.
>
Yup.  In fact, in general you have to toss concepts like "bandwidth" and 
"gain" out the window when you're dealing with nonlinear systems.  Not 
always -- which is part of the reason that nonlinear filter design can 
get weird -- but you can't count on the terms meaning anything.

The whole field of linear systems is this wonderful exact science that 
has absolutely no example in the real world.  There are systems that 
come awfully close -- but there aren't really _any_ linear systems in 
the real world.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On 01/13/2011 09:02 PM, Tim Wescott wrote:
> [...]
> The whole field of linear systems is this wonderful exact science that has absolutely no example in the real world. There are
> systems that come awfully close -- but there aren't really _any_ linear systems in the real world.

Sure, just as there aren't really any straight lines, round circles, quiet rooms,
pure colors, etc.
-- 
Randy Yates                      % "My Shangri-la has gone away, fading like
Digital Signal Labs              %  the Beatles on 'Hey Jude'"
yates@digitalsignallabs.com      %
http://www.digitalsignallabs.com % 'Shangri-La', *A New World Record*, ELO

>On 01/13/2011 09:02 PM, Tim Wescott wrote:
>> [...]
>> The whole field of linear systems is this wonderful exact science that
has absolutely no example in the real world. There are
>> systems that come awfully close -- but there aren't really _any_ linear
systems in the real world.
>
>Sure, just as there aren't really any straight lines, round circles, quiet
rooms,
>pure colors, etc.
>-- 
>Randy Yates                      % "My Shangri-la has gone away, fading
like
>Digital Signal Labs              %  the Beatles on 'Hey Jude'"
>yates@digitalsignallabs.com      %
>http://www.digitalsignallabs.com % 'Shangri-La', *A New World Record*,
ELO
>

I appreciate all the advice.  Can anyone recommend a book relevant to using
non-linear filters for filtering time series data?  Otherwise, I plan to
buy this book:

http://www.powells.com/biblio/62-9780123725363-1

..Hope that's the right place to start.

On 01/14/2011 01:40 AM, j26 wrote:
>> On 01/13/2011 09:02 PM, Tim Wescott wrote:
>>> [...]
>>> The whole field of linear systems is this wonderful exact science that
> has absolutely no example in the real world. There are
>>> systems that come awfully close -- but there aren't really _any_ linear
> systems in the real world.
>>
>> Sure, just as there aren't really any straight lines, round circles, quiet
> rooms,
>> pure colors, etc.
>> --
>> Randy Yates                      % "My Shangri-la has gone away, fading
> like
>> Digital Signal Labs              %  the Beatles on 'Hey Jude'"
>> yates@digitalsignallabs.com      %
>> http://www.digitalsignallabs.com % 'Shangri-La', *A New World Record*,
> ELO
>>
>
> I appreciate all the advice.  Can anyone recommend a book relevant to using
> non-linear filters for filtering time series data?  Otherwise, I plan to
> buy this book:
>
> http://www.powells.com/biblio/62-9780123725363-1
>
> ..Hope that's the right place to start.

It seems you want the impossible, e.g., to design a filter
that a) doesn't allow fast changes (i.e., smooths) and b)
that allows fast changes.
-- 
Randy Yates                      % "My Shangri-la has gone away, fading like
Digital Signal Labs              %  the Beatles on 'Hey Jude'"
yates@digitalsignallabs.com      %
http://www.digitalsignallabs.com % 'Shangri-La', *A New World Record*, ELO

On 01/13/2011 11:15 PM, Randy Yates wrote:
> On 01/14/2011 01:40 AM, j26 wrote:
>>> On 01/13/2011 09:02 PM, Tim Wescott wrote:
>>>> [...]
>>>> The whole field of linear systems is this wonderful exact science that
>> has absolutely no example in the real world. There are
>>>> systems that come awfully close -- but there aren't really _any_ linear
>> systems in the real world.
>>>
>>> Sure, just as there aren't really any straight lines, round circles,
>>> quiet
>> rooms,
>>> pure colors, etc.
>>> --
>>> Randy Yates % "My Shangri-la has gone away, fading
>> like
>>> Digital Signal Labs % the Beatles on 'Hey Jude'"
>>> yates@digitalsignallabs.com %
>>> http://www.digitalsignallabs.com % 'Shangri-La', *A New World Record*,
>> ELO
>>>
>>
>> I appreciate all the advice. Can anyone recommend a book relevant to
>> using
>> non-linear filters for filtering time series data? Otherwise, I plan to
>> buy this book:
>>
>> http://www.powells.com/biblio/62-9780123725363-1
>>
>> ..Hope that's the right place to start.
>
> It seems you want the impossible, e.g., to design a filter
> that a) doesn't allow fast changes (i.e., smooths) and b)
> that allows fast changes.

But to not allow fast changes _when the input is stable_ but allow them 
_when the input jumps_ -- that's not only possible, that's a nonlinear 
filter.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html