Discover filter coefficients of system.

I'm wanting to model a real world system using an FIR filter. I have
samples of the input and the expected output. How does one find the
filter coefficients from these? I've been using particle swarm
optimization, but it's very approximate. It's not getting as close as
I want it to so I'm looking to try something new. I'm sure there are
more exact techniques out there. I was told by a friend there is an
iterative technique using an auto-correlation matrix that might do
what I want, but we were talking fast and I didn't follow all that he
was saying. I did some quick searching and didn't find much. Can
anyone point me in the right direction? Thanks for the help.

-----Jay

horndude77@gmail.com wrote:
> I'm wanting to model a real world system using an FIR filter. I have
> samples of the input and the expected output. How does one find the
> filter coefficients from these? I've been using particle swarm
> optimization, but it's very approximate. It's not getting as close as
> I want it to so I'm looking to try something new. I'm sure there are
> more exact techniques out there. I was told by a friend there is an
> iterative technique using an auto-correlation matrix that might do
> what I want, but we were talking fast and I didn't follow all that he
> was saying. I did some quick searching and didn't find much. Can
> anyone point me in the right direction? Thanks for the help.

Filters change the relative amplitudes of frequency components in a 
signal. The problem of finding coefficients to achieve a particular 
frequency response is solved; in the literature, by available programs, 
and by a little known method invented by (I believe) Greg Berchin, who 
frequents this group. To you them, you need to determine the frequency 
response or alternatively the impulse response that will solve your 
problem. Presumably, that will mimic the impulse response of the system 
you want to model.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

horndude77@gmail.com wrote:
 > I'm wanting to model a real world system using an FIR filter. I have
 > samples of the input and the expected output. How does one find the
 > filter coefficients from these? I've been using particle swarm
 > optimization, but it's very approximate. It's not getting as close as
 > I want it to so I'm looking to try something new. I'm sure there are
 > more exact techniques out there. I was told by a friend there is an
 > iterative technique using an auto-correlation matrix that might do
 > what I want, but we were talking fast and I didn't follow all that he
 > was saying. I did some quick searching and didn't find much. Can
 > anyone point me in the right direction? Thanks for the help.

Filters change the relative amplitudes of frequency components in a 
signal. The problem of finding coefficients to achieve a particular 
frequency response is solved; in the literature, by available programs, 
and by a little known method invented by (I believe) Greg Berchin, who 
frequents this group. To use them, you need to determine the frequency 
response or alternatively the impulse response that will solve your 
problem. Presumably, that will mimic the impulse response of the system 
you want to model.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

On Feb 19, 10:19 pm, horndud...@gmail.com wrote:
> I'm wanting to model a real world system using an FIR filter. I have
> samples of the input and the expected output. How does one find the
> filter coefficients from these? I've been using particle swarm
> optimization, but it's very approximate. It's not getting as close as
> I want it to so I'm looking to try something new. I'm sure there are
> more exact techniques out there. I was told by a friend there is an
> iterative technique using an auto-correlation matrix that might do
> what I want, but we were talking fast and I didn't follow all that he
> was saying. I did some quick searching and didn't find much. Can
> anyone point me in the right direction? Thanks for the help.

isn't this what LMS adaptive filtering is about?  system
identification?

r b-j

On 20 Feb, 06:20, "robert bristow-johnson" <r...@audioimagination.com>
wrote:
> On Feb 19, 10:19 pm, horndud...@gmail.com wrote:
>
> > I'm wanting to model a real world system using an FIR filter. I have
> > samples of the input and the expected output. How does one find the
> > filter coefficients from these? I've been using particle swarm
> > optimization, but it's very approximate. It's not getting as close as
> > I want it to so I'm looking to try something new. I'm sure there are
> > more exact techniques out there. I was told by a friend there is an
> > iterative technique using an auto-correlation matrix that might do
> > what I want, but we were talking fast and I didn't follow all that he
> > was saying. I did some quick searching and didn't find much. Can
> > anyone point me in the right direction? Thanks for the help.
>
> isn't this what LMS adaptive filtering is about?  system
> identification?

Seems to me as if the OP is looking for system identification.
I would suggest AR models as a newbie's approach to the field.

Rune

On Feb 19, 9:19 pm, horndud...@gmail.com wrote:
> I'm wanting to model a real world system using an FIR filter. I have
> samples of the input and the expected output. How does one find the
> filter coefficients from these? I've been using particle swarm
> optimization, but it's very approximate. It's not getting as close as
> I want it to so I'm looking to try something new. I'm sure there are
> more exact techniques out there. I was told by a friend there is an
> iterative technique using an auto-correlation matrix that might do
> what I want, but we were talking fast and I didn't follow all that he
> was saying. I did some quick searching and didn't find much. Can
> anyone point me in the right direction? Thanks for the help.
>
> -----Jay


There are many, many methods.  The choice depends on
the noise in your measurements, some knowledge of the
system itself, and the number of samples that you get.

The correlation-based method that was mentioned probably
uses the following property.  Suppose that the input
signal is x[n], the system h[n], and the output y[n].

Let rxx[n] be the autocorrelation of x[n], and
rxy[n] be the cross-correlation of x[n] and y[n].
Then it can be shown that rxy[n] = rxx[n] * h[n].

So if you can estimate rxx and ryx, then you can
do a de-convolution to get h[n].  Further, it can
be shown that ryy[n] = rxx[n] * (h[n] * h[-n]).

Note that in this last one, the phase information
is gone and you can only solve up to h[n]*h[-n].
Since you are interested in FIR estimates, most
of the time people do what is called minimum-phase
decomposition.

Anyways, these are just a bunch of ideas,
extrapolated from your question.

Hope that helps,
Julius

On Feb 20, 12:20 am, "robert bristow-johnson"
<r...@audioimagination.com> wrote:
> On Feb 19, 10:19 pm, horndud...@gmail.com wrote:
>
> > I'm wanting to model a real world system using an FIR filter. I have
> > samples of the input and the expected output. How does one find the
> > filter coefficients from these? I've been using particle swarm
> > optimization, but it's very approximate. It's not getting as close as
> > I want it to so I'm looking to try something new. I'm sure there are
> > more exact techniques out there. I was told by a friend there is an
> > iterative technique using an auto-correlation matrix that might do
> > what I want, but we were talking fast and I didn't follow all that he
> > was saying. I did some quick searching and didn't find much. Can
> > anyone point me in the right direction? Thanks for the help.
>
> isn't this what LMS adaptive filtering is about?  system
> identification?
>
> r b-j

This is one application of an LMS filter. If the system is time-
varying (so the output is nonstationary), then this would be a good
solution to your problem. If the system is stationary and you know the
second-order statistics of the input and output, then you can get
optimal (minimum MSE) deconvolution using a single, fixed Wiener
filter.

Jason

Thanks a lot all. This is a lot of good information for me to look
over for a little while (and learn more vocab). It looks like you've
all suggested I look into:

- AR models (What does this stand for? Is it autoregressive?)
- The autocorrelation method I'd heard about (thanks for the
explanation)
- LMS Filters (I've found some good stuff at wikipedia to get me
started)

Thanks all!

-----Jay

On Feb 20, 8:34 pm, horndud...@gmail.com wrote:
> Thanks a lot all. This is a lot of good information for me to look
> over for a little while (and learn more vocab). It looks like you've
> all suggested I look into:
>
> - AR models (What does this stand for? Is it autoregressive?)
> - The autocorrelation method I'd heard about (thanks for the
> explanation)
> - LMS Filters (I've found some good stuff at wikipedia to get me
> started)
>
> Thanks all!
>
> -----Jay

Yes; AR = autoregressive.

A couple good books on these topics:

"Statistical Digital Signal Processing and Modeling" by Hayes
"Adaptive Filter Theory" by Haykin

Jason

Jay wrote:
> Thanks a lot all. This is a lot of good information for me to look
> over for a little while (and learn more vocab). It looks like you've
> all suggested I look into:
>
> - AR models (What does this stand for? Is it autoregressive?)
> - The autocorrelation method I'd heard about (thanks for the
> explanation)
> - LMS Filters (I've found some good stuff at wikipedia to get me
> started)

Looks like you missed the most relevant technical term: system
identification, which is concerned with the following task:

You are given x and y, assuming that y = h * x + e  (* = convolution),
e is some zero-mean noise process and h the impulse response of an LTI
system. Determine h.

One approach not mentioned yet is via inverse filtering. If you can
choose x (this is not always the case, or sometimes the possible
inputs x are limited), there are much better choices than the obvious
impulse, because you have to somehow get rid of the noise e.

Let's say x was chosen such that there existed an inverse filter x^-1,
with

x * x^-1 = delta_n,

where delta_n is the impulse delayed by n samples. Then h can be found
by convolving y with x^-1. The longer x^-1, the more the noise gets
averaged, and the better your approximation of h.

Also, before you mix everything up: there are two ways to approach
this task. One is to stipulate a model for h. A model is essentially
concerned with data compression. You want to explain the
characteristics of h with as few numbers (called parameters) as
possible. AR modeling is just one of many alternatives.

System ID via Wiener filter (of which LMS is just one of many possible
adaptive variants), cross-correlation methods (in time or frequency
domain) or inverse filtering gives a direct approximation of h. No
data compression. This requires more effort (for the same accuracy),
because you have to estimate many more numbers than for modeling.

Obviously, both methods have their pros and cons. The best choice
mainly depends on what you intend to do use h for.

Regards,
Andor