Reply by June 18, 20072007-06-18
On Jun 17, 7:35 pm, John_W_Her...@yahoo.com (John Herman) wrote:
> If you are looking for Levenburg-Marquardt, I would suggest levmar. > > http://www.ics.forth.gr/~lourakis/levmar/
I tried the version in the link above. It wasn't as robust as the lmfit.c version. When starting out the sum of squared errors can be very big and cause floaiting point overflow problems. This occured in our application when there are 2000+ data points. lmfit.c didn't have this problem as it had extra code to check the size of the squared error and kept a couple accumulators to keep them from over flowing. Finding this problem is how I learned what the algorithm was doing. I recommend https://sourceforge.net/projects/lmfit/ The code is ugly c code converted from fortran but it works well and is easy to implement. If there are other versions that work equally well then it would be good to hear. This post has saved the world 100s if not 1000s of man hours debugging code. I don't recommend writing this from scratch if you don't have too. One's numerical methods and programming techniques better be good. That is why a bypassed suggesting the books and directed Tom directly to the code. Peter Nachtwey
Reply by John Herman June 17, 20072007-06-17
If you are looking for Levenburg-Marquardt, I would suggest levmar.

http://www.ics.forth.gr/~lourakis/levmar/

You will need the BLAS 1 and LAPACK libraries.  I think the code from ATLAS is 
the best choice.

In article <1182034526.765823.116870@x35g2000prf.googlegroups.com>, 
pnachtwey@gmail.com wrote:
>On Jun 13, 10:53 am, "Tom" <tomda...@yahoo.com> wrote: >> I am looking for an algorithm to model a system using an ARMA filter. The >> input and the desired output are available. Is there an iterative algorithm >> that can give a global optimum solution ? Any good reference ? >> >> Regards >> >> Tom > >You could be more clear about what you want. A quick solution or in >depth knowledge. I use a minimizing or optimizing algorithm. These >exist in Scilab as lsqrsolve and optim. In Mathcad I use Minerr. In >C I use lmfit.c which is c code converted from ancient fortran but it >works. There are other versions out there that are buggy. > >If iterative means on-the-fly there is the recursive least squares >method but I would be careful with this. In my testing, a fast sample >rate yields poles close to the unit circle. The poles move around a >bit while updating and sometimes the poles would cross the unit >circle. Not good. > >What ever method you chose you must make a decision as to how many >coefficients you need. One can always get a smaller norm or mean >squared error by adding more coefficients but often the poles in the z >domain end up on the negative real axis which is an indicator that you >probably have too many coefficients. I don't use the recursive least >squares but I do find it interesting how the poles move around and by >how much. > >I read about the Levenberg-Marquardt algorithm in Numerical Recipes in >C but didn't like the code or the restrictions. After that I found >tracing through lmfit.c code helpful. I didn't need anything beyond >this. > >Peter Nachtwey >
Reply by June 16, 20072007-06-16
On Jun 13, 10:53 am, "Tom" <tomda...@yahoo.com> wrote:
> I am looking for an algorithm to model a system using an ARMA filter. The > input and the desired output are available. Is there an iterative algorithm > that can give a global optimum solution ? Any good reference ? > > Regards > > Tom
You could be more clear about what you want. A quick solution or in depth knowledge. I use a minimizing or optimizing algorithm. These exist in Scilab as lsqrsolve and optim. In Mathcad I use Minerr. In C I use lmfit.c which is c code converted from ancient fortran but it works. There are other versions out there that are buggy. If iterative means on-the-fly there is the recursive least squares method but I would be careful with this. In my testing, a fast sample rate yields poles close to the unit circle. The poles move around a bit while updating and sometimes the poles would cross the unit circle. Not good. What ever method you chose you must make a decision as to how many coefficients you need. One can always get a smaller norm or mean squared error by adding more coefficients but often the poles in the z domain end up on the negative real axis which is an indicator that you probably have too many coefficients. I don't use the recursive least squares but I do find it interesting how the poles move around and by how much. I read about the Levenberg-Marquardt algorithm in Numerical Recipes in C but didn't like the code or the restrictions. After that I found tracing through lmfit.c code helpful. I didn't need anything beyond this. Peter Nachtwey
Reply by June 16, 20072007-06-16
On Jun 14, 6:19 am, BERT <callm...@gmail.com> wrote:
> On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote: > > > I am looking for an algorithm to model a system using an ARMA filter. The > > input and the desired output are available. Is there an iterative algorithm > > that can give a global optimum solution ? Any good reference ? > > > Regards > > > Tom > > Simon Haykin, "Adaptive Filter Theory"
I don't think that book will help him much. A better book is Ljung and Sodestrom Theory and Practice of Recursive Estimation, MIT Press 1983.
Reply by June 13, 20072007-06-13
On Jun 13, 2:19 pm, BERT <callm...@gmail.com> wrote:
> On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote: > > > I am looking for an algorithm to model a system using an ARMA filter. The > > input and the desired output are available. Is there an iterative algorithm > > that can give a global optimum solution ? Any good reference ? > > > Regards > > > Tom > > Simon Haykin, "Adaptive Filter Theory"
I wouldn't recommend that book for ARMA modeling; there isn't much any information on that technique in the text. Haykin (rightfully) points out that ARMA modeling is in general a nonlinear problem, which is much more difficult to produce than a model with a simpler AR structure. On the other hand, it is, IMO, a great adaptive filtering text. Jason
Reply by John June 13, 20072007-06-13
Any optimization criteria is OK as long as  the solution is optimal 
(unique). In addition samples are available, iterative or non-iterative both 
are ok.

Thanks

Tom
"BERT" <callmevc@gmail.com> wrote in message 
news:1181758907.555174.71980@i38g2000prf.googlegroups.com...
> On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote: >> I am looking for an algorithm to model a system using an ARMA filter. The >> input and the desired output are available. Is there an iterative >> algorithm >> that can give a global optimum solution ? Any good reference ? >> >> Regards >> >> Tom > > > And oh, it might be useful to define what you mean by a "global > optimum solution" (a.k.a, your optimization criteria). >
Reply by Eric Jacobsen June 13, 20072007-06-13
On Wed, 13 Jun 2007 18:56:26 -0000, julius <juliusk@gmail.com> wrote:

>On Jun 13, 1:21 pm, BERT <callm...@gmail.com> wrote: >> On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote: >> >> > I am looking for an algorithm to model a system using an ARMA filter. The >> > input and the desired output are available. Is there an iterative algorithm >> > that can give a global optimum solution ? Any good reference ? >> >> > Regards >> >> > Tom >> >> And oh, it might be useful to define what you mean by a "global >> optimum solution" (a.k.a, your optimization criteria). > >And if the original poster defines what he means by "iterative". >Is it iterative in the samples received, or in terms of solving >what this "global optimum solution" is?
Yeah, Parks-McLellan is iterative, but it only takes care of the "MA" part of ARMA. Eric Jacobsen Minister of Algorithms Abineau Communications http://www.ericjacobsen.org
Reply by julius June 13, 20072007-06-13
On Jun 13, 1:21 pm, BERT <callm...@gmail.com> wrote:
> On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote: > > > I am looking for an algorithm to model a system using an ARMA filter. The > > input and the desired output are available. Is there an iterative algorithm > > that can give a global optimum solution ? Any good reference ? > > > Regards > > > Tom > > And oh, it might be useful to define what you mean by a "global > optimum solution" (a.k.a, your optimization criteria).
And if the original poster defines what he means by "iterative". Is it iterative in the samples received, or in terms of solving what this "global optimum solution" is?
Reply by BERT June 13, 20072007-06-13
On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote:
> I am looking for an algorithm to model a system using an ARMA filter. The > input and the desired output are available. Is there an iterative algorithm > that can give a global optimum solution ? Any good reference ? > > Regards > > Tom
And oh, it might be useful to define what you mean by a "global optimum solution" (a.k.a, your optimization criteria).
Reply by BERT June 13, 20072007-06-13
On Jun 13, 1:53 pm, "Tom" <tomda...@yahoo.com> wrote:
> I am looking for an algorithm to model a system using an ARMA filter. The > input and the desired output are available. Is there an iterative algorithm > that can give a global optimum solution ? Any good reference ? > > Regards > > Tom
Simon Haykin, "Adaptive Filter Theory"