Kalman filtering...

I am looking for Matlab codes for using Kalman filters to estimate unknown 
parameters, which, I hope, can be an alternative to the maximum likelihood 
estimate of the parameters...

How to do that? And could anybody point me to some samples, tutorials, 
resources, examples, Matlab codes?

Thanks a lot! 

gino wrote:
> I am looking for Matlab codes for using Kalman filters to estimate unknown
> parameters, which, I hope, can be an alternative to the maximum likelihood
> estimate of the parameters...

For many models (in particular those with normal errors) where the
state is the parameter vector or a function of it, the KF will
generally give the MLE.

The KF is nothing more than a way of estimating the mean & variance of
the state in a SSM.

Which model did you have in mind?

Why do you want something other than ML?

Glen

<glenbarnett@geocities.com> wrote in message 
news:1151039403.918516.72800@p79g2000cwp.googlegroups.com...
>
> gino wrote:
>> I am looking for Matlab codes for using Kalman filters to estimate 
>> unknown
>> parameters, which, I hope, can be an alternative to the maximum 
>> likelihood
>> estimate of the parameters...
>
> For many models (in particular those with normal errors) where the
> state is the parameter vector or a function of it, the KF will
> generally give the MLE.
>
> The KF is nothing more than a way of estimating the mean & variance of
> the state in a SSM.
>
> Which model did you have in mind?
>
> Why do you want something other than ML?
>
> Glen
>

Because MLE gives a lot of local minimum, and depending on the initial 
values, the optimum points jump around; it is really hard to harness. And in 
our optimization procedure using Matlab's "fmincon", some unwanted 
points(often boundary) give -infinity, which became our global minimum, 
which is definitely undesirable...

We'd like to try out other alternatives other than MLE. From data we don't 
neccessarily know if they are with normal errors or not... thus do you still 
have theoretical results showing that KF is exactly the same as MLE?

Dear Gino,

some years before I wrote a short artikle (20pp) that covers exactly
what you are asking for. It includes a MatLab source as well as a
couple of references and a tutorial introduction. Unfortunately I
cannot post this to the Usenet groups. Please ccontact me by email (see
above) directly - I will forward this article a *.pdf document.

Best

Christian

gino wrote:

> I am looking for Matlab codes for using Kalman filters to estimate unknown
> parameters, which, I hope, can be an alternative to the maximum likelihood
> estimate of the parameters...
>
> How to do that? And could anybody point me to some samples, tutorials,
> resources, examples, Matlab codes?
> 
> Thanks a lot!

HOPE YOU FINE THIS LINKS FOR KALMAN FILER USEFULL..

http://www.mathworks.com/matlabcentral/fileexchange/loadAuthor.do?objectType=author&objectId=1095153

http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=11144&objectType=file

gino wrote:
> Because MLE gives a lot of local minimum, and depending on the initial
> values, the optimum points jump around; it is really hard to harness. And in
> our optimization procedure using Matlab's "fmincon", some unwanted
> points(often boundary) give -infinity, which became our global minimum,
> which is definitely undesirable...
>
> We'd like to try out other alternatives other than MLE. From data we don't
> neccessarily know if they are with normal errors or not... thus do you still
> have theoretical results showing that KF is exactly the same as MLE?

The KF estimates the mean and variance of a state space model; the
usual SSM has normal errors.

If the actual likelihood of your model is badly behaved, the KF won't
solve that.

If the model is fine and it's a problem with MATLAB's optimization,
then many things may work.

What's the model?

Glen

"Christian Langen" <clangen@gmx.de> wrote in message 
news:1151043795.583400.100610@p79g2000cwp.googlegroups.com...
> Dear Gino,
>
> some years before I wrote a short artikle (20pp) that covers exactly
> what you are asking for. It includes a MatLab source as well as a
> couple of references and a tutorial introduction. Unfortunately I
> cannot post this to the Usenet groups. Please ccontact me by email (see
> above) directly - I will forward this article a *.pdf document.
>
> Best
>
> Christian
>
> gino wrote:
>
>> I am looking for Matlab codes for using Kalman filters to estimate 
>> unknown
>> parameters, which, I hope, can be an alternative to the maximum 
>> likelihood
>> estimate of the parameters...
>>
>> How to do that? And could anybody point me to some samples, tutorials,
>> resources, examples, Matlab codes?
>>
>> Thanks a lot!
>


Wow, that's great! Did you mean that you have code for doing KF as an 
alternative to MLE?

I sent you an email already! thanks so much! 

<glenbarnett@geocities.com> wrote in message 
news:1151254884.109217.317140@m73g2000cwd.googlegroups.com...
>
> gino wrote:
>> Because MLE gives a lot of local minimum, and depending on the initial
>> values, the optimum points jump around; it is really hard to harness. And 
>> in
>> our optimization procedure using Matlab's "fmincon", some unwanted
>> points(often boundary) give -infinity, which became our global minimum,
>> which is definitely undesirable...
>>
>> We'd like to try out other alternatives other than MLE. From data we 
>> don't
>> neccessarily know if they are with normal errors or not... thus do you 
>> still
>> have theoretical results showing that KF is exactly the same as MLE?
>
> The KF estimates the mean and variance of a state space model; the
> usual SSM has normal errors.
>
> If the actual likelihood of your model is badly behaved, the KF won't
> solve that.
>
> If the model is fine and it's a problem with MATLAB's optimization,
> then many things may work.
>
> What's the model?
>
> Glen
>

Thanks Glen.

The model is of typical loglikelihood function:

f(mu, sigma, a, b)=-0.5*sum_of_squares( (x - mu)/sigma) + log(p1) - log(p0).

The log(p1) and log(p0) terms are the logs of some probabilities.

Theoretically all the above functions are continuous and well-behaved.

However, when fitting this model to data, due to model error or data noise, 
the numerical results might not be always well-behaved.

Sigma being near 0 is a huge problem, although I've demanded the low bound 
of sigma to be 1e-5.

p1 and p0 can be evaluated to be 0 or negative too. Although theoretically a 
probability cannot be outside [0, 1], but when fitting it to data, it can 
give negative values and values that are greater than 1, depending on the 
data.

Thus this 4D log likelihood function is hard to maximize using data. I am 
trying many solvers.

I'd be interested in trying KF out, as an alternative to MLE. But you said 
KF is the same as MLE?

Thanks a lot!

"PARTICLEREDDY" <particlereddy@gmail.com> wrote in message 
news:1151047995.062889.139270@u72g2000cwu.googlegroups.com...
> HOPE YOU FINE THIS LINKS FOR KALMAN FILER USEFULL..
>
> http://www.mathworks.com/matlabcentral/fileexchange/loadAuthor.do?objectType=author&objectId=1095153
>
> http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=11144&objectType=file
>

I look a look, but they are for communications, not for my purpose, I want 
to use Kalman Filter as an alternative to MLE in my parameter estimation 
problem for fitting data. I am confused about how to cast MLE problem into 
KF framework...

gino wrote:
> The model is of typical loglikelihood function:
>
> f(mu, sigma, a, b)=-0.5*sum_of_squares( (x - mu)/sigma) + log(p1) - log(p0).
>
> The log(p1) and log(p0) terms are the logs of some probabilities.

That's not the model! What's the distribution for x?

What are p1 and p0?

The likelihood looks like some sort of Gaussian but I'm not clear on
what p1 and p0 represent then.

> p1 and p0 can be evaluated to be 0 or negative too. Although theoretically a
> probability cannot be outside [0, 1], but when fitting it to data, it can
> give negative values and values that are greater than 1, depending on the
> data.

It's possible to avoid this by suitable reparameterization (write your
p's in terms of - say - their logits, which are unrestricted).

> Thus this 4D log likelihood function is hard to maximize using data. I am
> trying many solvers.
>
> I'd be interested in trying KF out, as an alternative to MLE. But you said
> KF is the same as MLE?

The usual State Space model (SSM) will have Gaussian errors. The KF is
just a way to get efficient esitmates of the mean and variance of the
state (which presumably in your case will consist of the parameters),
given the data. It isn't clear to me that your model for x (though I'm
still not certain what the model is) can even be written in state space
form.

Can you describe the situation more?

Glen