Kalman Filter for predicing the center of a filter

Rune Allnor wrote:
> Jerry Avins skrev:
>> Peter Nachtwey wrote:
> 
>>> Yes, the trick is getting a good model of your system.  You can use
>>> your 1000 to 5000 data points for that.  What are you trying to model?
>>> What are your data points?
>> And when you get it working on stock market trends, please let us know.
> 
> Did I miss something? Where are the references to the stock market?

Peter is right: the trick is getting a good model. As far as I know, no 
one has yet found a good stock-market model. My comment was to those who 
try.

Jerry
-- 
        "The rights of the best of men are secured only as the
        rights of the vilest and most abhorrent are protected."
            - Chief Justice Charles Evans Hughes, 1927
���������������������������������������������������������������������

Mat wrote:
> Therefore something like Kriging Interpolation or Gaussian Process
> Regression -- which I understand are similar might reduce the
> level of complexity relative to Kalman and produce the same
> end.  I need a way to smooth my values and reduce or acheive
> zero-lag.  It seems clear to me that my variable is doing everything
> I want -- so I do not need to know anything other than what the
> last value would be using a interpolation process that also
> serves a smoothing function. I understand these are related to
> slines.
>
> Am I in the right group for a discussion on these two methods or
> would it be better to start a different post or move to another group.
>
>
> Mat

If you have the model AND the control then you can estimate positions
in the future without lag.   The key thing you mentioned is that you
are now relying on control.   In  my Philips and Nagle (sp?)control
book ( I forget the name of the book right now and it is at work )
there is an example of an observer can predict the state at time n*T
using data at time (n-1)*T.  This is in chapter 9.2 I think.

Both the observer and the Kalman filter rely on having an accurate
model to calculate the A and B matrices.  The real difference only
seems to be that the observer gains are fixed whereas Kalman gains
aren't.

There have been a few threads lately about Kalman filters.   In these
applications the model or system was assumed to keep doing what ever it
was doing before without input or control.   In these cases the Kalman
filter will lag.  The transition matrix A just acts like a system with
a lot of inertia.  

Peter Nachtwey

Peter Nachtwey skrev:
> Both the observer and the Kalman filter rely on having an accurate
> model to calculate the A and B matrices.

That's both the force and the Achilles heel of those types of models.
If you can't formulate a model (or you use one that was derived
under invalid assumptions), you can basically end up with a
very complicated and expensive random number generator.

Rune

You missed nothing as there is no reference to the stock market.

There is only the prediction this might be related.  This is wasted
speculation that would be better discussed under a related topic.

As far as those who try -- many are rewarded with $100 million
plus per year.  They know that they get paid on the curve of
how their model work relative to the competition.  So to say
that no one yet has found a good model -- makes two wrong
assumptions 1) That you know what they are doing 2) That a
good model and one that is more useful relative to the
competition are different animal.

It's generally more useful when those who contribute to a post
stay with subjects they understand and do not distract the issue
based on assumptions.

It's find with me to here no further comments if they are unrelated
to my specific post.

> > Did I miss something? Where are the references to the stock market?
>
> Peter is right: the trick is getting a good model. As far as I know, no
> one has yet found a good stock-market model. My comment was to those who
> try.
>
> Jerry
> --
>         "The rights of the best of men are secured only as the
>         rights of the vilest and most abhorrent are protected."
>             - Chief Justice Charles Evans Hughes, 1927
> =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF

Peter Nachtwey wrote:
> Mat wrote:
> > Therefore something like Kriging Interpolation or Gaussian Process
> > Regression -- which I understand are similar might reduce the
> > level of complexity relative to Kalman and produce the same
> > end.  I need a way to smooth my values and reduce or acheive
> > zero-lag.  It seems clear to me that my variable is doing everything
> > I want -- so I do not need to know anything other than what the
> > last value would be using a interpolation process that also
> > serves a smoothing function. I understand these are related to
> > slines.
> >
> > Am I in the right group for a discussion on these two methods or
> > would it be better to start a different post or move to another group.
> >
> >
> > Mat
>
> If you have the model AND the control then you can estimate positions
> in the future without lag.   The key thing you mentioned is that you
> are now relying on control.   In  my Philips and Nagle (sp?)control
> book ( I forget the name of the book right now and it is at work )
> there is an example of an observer can predict the state at time n*T
> using data at time (n-1)*T.  This is in chapter 9.2 I think.

It appears that Kriging Interpolation or Gaussian Process Regression
are very similar and smooth data using information about how the
points are statistically deviation over the period of interest.  I feel
either of these two processes might be more suitable for my purpose
since the end results would be a set of smoothed value via
interpolation.

Does anyone has any practical experience with either of these methods?

Mat

> Both the observer and the Kalman filter rely on having an accurate
> model to calculate the A and B matrices.  The real difference only
> seems to be that the observer gains are fixed whereas Kalman gains
> aren't.
>
> There have been a few threads lately about Kalman filters.   In these
> applications the model or system was assumed to keep doing what ever it
> was doing before without input or control.   In these cases the Kalman
> filter will lag.  The transition matrix A just acts like a system with
> a lot of inertia.  
> 
> Peter Nachtwey

Mat wrote:
> You missed nothing as there is no reference to the stock market.

I'm sorry that my attempt at humor derailed the discussion.

Jerry
-- 
        "The rights of the best of men are secured only as the
        rights of the vilest and most abhorrent are protected."
            - Chief Justice Charles Evans Hughes, 1927
���������������������������������������������������������������������

Jerry Avins wrote:
> Mat wrote:
> > You missed nothing as there is no reference to the stock market.

No problem -- I'm the loser here because you probably already know
what I was trying to find out.  I was just making an attempt to get
the discussion back where it might be possible to gain information.
I also off the original subject discussing interpolation in the same
post -- so you will not be allowed to accept all the blame on derailing
the Kalman discussion.

I'm just a vistor to comp.dsp and certainly don't want to come across
as rude or without a sense of humor.

Mat

> I'm sorry that my attempt at humor derailed the discussion.
>
> Jerry
> --
>         "The rights of the best of men are secured only as the
>         rights of the vilest and most abhorrent are protected."
>             - Chief Justice Charles Evans Hughes, 1927
> =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF

Mat wrote:
> [SNIP]
> 
> I'm just a vistor to comp.dsp and certainly don't want to come across
> as rude or without a sense of humor.
> 

comp.dsp is difficult to offend accidentally.

It can however have a strange sense of humor.
Having opened my mouth, I have (on occasion) been handed my foot 
somewhat past my clavicle ;)

I'm a raw neophyte, but I wonder.

If I understand correctly:
  1. You have a very good, but not perfect, model of your system.
  2. You have a "noisy" signal.
  3. You wish a "good" prediction of the next value(s) from the noisy
     source so that you can apply a suitable correction.
  4. *THE GOTCHA* You have an appropriately sensitive detector of mismatch
     between "predicted" and "actual" value to signal when you should
     change compensating algorithm.

I'm not attempting to claim "solution" but only "better" definition.


A side question to group
"Am I thinking in useful directions for solving my own problems?"

Richard Owlett wrote:
> Mat wrote:
> > [SNIP]
> >
> > I'm just a vistor to comp.dsp and certainly don't want to come across
> > as rude or without a sense of humor.
> >
>
> comp.dsp is difficult to offend accidentally.
>
> It can however have a strange sense of humor.
> Having opened my mouth, I have (on occasion) been handed my foot
> somewhat past my clavicle ;)
>
> I'm a raw neophyte, but I wonder.
>
> If I understand correctly:
>   1. You have a very good, but not perfect, model of your system.
>   2. You have a "noisy" signal.

I want a smoothed estimate of the current raw value relative to my
filter window. Extrapolation of an unknown next signal is "not
required".

>   3. You wish a "good" prediction of the next value(s) from the noisy
>      source so that you can apply a suitable correction.

I'm looking for an interpolation method and/or a suitable prediction
method than can estimate what the "known population" within
my raw data would be in a smoothed format. I suspect this
reduces to interpolation of all known raw values in a filter window
to smoothed values. It appears that kriging and Gaussian
Process Regression are frequently used for this process but
there might be other interpolation methods that work just as well.

Does anyone doing interpolation or something similiar to what I've
described know of models that might be suitable for this purpose?

Mat

>   4. *THE GOTCHA* You have an appropriately sensitive detector of mismatch
>      between "predicted" and "actual" value to signal when you should
>      change compensating algorithm.
>
> I'm not attempting to claim "solution" but only "better" definition.
>
>
> A side question to group
> "Am I thinking in useful directions for solving my own problems?"

Mat wrote:

> 
> I want a smoothed estimate of the current raw value relative to my
> filter window. Extrapolation of an unknown next signal is "not
> required".
> 

I don't understand. You are asking for a algorithm that would predict
the output of a filter? You apparently have the data and you have the
filter. Where's the problem?

-jim


> >   3. You wish a "good" prediction of the next value(s) from the noisy
> >      source so that you can apply a suitable correction.
> 
> I'm looking for an interpolation method and/or a suitable prediction
> method than can estimate what the "known population" within
> my raw data would be in a smoothed format. I suspect this
> reduces to interpolation of all known raw values in a filter window
> to smoothed values. It appears that kriging and Gaussian
> Process Regression are frequently used for this process but
> there might be other interpolation methods that work just as well.
> 
> Does anyone doing interpolation or something similiar to what I've
> described know of models that might be suitable for this purpose?
> 
> Mat
> 
> >   4. *THE GOTCHA* You have an appropriately sensitive detector of mismatch
> >      between "predicted" and "actual" value to signal when you should
> >      change compensating algorithm.
> >
> > I'm not attempting to claim "solution" but only "better" definition.
> >
> >
> > A side question to group
> > "Am I thinking in useful directions for solving my own problems?"

----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----