X-N-Archive:yes I'm interested in methods useful for predicting the center of a simple moving average with window length of 20 to 40 periods. If a Kalman filter or some other dsp method is not useful for this what other math or statistical related newgroup might know? I'm looking for sources, referrences or someone who actually used an algorithm that is helpful in predicting the smoothed signal from raw data 20 or more steps ahead. Any ideas, Mat
Kalman Filter for predicing the center of a filter
Started by ●October 5, 2006
Reply by ●October 6, 20062006-10-06
Mat wrote:> X-N-Archive:yes > > I'm interested in methods useful for predicting the center of a > simple moving average with window length of 20 to 40 periods. > > If a Kalman filter or some other dsp method is not useful for this > what other math or statistical related newgroup might know? > > I'm looking for sources, referrences or someone who actually > used an algorithm that is helpful in predicting the smoothed > signal from raw data 20 or more steps ahead.I think you had better explain what you mean by finding the center of a filter. The center of a five-tap filter is the third (twoth, counting from zero) tap. Jerry -- "The rights of the best of men are secured only as the rights of the vilest and most abhorrent are protected." - Chief Justice Charles Evans Hughes, 1927 ���������������������������������������������������������������������
Reply by ●October 6, 20062006-10-06
Jerry Avins wrote:> Mat wrote:X-N-Archive:yes> > > > I'm interested in methods useful for predicting the center of a > > simple moving average with window length of 20 to 40 periods. > > > > If a Kalman filter or some other dsp method is not useful for this > > what other math or statistical related newgroup might know? > > > > I'm looking for sources, referrences or someone who actually > > used an algorithm that is helpful in predicting the smoothed > > signal from raw data 20 or more steps ahead.It may have been better to say that I want to use my raw data to predict the smoothed trend without any lag. So I would like to know if I could use something like a Kalman filter to develop a model to do this on say 1000 to 5000 data points which would have enough predictive powers to predict the smoothed values using my raw data. I know prediction issues are often dealt with from the statistical viewpoint using regression and similar models. Can Kalman or some other dsp algorithm be trained on raw data to predict filtered values with zero lag? Mat> I think you had better explain what you mean by finding the center of a > filter. The center of a five-tap filter is the third (twoth, counting > from zero) tap. > > Jerry > -- > "The rights of the best of men are secured only as the > rights of the vilest and most abhorrent are protected." > - Chief Justice Charles Evans Hughes, 1927 > =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF==AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF= =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF
Reply by ●October 6, 20062006-10-06
Mat wrote:> It may have been better to say that I want to use my raw data to > predict the smoothed trend without any lag. So I would like to > know if I could use something like a Kalman filter to develop > a model to do this on say 1000 to 5000 data points which > would have enough predictive powers to predict the smoothed > values using my raw data. I know prediction issues are often > dealt with from the statistical viewpoint using regression and > similar models. > > Can Kalman or some other dsp algorithm be trained on > raw data to predict filtered values with zero lag?Yes, the trick is getting a good model of your system. You can use your 1000 to 5000 data points for that. What are you trying to model? What are your data points? Peter Nachtwey =20 =AF=AF=AF=AF=AF=AF=AF=AF
Reply by ●October 6, 20062006-10-06
Peter Nachtwey wrote:> Mat wrote: >> It may have been better to say that I want to use my raw data to >> predict the smoothed trend without any lag. So I would like to >> know if I could use something like a Kalman filter to develop >> a model to do this on say 1000 to 5000 data points which >> would have enough predictive powers to predict the smoothed >> values using my raw data. I know prediction issues are often >> dealt with from the statistical viewpoint using regression and >> similar models. >> >> Can Kalman or some other dsp algorithm be trained on >> raw data to predict filtered values with zero lag? > > Yes, the trick is getting a good model of your system. You can use > your 1000 to 5000 data points for that. What are you trying to model? > What are your data points?And when you get it working on stock market trends, please let us know. Jerry -- "The rights of the best of men are secured only as the rights of the vilest and most abhorrent are protected." - Chief Justice Charles Evans Hughes, 1927 ���������������������������������������������������������������������
Reply by ●October 7, 20062006-10-07
Jerry Avins wrote:> And when you get it working on stock market trends, please let us know.I am not joking. The market is emotional. Machinery isn't. Search for system identification or buy a control book that covers systems identification. A good auto tuning program has all of this worked out. Peter Nachtwey
Reply by ●October 7, 20062006-10-07
Jerry Avins skrev:> Peter Nachtwey wrote:> > Yes, the trick is getting a good model of your system. You can use > > your 1000 to 5000 data points for that. What are you trying to model? > > What are your data points? > > And when you get it working on stock market trends, please let us know.Did I miss something? Where are the references to the stock market? Rune
Reply by ●October 7, 20062006-10-07
Rune Allnor wrote:> Jerry Avins skrev: > >>Peter Nachtwey wrote: > > >>>Yes, the trick is getting a good model of your system. You can use >>>your 1000 to 5000 data points for that. What are you trying to model? >>>What are your data points? >> >>And when you get it working on stock market trends, please let us know. > > > Did I miss something? Where are the references to the stock market? > > RuneI'm not sure why Jerry said that, since the original poster doesn't indicate he is trying to achieve the unachievable. However, I think Jerry is referring to the basic problem of any state estimation predictor - it can only predict the predictable, and for many applications that is too dull to be interesting. For example, in sonar or radar Kalman can be used to get a great estimation of what a target will do next, based on what it has been doing. Its an excellent way to sustain automatic tracking from scan to scan for steadily moving targets. As soon as a target does something interesting though - a major manoevure - sustaining tracks based on Kalman estimation falls apart. Then, you have to resort to wonderful algorithms that grope around in the dark, trying to figure out what the damn target might possibly have just done. :-) Steve
Reply by ●October 7, 20062006-10-07
> >>>Yes, the trick is getting a good model of your system. You can use > >>>your 1000 to 5000 data points for that. What are you trying to model? > >>>What are your data points?Interesting comments about application usage.> >>And when you get it working on stock market trends, please let us know. > > > > > > Did I miss something? Where are the references to the stock market? > > > > RuneProbably because novices are always looking for an easy way to make money without a clue of non-stationary process, randomness or many other obstacles that render most models obsolete. Let me shed more light on the nature of what I would like to do without any discussion of my application which is more likely to lead to other assumptions or suggestions that would not be useful to me or anyone else. I know my data and environment extremely well. I've been in my field for over 25 years. I've always found another way around smoothing because lag is detrimental to my application since my data is time sensitive.> I'm not sure why Jerry said that, since the original poster doesn't > indicate he is trying to achieve the unachievable. However, I think > Jerry is referring to the basic problem of any state estimation > predictor - it can only predict the predictable, and for many > applications that is too dull to be interesting.Steve, here you go straight to the point of any prediction system in a stochastic environment. Fortunately each of my new values makes an adjustment that indicates whether a manoevure will render it useless. In those situations I wait without risk of the signal being inaccurate. While there are levels between usefulness and useless my application so far relies 100% on control. Doing a google search prediction of a smoothed filter has proved useful. Therefore something like Kriging Interpolation or Gaussian Process Regression -- which I understand are similar might reduce the level of complexity relative to Kalman and produce the same end. I need a way to smooth my values and reduce or acheive zero-lag. It seems clear to me that my variable is doing everything I want -- so I do not need to know anything other than what the last value would be using a interpolation process that also serves a smoothing function. I understand these are related to slines. Am I in the right group for a discussion on these two methods or would it be better to start a different post or move to another group. Mat> For example, in sonar or radar Kalman can be used to get a great > estimation of what a target will do next, based on what it has been > doing. Its an excellent way to sustain automatic tracking from scan to > scan for steadily moving targets. As soon as a target does something > interesting though - a major manoevure - sustaining tracks based on > Kalman estimation falls apart. Then, you have to resort to wonderful > algorithms that grope around in the dark, trying to figure out what the > damn target might possibly have just done. :-) > > Steve
Reply by ●October 7, 20062006-10-07
Peter Nachtwey wrote:> Jerry Avins wrote: >> And when you get it working on stock market trends, please let us know. > > I am not joking. The market is emotional. Machinery isn't. Search > for system identification or buy a control book that covers systems > identification. A good auto tuning program has all of this worked out.I know that. I'm amused that so many people apply statistics to market prediction. You and I can use statistics to prove that some of them will appear (for a while) to have a system that "works", just like some casino buffs. Jerry -- "The rights of the best of men are secured only as the rights of the vilest and most abhorrent are protected." - Chief Justice Charles Evans Hughes, 1927 ���������������������������������������������������������������������