Prediction

I need to predict the future value of a signal. I dont know anything about
the signal other than it looks like bandpass filtered white noise. I need
to predict the value say 10 samples ahead (I guess this is called a
ten-step predictor???). Does anyone has a good suggestion on how to do
this? Linear prediction? Adaptive filtering or system identification
techniques? 

sorenbirk wrote:

> I need to predict the future value of a signal. I dont know anything about
> the signal other than it looks like bandpass filtered white noise.

If you don't know anything about the signal, then you can't predict it.


  I need
> to predict the value say 10 samples ahead (I guess this is called a
> ten-step predictor???). Does anyone has a good suggestion on how to do
> this?

Tossing a coin.

  Linear prediction? Adaptive filtering or system identification
> techniques? 

First, you need to figure out what is the signal. I.e. is there any 
statistical dependency of the future from the past.

Vladimir Vassilevsky

DSP and Mixed Signal Design Consultant

http://www.abvolt.com

sorenbirk wrote:
> I need to predict the future value of a signal. I dont know anything about
> the signal other than it looks like bandpass filtered white noise. I need
> to predict the value say 10 samples ahead (I guess this is called a
> ten-step predictor???). Does anyone has a good suggestion on how to do
> this? Linear prediction? Adaptive filtering or system identification
> techniques? 

If it truly looks like white noise being filtered by a linear bandpass 
filter, then you would need to model the filter and design a predictor 
around it.  Assuming the bandpass characteristics are known and you want 
to reduce the mean-squared error then a Kalman filter is the way to go.

If the bandpass characteristics can't be known from run to run then 
you'll need to do some sort of on-line system identification followed by 
the Kalman or perhaps an H-infinity filter to account for imperfect 
knowledge of the filter coefficients.

If the pass band is 1/10th your sample rate or wider then there's not 
going to be much correlation between your filter's predictions and your 
signal, no matter what you do.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

Tim Wescott wrote:

>> I need to predict the future value of a signal. I dont know anything 
>> about
>> the signal other than it looks like bandpass filtered white noise.

> If it truly looks like white noise being filtered by a linear bandpass 
> filter, then you would need to model the filter and design a predictor 
> around it.  Assuming the bandpass characteristics are known and you want 
> to reduce the mean-squared error then a Kalman filter is the way to go.

Unfortunately this applies only to the linear systems.

> 
> If the bandpass characteristics can't be known from run to run then 
> you'll need to do some sort of on-line system identification followed by 
> the Kalman or perhaps an H-infinity filter to account for imperfect 
> knowledge of the filter coefficients.

There are many methods for linear systems, however not much can be done 
for the nonlinear. The task is unsoluable in the general.

> 
> If the pass band is 1/10th your sample rate or wider then there's not 
> going to be much correlation between your filter's predictions and your 
> signal, no matter what you do.

To do any predictions, one must have the parametric model of the system.
Then the parameters of the model can be determined. If there is no 
model, the best prediction is by tossing the coin.

Vladimir Vassilevsky

DSP and Mixed Signal Design Consultant

http://www.abvolt.com

Vladimir Vassilevsky wrote:
> 
> 
> Tim Wescott wrote:
> 
>>> I need to predict the future value of a signal. I dont know anything 
>>> about
>>> the signal other than it looks like bandpass filtered white noise.
> 
> 
>> If it truly looks like white noise being filtered by a linear bandpass 
>> filter, then you would need to model the filter and design a predictor 
>> around it.  Assuming the bandpass characteristics are known and you 
>> want to reduce the mean-squared error then a Kalman filter is the way 
>> to go.
> 
> 
> Unfortunately this applies only to the linear systems.
> 
That's why I stipulated linear.
>>
>> If the bandpass characteristics can't be known from run to run then 
>> you'll need to do some sort of on-line system identification followed 
>> by the Kalman or perhaps an H-infinity filter to account for imperfect 
>> knowledge of the filter coefficients.
> 
> 
> There are many methods for linear systems, however not much can be done 
> for the nonlinear. The task is unsoluable in the general.
> 
Insoluble in general, but often solvable in specific cases. 
Unfortunately, one has to evaluate each case individually, then often 
one must do a bunch of theoretical work just to find out if a solution 
is possible, much less likely.
>>
>> If the pass band is 1/10th your sample rate or wider then there's not 
>> going to be much correlation between your filter's predictions and 
>> your signal, no matter what you do.
> 
> 
> To do any predictions, one must have the parametric model of the system.
> Then the parameters of the model can be determined. If there is no 
> model, the best prediction is by tossing the coin.
>  
Yes yes and yes.  If the OP can't fit this into a linear model, and if 
his bandwidth isn't narrow enough, the best he can do is run away screaming.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

Tim Wescott wrote:

>>>> I need to predict the future value of a signal. I dont know anything 
>>>> about
>>>> the signal other than it looks like bandpass filtered white noise.

> Yes yes and yes.  If the OP can't fit this into a linear model, and if 
> his bandwidth isn't narrow enough, the best he can do is run away 
> screaming.

Although it looks to me that the original question has something to do 
with the stock exchange...

VLV

>sorenbirk wrote:
>> I need to predict the future value of a signal. I dont know anything
about
>> the signal other than it looks like bandpass filtered white noise. I
need
>> to predict the value say 10 samples ahead (I guess this is called a
>> ten-step predictor???). Does anyone has a good suggestion on how to do
>> this? Linear prediction? Adaptive filtering or system identification
>> techniques? 
>
>If it truly looks like white noise being filtered by a linear bandpass 
>filter, then you would need to model the filter and design a predictor 
>around it.  Assuming the bandpass characteristics are known and you want

>to reduce the mean-squared error then a Kalman filter is the way to go.
>
>If the bandpass characteristics can't be known from run to run then 
>you'll need to do some sort of on-line system identification followed by

>the Kalman or perhaps an H-infinity filter to account for imperfect 
>knowledge of the filter coefficients.
>

Just to make sure that I got it rigth: Assuming that the signal can be
modelled as white noise filtered by a linear filter, then I need to
perform some on-line or off-line system id, depending on the signal, and
then the Kalman filter performs the prediction. Or do I need the Kalman
filter to correct the coefficients found in the system id and then use a
predictor on the output from the Kalman filter? 


>If the pass band is 1/10th your sample rate or wider then there's not 
>going to be much correlation between your filter's predictions and your 
>signal, no matter what you do.

Meaning that my sample rate must be more than 10 times the pass band? Is
this some "golden rule" or is it based on theory(probably!). 

Do you have any experience on predicting the value of a signal 10 samples
ahead? Can one get some good results or is it not possible in practice?

Soren 

Tim Wescott <tim@seemywebsite.com> wrote in
news:Jd6dnYXhrfD-QvjYnZ2dnUVZ_umdnZ2d@web-ster.com: 

> sorenbirk wrote:
>> I need to predict the future value of a signal. I dont know anything
>> about the signal other than it looks like bandpass filtered white
>> noise. I need to predict the value say 10 samples ahead (I guess this
>> is called a ten-step predictor???). Does anyone has a good suggestion
>> on how to do this? Linear prediction? Adaptive filtering or system
>> identification techniques? 
> 
> If it truly looks like white noise being filtered by a linear bandpass
> filter, then you would need to model the filter and design a predictor
> around it.  Assuming the bandpass characteristics are known and you
> want to reduce the mean-squared error then a Kalman filter is the way
> to go. 
> 
> If the bandpass characteristics can't be known from run to run then 
> you'll need to do some sort of on-line system identification followed
> by the Kalman or perhaps an H-infinity filter to account for imperfect
> knowledge of the filter coefficients.
> 
> If the pass band is 1/10th your sample rate or wider then there's not 
> going to be much correlation between your filter's predictions and
> your signal, no matter what you do.
> 

Sounds like an awful long way to go to produce ten identical step-ahead 
samples that are the mean of the signal.  Unless I'm missing something 
obvious, isn't that the optimal solution for the ID of noise?  It's not a 
system ID problem, as you don't know the input.

-- 
Scott
Reverse name to reply

sorenbirk wrote:
>>sorenbirk wrote:
>>
>>>I need to predict the future value of a signal. I dont know anything
> 
> about
> 
>>>the signal other than it looks like bandpass filtered white noise. I
> 
> need
> 
>>>to predict the value say 10 samples ahead (I guess this is called a
>>>ten-step predictor???). Does anyone has a good suggestion on how to do
>>>this? Linear prediction? Adaptive filtering or system identification
>>>techniques? 
>>
>>If it truly looks like white noise being filtered by a linear bandpass 
>>filter, then you would need to model the filter and design a predictor 
>>around it.  Assuming the bandpass characteristics are known and you want
> 
> 
>>to reduce the mean-squared error then a Kalman filter is the way to go.
>>
>>If the bandpass characteristics can't be known from run to run then 
>>you'll need to do some sort of on-line system identification followed by
> 
> 
>>the Kalman or perhaps an H-infinity filter to account for imperfect 
>>knowledge of the filter coefficients.
>>
> 
> 
> Just to make sure that I got it rigth: Assuming that the signal can be
> modelled as white noise filtered by a linear filter, then I need to
> perform some on-line or off-line system id, depending on the signal, and
> then the Kalman filter performs the prediction.

Correct.

> Or do I need the Kalman
> filter to correct the coefficients found in the system id

Least-squares system ID can be cast as a Kalman filter, at least if you 
know the system input.  I'm not sure how it'll work if you're assuming 
noise in and looking at signal out, but I suspect it'll be the same.

> and then use a
> predictor on the output from the Kalman filter? 
> 
No, then use a Kalman filter configured as a predictor (this is just a 
matter of interpreting the states in the filter differently).
> 
> 
>>If the pass band is 1/10th your sample rate or wider then there's not 
>>going to be much correlation between your filter's predictions and your 
>>signal, no matter what you do.
> 
> 
> Meaning that my sample rate must be more than 10 times the pass band? Is
> this some "golden rule" or is it based on theory(probably!). 

It's based on theory.  Forget sampling for a moment and think of a 
continuous-time system, otherwise as you have described.  If you could 
know the system parameters and state perfectly at time t0, then with no 
noise you could predict the system state (and output) for all time t > 
t0.  With noise input, as time goes on the correlation between your 
prediction and fact drops, and the speed that it drops is roughly 
proportional to the bandwidth of the system.
> 
> Do you have any experience on predicting the value of a signal 10 samples
> ahead?

I answered "no", but then I remembered that yes, I have done this with 
good success, using a predictor to cancel out noise up to 40 samples 
ahead and beyond.  In my case the overall system started failing to 
cancel at around 1/40th of the sampling rate not because of my ability 
to predict the signal, but because we were driving a mechanical stage 
with the prediction, and the mechanism had limited bandwidth.

> Can one get some good results or is it not possible in practice?
> 
That depends entirely on the details of your system, and what you 
consider to be good results.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

sorenbirk wrote:
> I need to predict the future value of a signal. I dont know anything about
> the signal other than it looks like bandpass filtered white noise. I need
> to predict the value say 10 samples ahead (I guess this is called a
> ten-step predictor???). Does anyone has a good suggestion on how to do
> this? Linear prediction? Adaptive filtering or system identification
> techniques? 

Roll dice and hope?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������