Reply by Greg Berchin September 12, 20082008-09-12
On Tue, 9 Sep 2008 11:58:33 -0700 (PDT), HardySpicer
<gyansorova@gmail.com> wrote:

>If you have a signal + noise then estimate an ARMA model based on >this, you will also model the additive noise.
Yes, exactly. Try some of the Total Least Squares ARMA techniques. Greg
Reply by Rune Allnor September 10, 20082008-09-10
On 10 Sep, 09:50, "Nick32" <fatmansincl...@hotmail.com> wrote:
> Hi > > Thanks for all of your responses. > > The data I am trying to fit an (8,5) ARMA model to is a physiological > signal. > > The reason for specifically using an (8,5) ARMA model comes from a > physiological modelling theory.
Never expect a model to fit data just because you have an analytical model. Real-life measurements *never* comply to analytical models.
> Essentially, we want to fit (8,5) ARMA models to segments of data and see > how the model system properties (poles and zeros) change over time. By > relating these results back to the theory we hope to gain some > physiological insight.
Famous last words. As you already have discovered, the properties of the data depend at least as much on noise and imperfections as on whatever parameters you look for. Not to say that what you want is impossible, but it will take a lot more skill and effort than you might have abticipated, and the results might not be quite as relevant as you expect.
> As the amplitude of our signal is very small (microvolts), the signal to > noise ratio is low (10-20dB).
Have a look at the sensors. I don't have much hands-on experience with hardware design, but I did a student project ages ago where we converted a voltage source (a piezoelectric element which was nothing more than a weird capacitor, really) to a current source by attaching a high-impedance OpAmp as a voltage follower. As a general rule of thumb do whatever is in your grasp, technically and economically, to get good data. While DSP can help you out from time to time, the easiest problems to deal with are still those that never occur.
> A lot of the testing I have been doing has been with synthetic data. > Basically, I have been generating data by filtering Gaussian white noise > with &#2013266080;known (8,5) ARMA models to create a clean data set and then adding > noise to it to create a noisy data set. I then fitted models (using the > methods mentioned previously) to the clean and noisy data and compared > them.
That's the only way to start. Try and fit higher-order ARMA models (say, ARMA(10,8)) to the low-SNR data and see if the 'best' 5 poles and 3 zeros behave better. Rune
Reply by Nick32 September 10, 20082008-09-10
Hi

Thanks for all of your responses.

The data I am trying to fit an (8,5) ARMA model to is a physiological
signal.

The reason for specifically using an (8,5) ARMA model comes from a
physiological modelling theory.

Essentially, we want to fit (8,5) ARMA models to segments of data and see
how the model system properties (poles and zeros) change over time. By
relating these results back to the theory we hope to gain some
physiological insight.

As the amplitude of our signal is very small (microvolts), the signal to
noise ratio is low (10-20dB).

A lot of the testing I have been doing has been with synthetic data.
Basically, I have been generating data by filtering Gaussian white noise
with  known (8,5) ARMA models to create a clean data set and then adding
noise to it to create a noisy data set. I then fitted models (using the
methods mentioned previously) to the clean and noisy data and compared
them.

Thanks again.

Nick
Reply by Rune Allnor September 9, 20082008-09-09
On 9 Sep, 15:15, "Nick32" <fatmansincl...@hotmail.com> wrote:
> Hi > > I am trying to fit an (8,5) ARMA model to a time series in the presence of > additive white noise. The SNR is around 10-20dB.
What do you try to achieve? Why ARMA? Why ARMA(8,5)?
> So far I have tried the armax function from the Matlab system > identification toolbox, the ARMASA toolbox for Matlab and a reduced > statistics algorithm. Unfortunately I have found that these methods are > unreliable when the SNR is less than about 60dB, and the models produced > contain significant errors.
Well, I've never been impressed by matlab's DSP stuff...
> I have also tried some methods based on the modified Yule Walker > equations, which avoid the zero lag of the autocorrelation, but these too > are unreliable below 60dB and can return unstable models. > > What I would like to know (and hopefully someone here can help me) is: > > -Is it actually possible to get an accurate ARMA model when the SNR is > around 10-20dB?
I've worked with AR models (not ARMA) with SNR in the range 5-10 dB without awful lots of difficulties. Well, I had hundreds of data sets so I could see trends across the whole data set instead of relying on individual results. Which certainly helped. It's not impossible to get results at those SNRs, but it's not straight-forward.
> -Are there any "tried and true" robust methods for ARMA parameter > estimation at low SNR?
No. You need to try and test lots of different combinations of methods, and see which combos produce the desired results. Which means you will need to do the tests in controlled settings where you know the properties of the data and thus know how well the methods performed. Which means you need to work with simulated data.
> -Can anyone suggest a good resource for ARMA estimation from noisy > observations, or a resource that lists possible ARMA estimation methods
Kay wrote several books in the '80s and '90s; Therrien's 1992 book is my personal favourite (but it has already been out of print for more than a decade). But those deal *almost* exclusively with AR methods. Therrien has a reasonable treatise on practical ARMA methods, but don't expect miracles.
> -Alternatively, are there any reliable methods for estimating the > amplitude &#2013266080;of the additive white noise component, so that it can > compensated for.
Again, what are you trying to do? You are approaching specialist's territory where every single detail can be the decider of whether what you want can be done at all, and if so, what approaches might work. Rune
Reply by HardySpicer September 9, 20082008-09-09
On Sep 10, 1:15 am, "Nick32" <fatmansincl...@hotmail.com> wrote:
> Hi > > I am trying to fit an (8,5) ARMA model to a time series in the presence of > additive white noise. The SNR is around 10-20dB. > > So far I have tried the armax function from the Matlab system > identification toolbox, the ARMASA toolbox for Matlab and a reduced > statistics algorithm. Unfortunately I have found that these methods are > unreliable when the SNR is less than about 60dB, and the models produced > contain significant errors. > > I have also tried some methods based on the modified Yule Walker > equations, which avoid the zero lag of the autocorrelation, but these too > are unreliable below 60dB and can return unstable models. > > What I would like to know (and hopefully someone here can help me) is: > > -Is it actually possible to get an accurate ARMA model when the SNR is > around 10-20dB? > > -Are there any "tried and true" robust methods for ARMA parameter > estimation at low SNR? > > -Can anyone suggest a good resource for ARMA estimation from noisy > observations, or a resource that lists possible ARMA estimation methods > > -Alternatively, are there any reliable methods for estimating the > amplitude of the additive white noise component, so that it can > compensated for. > > Thanks > Nick
If you have a signal + noise then estimate an ARMA model based on this, you will also model the additive noise.
Reply by Yihui September 9, 20082008-09-09
ARMA is used for spectrum estimation. Like all learning process, ARMA
learns the high frequency first. If the signal is too noisy, ARMA only
learns the noise, thus become unstable. 

I always smooth a noisy signal before apply ARMA.

By the way, Yule Walker method should guarantee the stability, I think.


Yihui


Reply by Vladimir Vassilevsky September 9, 20082008-09-09

Nick32 wrote:

> Hi > > I am trying to fit an (8,5) ARMA model to a time series in the presence of > additive white noise. The SNR is around 10-20dB.
To begin with, what are you trying to accomplish?
> So far I have tried the armax function from the Matlab system > identification toolbox, the ARMASA toolbox for Matlab and a reduced > statistics algorithm. Unfortunately I have found that these methods are > unreliable when the SNR is less than about 60dB, and the models produced > contain significant errors.
The analytical methods are the simplified tractable approaches to the non-tractable original problem of maximum likelihood. Because of that, in the presense of noise and distortion, the results of the methods can be very confusing.
> I have also tried some methods based on the modified Yule Walker > equations, which avoid the zero lag of the autocorrelation, but these too > are unreliable below 60dB and can return unstable models. > > What I would like to know (and hopefully someone here can help me) is: > > -Is it actually possible to get an accurate ARMA model when the SNR is > around 10-20dB? > > -Are there any "tried and true" robust methods for ARMA parameter > estimation at low SNR? > > -Can anyone suggest a good resource for ARMA estimation from noisy > observations, or a resource that lists possible ARMA estimation methods > > -Alternatively, are there any reliable methods for estimating the > amplitude of the additive white noise component, so that it can > compensated for.
The most robust method is the brute force optimization of the parameters of the model (if this search is computationally feasible). The AR, ARMA, FDLS or any other method can provide the starting point for the search. From my experience, it is unlikely to get the correct results from the one shot application of an analytical method. Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
Reply by Nick32 September 9, 20082008-09-09
Hi

I am trying to fit an (8,5) ARMA model to a time series in the presence of
additive white noise. The SNR is around 10-20dB.

So far I have tried the armax function from the Matlab system
identification toolbox, the ARMASA toolbox for Matlab and a reduced
statistics algorithm. Unfortunately I have found that these methods are
unreliable when the SNR is less than about 60dB, and the models produced
contain significant errors.

I have also tried some methods based on the modified Yule Walker
equations, which avoid the zero lag of the autocorrelation, but these too
are unreliable below 60dB and can return unstable models.

What I would like to know (and hopefully someone here can help me) is:

-Is it actually possible to get an accurate ARMA model when the SNR is
around 10-20dB?

-Are there any "tried and true" robust methods for ARMA parameter
estimation at low SNR?

-Can anyone suggest a good resource for ARMA estimation from noisy
observations, or a resource that lists possible ARMA estimation methods

-Alternatively, are there any reliable methods for estimating the
amplitude  of the additive white noise component, so that it can
compensated for.

Thanks
Nick