# estimating data using parametric methods

Started by October 1, 2008
```Hi!
I have some questions which i would have some answers to.
I have some data that i have to study using its spectra. To get the
spectra i use AR and ARMA estimators.
How do i choose an AR and ARMA method. I have used Least-square method and
two stage least-square method because i have heard that yule walker can for
nearly periodic signals estimate incorrect parameters.
But i have also heard that least-square method can give you an unstable
model. So which do i pick?

When choosing the order n and m i just tried some number until i got some
result that i thought was good. Is there any better method when choosing
order?

I remember that i read somewhere that it is better to use the logarithmic
transform of the data instead of using the data it self, when and why is
this better?

How can i see in my spectrum that the data is periodic or sinusoidal.

And also what is the difference between parametric and non parametric
methods, can they be combined to get a better estimation?

It would be nice to get some of these answers :)

Thx for the help!

```
```On 1 Okt, 13:52, "Unskilled" <star850...@hotmail.com> wrote:
> Hi!
> I have some questions which i would have some answers to.
> I have some data that i have to study using its spectra. To get the
> spectra i use AR and ARMA estimators.
> How do i choose an AR and ARMA method.

That's the art of signal analysis. Choosing what method
to use (AR or ARMA, what algorithm to crunch the numbers)
is a subjective decision which might be based on skill,
experience or pure chance.

Once you, the analyst, have chosen the method you might
get help by order estimators to select model orders.
More below.

> I have used Least-square method and
> two stage least-square method because i have heard that yule walker can for
> nearly periodic signals estimate incorrect parameters.

The different methods have different strengths and
weakneses. Don't dismiss one particular method based
on hear-say. Test it first, in the context where you
intend to use it, and see if you agree.

You might want to consider aspects such as ease of
implementation and robustness wrt noise in addition
to the accuracy of the results.

> But i have also heard that least-square method can give you an unstable
> model. So which do i pick?

That's *your* decision, as analyst. I get the impresion
that you are in a classical position where you have
to chosse between one method which gives accurate
results but is unstable, and another method which
is robust but gives inaccurate results.

The trade-off is between accurate parameters and
robustness. No one other than you can make that
choise, since no one else know your application
and can make an informed judgement about what is
more important and what can be sacrificed.

It is usually better to play safe and select the
robust method first, and then switch to the accurate
method later if it can be demonstrated that the
results from the robust method make the overall
analysis fail.

In other words, a robust method that is
*sufficiently* accurate is better than a method
that gives perfect results but might blow up in

> When choosing the order n and m i just tried some number until i got some
> result that i thought was good. Is there any better method when choosing
> order?

Order estimators like Akaike's Information Criterion
and Rissanen & Schwartz' Minimum Description Length
come to mind. Use with care, though, as they exist
in different versions based on different models.

> I remember that i read somewhere that it is better to use the logarithmic
> transform of the data instead of using the data it self, when and why is
> this better?

I have no idea. This seems to be some application-
dependent argument.

> How can i see in my spectrum that the data is periodic or sinusoidal.

You can't. You can look for narrow peaks in the spectrum
which might indicate a "quasi-priodic" signal, but there
are no rules or guarantees. It could be a damped sinosoidal
or some other narrow-band signal which requires a different
approach than would a perfectly periodic signal.

You can make a *subjective* judgement based on knowledge
about the context and application where the data were
collected. You can then test a sinusoidal model and see
if you can get what you want from the data. Just be aware
that deviations from the sinusoidal model may or may not
be improtant.

Again, data analysis is a subjective dicipline, where
different analysts might choose different apporaches
and solutions based on the same data and information.

> And also what is the difference between parametric and non parametric
> methods, can they be combined to get a better estimation?

This is again a choise between 'perfection' and robustness.
There are no easy answers, everything depends on the
context and purpose of the analysis. Non-parametric
methods are generally robust, but might not provide
very accurate answers. Everything depends on the
purpose and context of the analysis.

I have done combined parametric and nonparametric
analyses in offline interactive processing applications,
which worked quite well. I would be very careful when
selecting methods for use in unsupervised systems, though.

> It would be nice to get some of these answers :)

No answers, only opinions and experineces. The only
way to make an informed decision is to try the different
methods yourself, both with simulated data where you
know the 'ground truth', and real-life data which don't
comply to any analytic models.

Rune
```
```
Unskilled wrote:

> Hi!
> I have some questions which i would have some answers to.
> I have some data that i have to study using its spectra. To get the
> spectra i use AR and ARMA estimators.
> How do i choose an AR and ARMA method.

That entirely depends on your data and what are you trying to accomplish.

I have used Least-square method and
> two stage least-square method because i have heard that yule walker can for
> nearly periodic signals estimate incorrect parameters.
> But i have also heard that least-square method can give you an unstable
> model. So which do i pick?
>
> When choosing the order n and m i just tried some number until i got some
> result that i thought was good. Is there any better method when choosing
> order?

Depending on your application, there should be a rational consideration
for selecting N and M.

> I remember that i read somewhere that it is better to use the logarithmic
> transform of the data instead of using the data it self, when and why is
> this better?

It depends. This could be a good advice for some cases and the bad

> How can i see in my spectrum that the data is periodic or sinusoidal.

Peaking of Fourier.

> And also what is the difference between parametric and non parametric
> methods, can they be combined to get a better estimation?

The parametric methods (like AR) are based on a model of data. The non
parametric methods (like Fourier) are not based on a model. To choose
the appropriate model, you can look at the result of the non-parametric
method.

> It would be nice to get some of these answers :)

It would be nice to know what is the real problem.

> Thx for the help!

DSP and Mixed Signal Design Consultant
http://www.abvolt.com

```
```Vladimir Vassilevsky wrote:

> The parametric methods (like AR) are based on a model of data. The non
> parametric methods (like Fourier) are not based on a model. To choose
> the appropriate model, you can look at the result of the
> non-parametric method.

Vladimir makes a key point here.  I would have hoped that you'd mentioned
models in your original question.  There's all that mathematical stuff but
not any particular mention of physics.  That makes me nervous.  Not a
criticism, just a caution.  Rune's response was in line with how you
presented it - so I'd want to assume that you understand it all quite well.

I'm no expert in the model-based methods.  That's where there are
"parameters" or model characteristics.  Seems to me it would take quite a
bit of experience to know when one thing works over another - and that would
have a lot to do with the nature of the signals and the objectives.  My
first advice would be: "keep it simple" or "keep it of low order".

Adaptive FIR filter systems seem to me to be sort of in between because they
have "parameters" (the filter coefficients) which tend to stabilize for a
given situation - and yet they are most likely considered to be
non-parametric methods.

If you want to use a model-based approach then I'd think (I don't really
know) that you'd want to focus on the thing you're trying to model.  Maybe
there are "black box" approaches to doing that but, again, that makes me
nervous.  It seems to me that the most successful
applications/implementations are based on *understanding* the physics
involved - not just crunching numbers around it.  I'm reminded of speech
coding and synthesis - where I believe many of these techniques originated.

Just a perspective .....  Others will no doubt set me straight on how things
fit together.

Fred

```
```On 1 Okt, 18:58, "Fred Marshall" <fmarshallx@remove_the_x.acm.org>
wrote:

> I'm no expert in the model-based methods. &#2013266080;That's where there are
> "parameters" or model characteristics. &#2013266080;Seems to me it would take quite a
> bit of experience to know when one thing works over another -

Actually, knowing when things don't work is quite easy, at least
if one approaches the problem systematically by testing methods
on simulated data before trying with measured data:

If you don't get the expected results, the methods don't work.

The hard part is to see the limitations of any given method and
evaluate their impact under real-world operational conditions.

> and that would
> have a lot to do with the nature of the signals and the objectives. &#2013266080;My
> first advice would be: "keep it simple" or "keep it of low order".

I'd say "keep it non-parametric" if robustness or operation
safety is any concern at all.

> Adaptive FIR filter systems seem to me to be sort of in between because they
> have "parameters" (the filter coefficients) which tend to stabilize for a
> given situation - and yet they are most likely considered to be
> non-parametric methods.

Correct.

The terms 'parametric' and a 'nonparametric' point to
the description of the signals, not the processing
algorithms as such (I can't think of a useful algorithm
which would not contain 'parameters' of some sort).

A 'parametric' methodis based on a specific mathematical
model of the signal, e.g. the sum-of-sines,

D
x[n] = sum A_d *sin(w_d*n+phi_d)
d=1

whereas the 'non-parametric' method is based on more
elusive charactersistics like 'stationary zero-mean
Gaussian integrable signal'. Since adaptive FIRs are
designed to work on random data of rather general
description, they are considered to be non-parametric.

Generally speaking, the more general class of signals a
method can handle, the more robust it is (and the less

It's obvious, really: It is a bit naive to expect a
real-life signal to comply perfectly to an analytic
signal model, as required by the parametric methods.
It's perfectly obvious that they will be less robust
than the more general non-parametric methods.

> If you want to use a model-based approach then I'd think (I don't really
> know) that you'd want to focus on the thing you're trying to model. &#2013266080;Maybe
> there are "black box" approaches to doing that but, again, that makes me
> nervous.

And rightly so!

Selecting the correct model is the killer: Do you need
the daming terms on that wave phenomenon? Do you need the
1/sqrt(x) scaling term in the approximation to the Bessel
function?

If the answers to these questions is 'no' you can use the
whole macinery available in the DSP toolkit; if it is 'yes'
you might find yourself developing stuff more or less from
scratch.

And that's before one even mentions operational aspects.

>&#2013266080;It seems to me that the most successful
> applications/implementations are based on *understanding* the physics
> involved - not just crunching numbers around it.

The successful applications of parametic models are those
where one have obtained a working compromise between robustness
and accuracy. Unfortunately, the known successful areas (e.g.