Hi! I have some questions which i would have some answers to. I have some data that i have to study using its spectra. To get the spectra i use AR and ARMA estimators. How do i choose an AR and ARMA method. I have used Least-square method and two stage least-square method because i have heard that yule walker can for nearly periodic signals estimate incorrect parameters. But i have also heard that least-square method can give you an unstable model. So which do i pick? When choosing the order n and m i just tried some number until i got some result that i thought was good. Is there any better method when choosing order? I remember that i read somewhere that it is better to use the logarithmic transform of the data instead of using the data it self, when and why is this better? How can i see in my spectrum that the data is periodic or sinusoidal. And also what is the difference between parametric and non parametric methods, can they be combined to get a better estimation? It would be nice to get some of these answers :) Thx for the help!
estimating data using parametric methods
Started by ●October 1, 2008
Reply by ●October 1, 20082008-10-01
On 1 Okt, 13:52, "Unskilled" <star850...@hotmail.com> wrote:> Hi! > I have some questions which i would have some answers to. > I have some data that i have to study using its spectra. To get the > spectra i use AR and ARMA estimators. > How do i choose an AR and ARMA method.That's the art of signal analysis. Choosing what method to use (AR or ARMA, what algorithm to crunch the numbers) is a subjective decision which might be based on skill, experience or pure chance. Once you, the analyst, have chosen the method you might get help by order estimators to select model orders. More below.> I have used Least-square method and > two stage least-square method because i have heard that yule walker can for > nearly periodic signals estimate incorrect parameters.The different methods have different strengths and weakneses. Don't dismiss one particular method based on hear-say. Test it first, in the context where you intend to use it, and see if you agree. You might want to consider aspects such as ease of implementation and robustness wrt noise in addition to the accuracy of the results.> But i have also heard that least-square method can give you an unstable > model. So which do i pick?That's *your* decision, as analyst. I get the impresion that you are in a classical position where you have to chosse between one method which gives accurate results but is unstable, and another method which is robust but gives inaccurate results. The trade-off is between accurate parameters and robustness. No one other than you can make that choise, since no one else know your application and can make an informed judgement about what is more important and what can be sacrificed. It is usually better to play safe and select the robust method first, and then switch to the accurate method later if it can be demonstrated that the results from the robust method make the overall analysis fail. In other words, a robust method that is *sufficiently* accurate is better than a method that gives perfect results but might blow up in your face without warning.> When choosing the order n and m i just tried some number until i got some > result that i thought was good. Is there any better method when choosing > order?Order estimators like Akaike's Information Criterion and Rissanen & Schwartz' Minimum Description Length come to mind. Use with care, though, as they exist in different versions based on different models.> I remember that i read somewhere that it is better to use the logarithmic > transform of the data instead of using the data it self, when and why is > this better?I have no idea. This seems to be some application- dependent argument.> How can i see in my spectrum that the data is periodic or sinusoidal.You can't. You can look for narrow peaks in the spectrum which might indicate a "quasi-priodic" signal, but there are no rules or guarantees. It could be a damped sinosoidal or some other narrow-band signal which requires a different approach than would a perfectly periodic signal. You can make a *subjective* judgement based on knowledge about the context and application where the data were collected. You can then test a sinusoidal model and see if you can get what you want from the data. Just be aware that deviations from the sinusoidal model may or may not be improtant. Again, data analysis is a subjective dicipline, where different analysts might choose different apporaches and solutions based on the same data and information.> And also what is the difference between parametric and non parametric > methods, can they be combined to get a better estimation?This is again a choise between 'perfection' and robustness. There are no easy answers, everything depends on the context and purpose of the analysis. Non-parametric methods are generally robust, but might not provide very accurate answers. Everything depends on the purpose and context of the analysis. I have done combined parametric and nonparametric analyses in offline interactive processing applications, which worked quite well. I would be very careful when selecting methods for use in unsupervised systems, though.> It would be nice to get some of these answers :)No answers, only opinions and experineces. The only way to make an informed decision is to try the different methods yourself, both with simulated data where you know the 'ground truth', and real-life data which don't comply to any analytic models. Rune
Reply by ●October 1, 20082008-10-01
Unskilled wrote:> Hi! > I have some questions which i would have some answers to. > I have some data that i have to study using its spectra. To get the > spectra i use AR and ARMA estimators. > How do i choose an AR and ARMA method.That entirely depends on your data and what are you trying to accomplish. I have used Least-square method and> two stage least-square method because i have heard that yule walker can for > nearly periodic signals estimate incorrect parameters. > But i have also heard that least-square method can give you an unstable > model. So which do i pick? > > When choosing the order n and m i just tried some number until i got some > result that i thought was good. Is there any better method when choosing > order?Depending on your application, there should be a rational consideration for selecting N and M.> I remember that i read somewhere that it is better to use the logarithmic > transform of the data instead of using the data it self, when and why is > this better?It depends. This could be a good advice for some cases and the bad advice for some other cases.> How can i see in my spectrum that the data is periodic or sinusoidal.Peaking of Fourier.> And also what is the difference between parametric and non parametric > methods, can they be combined to get a better estimation?The parametric methods (like AR) are based on a model of data. The non parametric methods (like Fourier) are not based on a model. To choose the appropriate model, you can look at the result of the non-parametric method.> It would be nice to get some of these answers :)It would be nice to know what is the real problem.> Thx for the help!Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
Reply by ●October 1, 20082008-10-01
Vladimir Vassilevsky wrote:> The parametric methods (like AR) are based on a model of data. The non > parametric methods (like Fourier) are not based on a model. To choose > the appropriate model, you can look at the result of the > non-parametric method.Vladimir makes a key point here. I would have hoped that you'd mentioned models in your original question. There's all that mathematical stuff but not any particular mention of physics. That makes me nervous. Not a criticism, just a caution. Rune's response was in line with how you presented it - so I'd want to assume that you understand it all quite well. I'm no expert in the model-based methods. That's where there are "parameters" or model characteristics. Seems to me it would take quite a bit of experience to know when one thing works over another - and that would have a lot to do with the nature of the signals and the objectives. My first advice would be: "keep it simple" or "keep it of low order". Adaptive FIR filter systems seem to me to be sort of in between because they have "parameters" (the filter coefficients) which tend to stabilize for a given situation - and yet they are most likely considered to be non-parametric methods. If you want to use a model-based approach then I'd think (I don't really know) that you'd want to focus on the thing you're trying to model. Maybe there are "black box" approaches to doing that but, again, that makes me nervous. It seems to me that the most successful applications/implementations are based on *understanding* the physics involved - not just crunching numbers around it. I'm reminded of speech coding and synthesis - where I believe many of these techniques originated. Just a perspective ..... Others will no doubt set me straight on how things fit together. Fred
Reply by ●October 1, 20082008-10-01
On 1 Okt, 18:58, "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote:> I'm no expert in the model-based methods. �That's where there are > "parameters" or model characteristics. �Seems to me it would take quite a > bit of experience to know when one thing works over another -Actually, knowing when things don't work is quite easy, at least if one approaches the problem systematically by testing methods on simulated data before trying with measured data: If you don't get the expected results, the methods don't work. The hard part is to see the limitations of any given method and evaluate their impact under real-world operational conditions.> and that would > have a lot to do with the nature of the signals and the objectives. �My > first advice would be: "keep it simple" or "keep it of low order".I'd say "keep it non-parametric" if robustness or operation safety is any concern at all.> Adaptive FIR filter systems seem to me to be sort of in between because they > have "parameters" (the filter coefficients) which tend to stabilize for a > given situation - and yet they are most likely considered to be > non-parametric methods.Correct. The terms 'parametric' and a 'nonparametric' point to the description of the signals, not the processing algorithms as such (I can't think of a useful algorithm which would not contain 'parameters' of some sort). A 'parametric' methodis based on a specific mathematical model of the signal, e.g. the sum-of-sines, D x[n] = sum A_d *sin(w_d*n+phi_d) d=1 whereas the 'non-parametric' method is based on more elusive charactersistics like 'stationary zero-mean Gaussian integrable signal'. Since adaptive FIRs are designed to work on random data of rather general description, they are considered to be non-parametric. Generally speaking, the more general class of signals a method can handle, the more robust it is (and the less accurate answers one gets). It's obvious, really: It is a bit naive to expect a real-life signal to comply perfectly to an analytic signal model, as required by the parametric methods. It's perfectly obvious that they will be less robust than the more general non-parametric methods.> If you want to use a model-based approach then I'd think (I don't really > know) that you'd want to focus on the thing you're trying to model. �Maybe > there are "black box" approaches to doing that but, again, that makes me > nervous.And rightly so! Selecting the correct model is the killer: Do you need the daming terms on that wave phenomenon? Do you need the 1/sqrt(x) scaling term in the approximation to the Bessel function? If the answers to these questions is 'no' you can use the whole macinery available in the DSP toolkit; if it is 'yes' you might find yourself developing stuff more or less from scratch. And that's before one even mentions operational aspects.>�It seems to me that the most successful > applications/implementations are based on *understanding* the physics > involved - not just crunching numbers around it.The successful applications of parametic models are those where one have obtained a working compromise between robustness and accuracy. Unfortunately, the known successful areas (e.g. seismics, speech processing) have already been around for decades; new applications have to be tested to destruction to see what methods might work. Rune
