Hi All, I am working on ARMA coefficient determination using a data sequence. I have no idea of the underlying system from which the data was generated . What i only have is the data and thsi si as shown in the vector form below. If the data is labeled d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... d(45)] if the ARMA equation is y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) and the main task is to determine the value of a(1),a(2), b(0) and b(1). In order to do that, i am trying to model the data as the output of an ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n). My questions are 1) Is it logical to assume that w (white noise) above is similar to x in the ARMA equation. 2) If yes to question (1) above, for example can i assume that using the ARMA equation above, d(5) was generated by d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) similarly d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) I have been told not to use any of the least square approach in determining my ARMA coefficents and not to use any of the sub optimal technique but to formulate it along that line. Thanks for anticipated response. ARMA-GUY

# ESTIMATING MA COEFFICIENT FROM ARMA

Started by ●January 25, 2009

Reply by ●January 25, 20092009-01-25

On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote:> Hi All, > > I am working on ARMA coefficient determination using a data sequence. I > have no idea of the underlying system from which the data was generated . > What i only have is the data and thsi si as shown in the vector form below. > > If the data is labeled > d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... > d(45)] > > if the ARMA equation is > y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) > > and the main task is to determine the value of a(1),a(2), b(0) and b(1). > > In order to do that, i am trying to model the data as the output of an > ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n). > > My questions are > > 1) Is it logical to assume that w (white noise) above is similar to x in > the ARMA equation.The noise w seems to be the input signal. Whether you call it 'x' or 'w' doesn't matter.> 2) If yes to question (1) above, for example can i assume that using the > ARMA equation above, d(5) was generated by > > d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) > > similarly > > d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) > > �I have been told not to use any of the least square approach in > determining my ARMA coefficents and not to use any of the sub optimal > technique but to formulate it along that line.And your question is...? The first approach would be to ask your boss or supervisor exactly why you are not allowed to use the sub-optimum LMS approaces. Mind you, there is a chance that this is a 'trick question' so judge the answer you get very carefully. In the case this is *not* a trick question and there exists an optimum approach that this boss or supervisor knows about, he or she ought to tell you about it. If the boss doesn't know of an optimum non-LMS approach, you ought to review the sub- optimum LMS methods anyway to find out why people use them, and not the optimum non-LMS approaches. Basically, your stated task seems a bit dodgy, so everything boils down to whether you trust that your boss knows what he or she is doing. In that case this is a trick question which has a trivial answer where no computations are required. Rune

Reply by ●January 25, 20092009-01-25

Hi All, I have asked my boss before now, he said , that determining the AR section from an ARMA using Yuke-Walker (and its variants) method is a linear approach while determining the MA coefficient is a non linear approach and that is the reason for using the sub optimal technique. This fact is contained in so many textbooks (e.g Digital; Signal and Image Processing using MATLAB, Page 316, " Finally, remember that estimating the coefficients of an MA model usually is not simple, since the relations 8.49 between the model coefficients and the covariances are not linear, whereas they are for an AR model..." , also in Biomedical signal processing and signal modeling by Eugene N. Bruce pg 381 " ... the coefficients b(k) are not linearly related to the auto correlation values as was the case for AR process" Based on this, he said there should be another method of determining the MA coefficients. So from your earlier response, the w or x in ARMA equation simply means the input signal. ARMA GUY.>On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote: >> Hi All, >> >> I am working on ARMA coefficient determination using a data sequence.I>> have no idea of the underlying system from which the data was generated.>> What i only have is the data and thsi si as shown in the vector formbelo=>w. >> >> If the data is labeled >> d =3D [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... >> d(45)] >> >> if the ARMA equation is >> y(n) =3D a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) >> >> and the main task is to determine the value of a(1),a(2), b(0) andb(1).>> >> In order to do that, i am trying to model the data as the output of an >> ARMA system driving by white noise, w=3D w(1),w(2),w(3),w(4)...w(n). >> >> My questions are >> >> 1) Is it logical to assume that w (white noise) above is similar to xin>> the ARMA equation. > >The noise w seems to be the input signal. Whether you call it 'x' or >'w' >doesn't matter. > >> 2) If yes to question (1) above, for example can i assume that usingthe>> ARMA equation above, d(5) was generated by >> >> d(5) =3D a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) >> >> similarly >> >> d(25) =3D a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) >> >> =A0I have been told not to use any of the least square approach in >> determining my ARMA coefficents and not to use any of the sub optimal >> technique but to formulate it along that line. > >And your question is...? The first approach would be to >ask your boss or supervisor exactly why you are not allowed >to use the sub-optimum LMS approaces. Mind you, there is a >chance that this is a 'trick question' so judge the answer >you get very carefully. > >In the case this is *not* a trick question and there exists an >optimum approach that this boss or supervisor knows about, he >or she ought to tell you about it. If the boss doesn't know >of an optimum non-LMS approach, you ought to review the sub- >optimum LMS methods anyway to find out why people use them, >and not the optimum non-LMS approaches. > >Basically, your stated task seems a bit dodgy, so everything >boils down to whether you trust that your boss knows what >he or she is doing. In that case this is a trick question >which has a trivial answer where no computations are >required. > >Rune >

Reply by ●January 26, 20092009-01-26

On 26 Jan, 04:05, "ARMA GUY" <arma_...@yahoo.com> wrote:> Based on this, he said there should be another method of determining the > MA coefficients.So your task is to find this optimum way of estimating the MA parameters? If so, don't expect help here. First of all, such a method might not be known (if it was, the textbooks would mention it). Even if the method exists and somebody here knwos about it, they wouldn't do your homework for you. The best you can hope for is a pointer to the literature, if anything has been published. So all in all, your best choise is to do a literature search yourself. Rune

Reply by ●January 26, 20092009-01-26

Hi, I am not asking you for the solution to the assignment. Perhaps go through the main question to see my question. The main question have been answered before now. That is, the x(n) or w(n) in the equation is the white noise input. nevertheless, thanks for answering the main question regarding x(n) or w(n). Bye.>On 26 Jan, 04:05, "ARMA GUY" <arma_...@yahoo.com> wrote: > >> Based on this, he said there should be another method of determiningthe>> MA coefficients. > >So your task is to find this optimum way of estimating the MA >parameters? If so, don't expect help here. > >First of all, such a method might not be known (if it was, >the textbooks would mention it). Even if the method exists >and somebody here knwos about it, they wouldn't do your >homework for you. > >The best you can hope for is a pointer to the literature, if >anything has been published. So all in all, your best choise >is to do a literature search yourself. > >Rune >

Reply by ●January 26, 20092009-01-26

On Jan 26, 4:05�pm, "ARMA GUY" <arma_...@yahoo.com> wrote:> Hi All, > > I have asked my boss before now, he said , that determining the AR section > from an ARMA using Yuke-Walker (and its variants) method is a linear > approach while determining the MA coefficient is a non linear approach and > that is the reason for using the sub optimal technique. This fact is > contained in so many textbooks (e.g Digital; Signal and Image Processing > using MATLAB, Page 316, " Finally, remember that estimating the > coefficients of an MA model usually is not simple, since the relations 8.49 > between the model coefficients and the covariances are not linear, whereas > they are for an AR model..." , also in Biomedical signal processing and > signal modeling by Eugene N. Bruce pg 381 " ... the coefficients b(k) are > not linearly related to the auto correlation values as was the case for �AR > process" > > Based on this, he said there should be another method of determining the > MA coefficients. > > So from your earlier response, the w or x in ARMA equation simply means > the input signal. > > ARMA GUY. > > > > >On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote: > >> Hi All, > > >> I am working on ARMA coefficient determination using a data sequence. > I > >> have no idea of the underlying system from which the data was generated > . > >> What i only have is the data and thsi si as shown in the vector form > belo= > >w. > > >> If the data is labeled > >> d =3D [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... > >> d(45)] > > >> if the ARMA equation is > >> y(n) =3D a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) > > >> and the main task is to determine the value of a(1),a(2), b(0) and > b(1). > > >> In order to do that, i am trying to model the data as the output of an > >> ARMA system driving by white noise, w=3D w(1),w(2),w(3),w(4)...w(n). > > >> My questions are > > >> 1) Is it logical to assume that w (white noise) above is similar to x > in > >> the ARMA equation. > > >The noise w seems to be the input signal. Whether you call it 'x' or > >'w' > >doesn't matter. > > >> 2) If yes to question (1) above, for example can i assume that using > the > >> ARMA equation above, d(5) was generated by > > >> d(5) =3D a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) > > >> similarly > > >> d(25) =3D a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) > > >> =A0I have been told not to use any of the least square approach in > >> determining my ARMA coefficents and not to use any of the sub optimal > >> technique but to formulate it along that line. > > >And your question is...? The first approach would be to > >ask your boss or supervisor exactly why you are not allowed > >to use the sub-optimum LMS approaces. Mind you, there is a > >chance that this is a 'trick question' so judge the answer > >you get very carefully. > > >In the case this is *not* a trick question and there exists an > >optimum approach that this boss or supervisor knows about, he > >or she ought to tell you about it. If the boss doesn't know > >of an optimum non-LMS approach, you ought to review the sub- > >optimum LMS methods anyway to find out why people use them, > >and not the optimum non-LMS approaches. > > >Basically, your stated task seems a bit dodgy, so everything > >boils down to whether you trust that your boss knows what > >he or she is doing. In that case this is a trick question > >which has a trivial answer where no computations are > >required. > > >RuneEasiest way is by long division of the AR parameters into 1. This gives an infinite MA series whose values get smaller progressively. It must be a convergent series otherwise the original model will be unstable. There are no "exacts" in any case since this is all statistical analysis. or are you interested in an ARMA model? If so you can use recursive maximum likelihood or extended least-squares. Also the instrumental variables method. Hardy

Reply by ●January 26, 20092009-01-26

>Hi All, > >I am working on ARMA coefficient determination using a data sequence. I >have no idea of the underlying system from which the data was generated.>What i only have is the data and thsi si as shown in the vector formbelow.> > >If the data is labeled >d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... >d(45)] > >if the ARMA equation is >y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) > >and the main task is to determine the value of a(1),a(2), b(0) and b(1).> >In order to do that, i am trying to model the data as the output of an >ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n). > >My questions are > >1) Is it logical to assume that w (white noise) above is similar to x in >the ARMA equation. > >2) If yes to question (1) above, for example can i assume that using the >ARMA equation above, d(5) was generated by > >d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) > >similarly > >d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) > > > I have been told not to use any of the least square approach in >determining my ARMA coefficents and not to use any of the sub optimal >technique but to formulate it along that line. > >Thanks for anticipated response. >ARMA-GUY > > > > >ARMA-guy - The w is the driving white noise process. Any system noise, etc., should be modeled seperately. To solve for the AR and MA parameters simultaneously (optimal) you need to solve nonlinear equations usng an iterative algorithm on the estimated autocorrelation sequence. No convergence is garanteed (see text by Kay or the Matlab system identification toolbox by Ljung). Or you can use a high order AR approximation that involves linear equations (see text by Marple). A bootstrap method is the system identification approach; estimate parameters using linear system identification equations, filter to produce residuals, use residuals to drive estimated ARMA filter, iterate...This is also not garanteed to converge. The sub-optimal approach is to estimate the AR parameters using high-order Yule Walker equations, fitting overdetermined least squares equations by minimizing squared error. Filter the originalsignal with the inverse AR filter to produce MA residuals. Estimate MA parameters of residuals. I hope this helps. I do have Matlab code for this if you are in need. Let me know. Dave

Reply by ●January 27, 20092009-01-27

On 27 Jan, 00:52, "dohm" <d....@comcast.net> wrote:> To solve for the AR and MA parameters simultaneously (optimal) you need to...Whatever method you use, just keep in mind the order estimation problem. It's not sufficient to know that a system is ARMA. You also need to come up with estimates for the orders p and q of the AR and MA terms. If a system actually is ARMA(p,q) and you only can find parameters for an ARMA(r,s) where at least one of the conditions r =/= p s =/= q hold, you basically have spent a lot of effort to come up with an *approximation* to the true system. Solving nonlinear systems can require substantial amounts of work, so the questions 1) Can I estimate the orders p and q with a reasonable effort? 2) If 'no', is the extra work needed to find merely an approximate ARMA(r,s) model (compared to finding an approximate AR(t) model) worth the extra effort? need to be considered very carefully before you go on. Rune

Reply by ●January 28, 20092009-01-28

Dave, You can send the code to me, I have also implemented something of such using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is arma_ann at yahoo.com. The only disadvantage i saw in that is that the coefficients obtained for both AR and MA are not same with what i used for generating the system especially when the variance of additive white gaussian noise is very high or the SNR is very low. Rune, Thanks for the suggestion on the model order. My own task is just on the model coefficients, I am presently evaluating the use of correct model order. The next step is to use different model orders for the AR and MA part respectively. ARMA GUY.>>Hi All, >> >>I am working on ARMA coefficient determination using a data sequence. I >>have no idea of the underlying system from which the data was generated >. >>What i only have is the data and thsi si as shown in the vector form >below. >> >> >>If the data is labeled >>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... >>d(45)] >> >>if the ARMA equation is >>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) >> >>and the main task is to determine the value of a(1),a(2), b(0) andb(1).> >> >>In order to do that, i am trying to model the data as the output of an >>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n). >> >>My questions are >> >>1) Is it logical to assume that w (white noise) above is similar to xin>>the ARMA equation. >> >>2) If yes to question (1) above, for example can i assume that usingthe>>ARMA equation above, d(5) was generated by >> >>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) >> >>similarly >> >>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) >> >> >> I have been told not to use any of the least square approach in >>determining my ARMA coefficents and not to use any of the sub optimal >>technique but to formulate it along that line. >> >>Thanks for anticipated response. >>ARMA-GUY >> >> >> >> >> > >ARMA-guy - > >The w is the driving white noise process. Any system noise, etc., should >be modeled seperately. > >To solve for the AR and MA parameters simultaneously (optimal) you needto>solve nonlinear equations usng an iterative algorithm on the estimated >autocorrelation sequence. No convergence is garanteed (see text by Kayor>the Matlab system identification toolbox by Ljung). Or you can use ahigh>order AR approximation that involves linear equations (see text by >Marple). > >A bootstrap method is the system identification approach; estimate >parameters using linear system identification equations, filter toproduce>residuals, use residuals to drive estimated ARMA filter, iterate...Thisis>also not garanteed to converge. > >The sub-optimal approach is to estimate the AR parameters usinghigh-order>Yule Walker equations, fitting overdetermined least squares equations by >minimizing squared error. Filter the originalsignal with the inverse AR >filter to produce MA residuals. Estimate MA parameters of residuals. > >I hope this helps. I do have Matlab code for this if you are in need.Let>me know. > >Dave > > >

Reply by ●January 28, 20092009-01-28

Dave, You can send the code to me, I have also implemented something of such using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is arma_ann at yahoo.com. The only disadvantage i saw in that is that the coefficients obtained for both AR and MA are not same with what i used for generating the system especially when the variance of additive white gaussian noise is very high or the SNR is very low. Rune, Thanks for the suggestion on the model order. My own task is just on the model coefficients, I am presently evaluating the use of correct model order. The next step is to use different model orders for the AR and MA part respectively. ARMA GUY.>>Hi All, >> >>I am working on ARMA coefficient determination using a data sequence. I >>have no idea of the underlying system from which the data was generated >. >>What i only have is the data and thsi si as shown in the vector form >below. >> >> >>If the data is labeled >>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),...... >>d(45)] >> >>if the ARMA equation is >>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1) >> >>and the main task is to determine the value of a(1),a(2), b(0) andb(1).> >> >>In order to do that, i am trying to model the data as the output of an >>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n). >> >>My questions are >> >>1) Is it logical to assume that w (white noise) above is similar to xin>>the ARMA equation. >> >>2) If yes to question (1) above, for example can i assume that usingthe>>ARMA equation above, d(5) was generated by >> >>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4) >> >>similarly >> >>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24) >> >> >> I have been told not to use any of the least square approach in >>determining my ARMA coefficents and not to use any of the sub optimal >>technique but to formulate it along that line. >> >>Thanks for anticipated response. >>ARMA-GUY >> >> >> >> >> > >ARMA-guy - > >The w is the driving white noise process. Any system noise, etc., should >be modeled seperately. > >To solve for the AR and MA parameters simultaneously (optimal) you needto>solve nonlinear equations usng an iterative algorithm on the estimated >autocorrelation sequence. No convergence is garanteed (see text by Kayor>the Matlab system identification toolbox by Ljung). Or you can use ahigh>order AR approximation that involves linear equations (see text by >Marple). > >A bootstrap method is the system identification approach; estimate >parameters using linear system identification equations, filter toproduce>residuals, use residuals to drive estimated ARMA filter, iterate...Thisis>also not garanteed to converge. > >The sub-optimal approach is to estimate the AR parameters usinghigh-order>Yule Walker equations, fitting overdetermined least squares equations by >minimizing squared error. Filter the originalsignal with the inverse AR >filter to produce MA residuals. Estimate MA parameters of residuals. > >I hope this helps. I do have Matlab code for this if you are in need.Let>me know. > >Dave > > >