ESTIMATING MA COEFFICIENT FROM ARMA

Hi All,

I am working on ARMA coefficient determination using a data sequence. I
have no idea of the underlying system from which the data was generated .
What i only have is the data and thsi si as shown in the vector form below.


If the data is labeled
d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
d(45)]

if the ARMA equation is 
y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)

and the main task is to determine the value of a(1),a(2), b(0) and b(1). 

In order to do that, i am trying to model the data as the output of an
ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).

My questions are

1) Is it logical to assume that w (white noise) above is similar to x in
the ARMA equation.

2) If yes to question (1) above, for example can i assume that using the
ARMA equation above, d(5) was generated by 

d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)

similarly

d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)


 I have been told not to use any of the least square approach in
determining my ARMA coefficents and not to use any of the sub optimal
technique but to formulate it along that line.

Thanks for anticipated response.
ARMA-GUY

On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote:
> Hi All,
>
> I am working on ARMA coefficient determination using a data sequence. I
> have no idea of the underlying system from which the data was generated .
> What i only have is the data and thsi si as shown in the vector form below.
>
> If the data is labeled
> d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
> d(45)]
>
> if the ARMA equation is
> y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>
> and the main task is to determine the value of a(1),a(2), b(0) and b(1).
>
> In order to do that, i am trying to model the data as the output of an
> ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>
> My questions are
>
> 1) Is it logical to assume that w (white noise) above is similar to x in
> the ARMA equation.

The noise w seems to be the input signal. Whether you call it 'x' or
'w'
doesn't matter.

> 2) If yes to question (1) above, for example can i assume that using the
> ARMA equation above, d(5) was generated by
>
> d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>
> similarly
>
> d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>
> &#4294967295;I have been told not to use any of the least square approach in
> determining my ARMA coefficents and not to use any of the sub optimal
> technique but to formulate it along that line.

And your question is...? The first approach would be to
ask your boss or supervisor exactly why you are not allowed
to use the sub-optimum LMS approaces. Mind you, there is a
chance that this is a 'trick question' so judge the answer
you get very carefully.

In the case this is *not* a trick question and there exists an
optimum approach that this boss or supervisor knows about, he
or she ought to tell you about it. If the boss doesn't know
of an optimum non-LMS approach, you ought to review the sub-
optimum LMS methods anyway to find out why people use them,
and not the optimum non-LMS approaches.

Basically, your stated task seems a bit dodgy, so everything
boils down to whether you trust that your boss knows what
he or she is doing. In that case this is a trick question
which has a trivial answer where no computations are
required.

Rune

Hi All,

I have asked my boss before now, he said , that determining the AR section
from an ARMA using Yuke-Walker (and its variants) method is a linear
approach while determining the MA coefficient is a non linear approach and
that is the reason for using the sub optimal technique. This fact is
contained in so many textbooks (e.g Digital; Signal and Image Processing
using MATLAB, Page 316, " Finally, remember that estimating the
coefficients of an MA model usually is not simple, since the relations 8.49
between the model coefficients and the covariances are not linear, whereas
they are for an AR model..." , also in Biomedical signal processing and
signal modeling by Eugene N. Bruce pg 381 " ... the coefficients b(k) are
not linearly related to the auto correlation values as was the case for  AR
process"

Based on this, he said there should be another method of determining the
MA coefficients. 

So from your earlier response, the w or x in ARMA equation simply means
the input signal.

ARMA GUY.

>On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote:
>> Hi All,
>>
>> I am working on ARMA coefficient determination using a data sequence.
I
>> have no idea of the underlying system from which the data was generated
.
>> What i only have is the data and thsi si as shown in the vector form
belo=
>w.
>>
>> If the data is labeled
>> d =3D [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>> d(45)]
>>
>> if the ARMA equation is
>> y(n) =3D a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>
>> and the main task is to determine the value of a(1),a(2), b(0) and
b(1).
>>
>> In order to do that, i am trying to model the data as the output of an
>> ARMA system driving by white noise, w=3D w(1),w(2),w(3),w(4)...w(n).
>>
>> My questions are
>>
>> 1) Is it logical to assume that w (white noise) above is similar to x
in
>> the ARMA equation.
>
>The noise w seems to be the input signal. Whether you call it 'x' or
>'w'
>doesn't matter.
>
>> 2) If yes to question (1) above, for example can i assume that using
the
>> ARMA equation above, d(5) was generated by
>>
>> d(5) =3D a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>
>> similarly
>>
>> d(25) =3D a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>
>> =A0I have been told not to use any of the least square approach in
>> determining my ARMA coefficents and not to use any of the sub optimal
>> technique but to formulate it along that line.
>
>And your question is...? The first approach would be to
>ask your boss or supervisor exactly why you are not allowed
>to use the sub-optimum LMS approaces. Mind you, there is a
>chance that this is a 'trick question' so judge the answer
>you get very carefully.
>
>In the case this is *not* a trick question and there exists an
>optimum approach that this boss or supervisor knows about, he
>or she ought to tell you about it. If the boss doesn't know
>of an optimum non-LMS approach, you ought to review the sub-
>optimum LMS methods anyway to find out why people use them,
>and not the optimum non-LMS approaches.
>
>Basically, your stated task seems a bit dodgy, so everything
>boils down to whether you trust that your boss knows what
>he or she is doing. In that case this is a trick question
>which has a trivial answer where no computations are
>required.
>
>Rune
>

On 26 Jan, 04:05, "ARMA GUY" <arma_...@yahoo.com> wrote:

> Based on this, he said there should be another method of determining the
> MA coefficients.

So your task is to find this optimum way of estimating the MA
parameters? If so, don't expect help here.

First of all, such a method might not be known (if it was,
the textbooks would mention it). Even if the method exists
and somebody here knwos about it, they wouldn't do your
homework for you.

The best you can hope for is a pointer to the literature, if
anything has been published. So all in all, your best choise
is to do a literature search yourself.

Rune

Hi,

I am not asking you for the solution to the assignment. Perhaps go through
the main question to see my question.

The main question have been answered before now. That is, the x(n) or w(n)
in the equation is the white noise input. 

nevertheless, thanks for answering the main question regarding x(n) or
w(n).

Bye.

>On 26 Jan, 04:05, "ARMA GUY" <arma_...@yahoo.com> wrote:
>
>> Based on this, he said there should be another method of determining
the
>> MA coefficients.
>
>So your task is to find this optimum way of estimating the MA
>parameters? If so, don't expect help here.
>
>First of all, such a method might not be known (if it was,
>the textbooks would mention it). Even if the method exists
>and somebody here knwos about it, they wouldn't do your
>homework for you.
>
>The best you can hope for is a pointer to the literature, if
>anything has been published. So all in all, your best choise
>is to do a literature search yourself.
>
>Rune
>

On Jan 26, 4:05&#4294967295;pm, "ARMA GUY" <arma_...@yahoo.com> wrote:
> Hi All,
>
> I have asked my boss before now, he said , that determining the AR section
> from an ARMA using Yuke-Walker (and its variants) method is a linear
> approach while determining the MA coefficient is a non linear approach and
> that is the reason for using the sub optimal technique. This fact is
> contained in so many textbooks (e.g Digital; Signal and Image Processing
> using MATLAB, Page 316, " Finally, remember that estimating the
> coefficients of an MA model usually is not simple, since the relations 8.49
> between the model coefficients and the covariances are not linear, whereas
> they are for an AR model..." , also in Biomedical signal processing and
> signal modeling by Eugene N. Bruce pg 381 " ... the coefficients b(k) are
> not linearly related to the auto correlation values as was the case for &#4294967295;AR
> process"
>
> Based on this, he said there should be another method of determining the
> MA coefficients.
>
> So from your earlier response, the w or x in ARMA equation simply means
> the input signal.
>
> ARMA GUY.
>
>
>
> >On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote:
> >> Hi All,
>
> >> I am working on ARMA coefficient determination using a data sequence.
> I
> >> have no idea of the underlying system from which the data was generated
> .
> >> What i only have is the data and thsi si as shown in the vector form
> belo=
> >w.
>
> >> If the data is labeled
> >> d =3D [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
> >> d(45)]
>
> >> if the ARMA equation is
> >> y(n) =3D a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>
> >> and the main task is to determine the value of a(1),a(2), b(0) and
> b(1).
>
> >> In order to do that, i am trying to model the data as the output of an
> >> ARMA system driving by white noise, w=3D w(1),w(2),w(3),w(4)...w(n).
>
> >> My questions are
>
> >> 1) Is it logical to assume that w (white noise) above is similar to x
> in
> >> the ARMA equation.
>
> >The noise w seems to be the input signal. Whether you call it 'x' or
> >'w'
> >doesn't matter.
>
> >> 2) If yes to question (1) above, for example can i assume that using
> the
> >> ARMA equation above, d(5) was generated by
>
> >> d(5) =3D a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>
> >> similarly
>
> >> d(25) =3D a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>
> >> =A0I have been told not to use any of the least square approach in
> >> determining my ARMA coefficents and not to use any of the sub optimal
> >> technique but to formulate it along that line.
>
> >And your question is...? The first approach would be to
> >ask your boss or supervisor exactly why you are not allowed
> >to use the sub-optimum LMS approaces. Mind you, there is a
> >chance that this is a 'trick question' so judge the answer
> >you get very carefully.
>
> >In the case this is *not* a trick question and there exists an
> >optimum approach that this boss or supervisor knows about, he
> >or she ought to tell you about it. If the boss doesn't know
> >of an optimum non-LMS approach, you ought to review the sub-
> >optimum LMS methods anyway to find out why people use them,
> >and not the optimum non-LMS approaches.
>
> >Basically, your stated task seems a bit dodgy, so everything
> >boils down to whether you trust that your boss knows what
> >he or she is doing. In that case this is a trick question
> >which has a trivial answer where no computations are
> >required.
>
> >Rune

Easiest way is by long division of the AR parameters into 1. This
gives an infinite MA  series whose values get smaller progressively.
It must be a convergent series otherwise the original model will be
unstable. There are no "exacts" in any case since this is all
statistical analysis.

or are you interested in an ARMA model? If so you can use recursive
maximum likelihood or extended least-squares. Also the instrumental
variables method.


Hardy

>Hi All,
>
>I am working on ARMA coefficient determination using a data sequence. I
>have no idea of the underlying system from which the data was generated
.
>What i only have is the data and thsi si as shown in the vector form
below.
>
>
>If the data is labeled
>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>d(45)]
>
>if the ARMA equation is 
>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>
>and the main task is to determine the value of a(1),a(2), b(0) and b(1).

>
>In order to do that, i am trying to model the data as the output of an
>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>
>My questions are
>
>1) Is it logical to assume that w (white noise) above is similar to x in
>the ARMA equation.
>
>2) If yes to question (1) above, for example can i assume that using the
>ARMA equation above, d(5) was generated by 
>
>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>
>similarly
>
>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>
>
> I have been told not to use any of the least square approach in
>determining my ARMA coefficents and not to use any of the sub optimal
>technique but to formulate it along that line.
>
>Thanks for anticipated response.
>ARMA-GUY
>
>
>
>
>

ARMA-guy - 

The w is the driving white noise process. Any system noise, etc., should
be modeled seperately.

To solve for the AR and MA parameters simultaneously (optimal) you need to
solve nonlinear equations usng an iterative algorithm on the estimated
autocorrelation sequence. No convergence is garanteed (see text by Kay or
the Matlab system identification toolbox by Ljung). Or you can use a high
order AR approximation that involves linear equations (see text by
Marple).

A bootstrap method is the system identification approach; estimate
parameters using linear system identification equations, filter to produce
residuals, use residuals to drive estimated ARMA filter, iterate...This is
also not garanteed to converge.

The sub-optimal approach is to estimate the AR parameters using high-order
Yule Walker equations, fitting overdetermined least squares equations by
minimizing squared error. Filter the originalsignal with the inverse AR
filter to produce MA residuals. Estimate MA parameters of residuals.

I hope this helps. I do have Matlab code for this if you are in need. Let
me know.

Dave

On 27 Jan, 00:52, "dohm" <d....@comcast.net> wrote:

> To solve for the AR and MA parameters simultaneously (optimal) you need to...

Whatever method you use, just keep in mind the order estimation
problem. It's not sufficient to know that a system is ARMA. You
also need to come up with estimates for the orders p and q of
the AR and MA terms. If a system actually is ARMA(p,q) and you
only can find parameters for an ARMA(r,s) where at least one
of the conditions

r =/= p
s =/= q

hold, you basically have spent a lot of effort to come up with
an *approximation* to the true system. Solving nonlinear systems
can require substantial amounts of work, so the questions

1) Can I estimate the orders p and q with a reasonable effort?
2) If 'no', is the extra work needed to find merely an approximate
   ARMA(r,s) model (compared to finding an approximate AR(t) model)
   worth the extra effort?

need to be considered very carefully before you go on.

Rune

Dave,

You can send the code to me, I have also implemented something of such
using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is 
arma_ann at yahoo.com. The only disadvantage i saw in that is that the
coefficients obtained for both AR and MA are not same with what i used for
generating the system especially when the variance of additive white
gaussian noise is very high or the SNR is very low. 

Rune, 

Thanks for the suggestion on the model order. My own task is just on the
model coefficients, I am presently evaluating the use of correct model
order. The next step is to use different model orders for the AR and MA
part respectively.


ARMA GUY.
>>Hi All,
>>
>>I am working on ARMA coefficient determination using a data sequence. I
>>have no idea of the underlying system from which the data was generated
>.
>>What i only have is the data and thsi si as shown in the vector form
>below.
>>
>>
>>If the data is labeled
>>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>>d(45)]
>>
>>if the ARMA equation is 
>>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>
>>and the main task is to determine the value of a(1),a(2), b(0) and
b(1).
>
>>
>>In order to do that, i am trying to model the data as the output of an
>>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>>
>>My questions are
>>
>>1) Is it logical to assume that w (white noise) above is similar to x
in
>>the ARMA equation.
>>
>>2) If yes to question (1) above, for example can i assume that using
the
>>ARMA equation above, d(5) was generated by 
>>
>>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>
>>similarly
>>
>>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>
>>
>> I have been told not to use any of the least square approach in
>>determining my ARMA coefficents and not to use any of the sub optimal
>>technique but to formulate it along that line.
>>
>>Thanks for anticipated response.
>>ARMA-GUY
>>
>>
>>
>>
>>
>
>ARMA-guy - 
>
>The w is the driving white noise process. Any system noise, etc., should
>be modeled seperately.
>
>To solve for the AR and MA parameters simultaneously (optimal) you need
to
>solve nonlinear equations usng an iterative algorithm on the estimated
>autocorrelation sequence. No convergence is garanteed (see text by Kay
or
>the Matlab system identification toolbox by Ljung). Or you can use a
high
>order AR approximation that involves linear equations (see text by
>Marple).
>
>A bootstrap method is the system identification approach; estimate
>parameters using linear system identification equations, filter to
produce
>residuals, use residuals to drive estimated ARMA filter, iterate...This
is
>also not garanteed to converge.
>
>The sub-optimal approach is to estimate the AR parameters using
high-order
>Yule Walker equations, fitting overdetermined least squares equations by
>minimizing squared error. Filter the originalsignal with the inverse AR
>filter to produce MA residuals. Estimate MA parameters of residuals.
>
>I hope this helps. I do have Matlab code for this if you are in need.
Let
>me know.
>
>Dave
>
>
>

Dave,

You can send the code to me, I have also implemented something of such
using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is 
arma_ann at yahoo.com. The only disadvantage i saw in that is that the
coefficients obtained for both AR and MA are not same with what i used for
generating the system especially when the variance of additive white
gaussian noise is very high or the SNR is very low. 

Rune, 

Thanks for the suggestion on the model order. My own task is just on the
model coefficients, I am presently evaluating the use of correct model
order. The next step is to use different model orders for the AR and MA
part respectively.


ARMA GUY.
>>Hi All,
>>
>>I am working on ARMA coefficient determination using a data sequence. I
>>have no idea of the underlying system from which the data was generated
>.
>>What i only have is the data and thsi si as shown in the vector form
>below.
>>
>>
>>If the data is labeled
>>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>>d(45)]
>>
>>if the ARMA equation is 
>>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>
>>and the main task is to determine the value of a(1),a(2), b(0) and
b(1).
>
>>
>>In order to do that, i am trying to model the data as the output of an
>>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>>
>>My questions are
>>
>>1) Is it logical to assume that w (white noise) above is similar to x
in
>>the ARMA equation.
>>
>>2) If yes to question (1) above, for example can i assume that using
the
>>ARMA equation above, d(5) was generated by 
>>
>>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>
>>similarly
>>
>>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>
>>
>> I have been told not to use any of the least square approach in
>>determining my ARMA coefficents and not to use any of the sub optimal
>>technique but to formulate it along that line.
>>
>>Thanks for anticipated response.
>>ARMA-GUY
>>
>>
>>
>>
>>
>
>ARMA-guy - 
>
>The w is the driving white noise process. Any system noise, etc., should
>be modeled seperately.
>
>To solve for the AR and MA parameters simultaneously (optimal) you need
to
>solve nonlinear equations usng an iterative algorithm on the estimated
>autocorrelation sequence. No convergence is garanteed (see text by Kay
or
>the Matlab system identification toolbox by Ljung). Or you can use a
high
>order AR approximation that involves linear equations (see text by
>Marple).
>
>A bootstrap method is the system identification approach; estimate
>parameters using linear system identification equations, filter to
produce
>residuals, use residuals to drive estimated ARMA filter, iterate...This
is
>also not garanteed to converge.
>
>The sub-optimal approach is to estimate the AR parameters using
high-order
>Yule Walker equations, fitting overdetermined least squares equations by
>minimizing squared error. Filter the originalsignal with the inverse AR
>filter to produce MA residuals. Estimate MA parameters of residuals.
>
>I hope this helps. I do have Matlab code for this if you are in need.
Let
>me know.
>
>Dave
>
>
>