Dave,

The code on SUB-OPTIMAL APPROACH.

"The sub-optimal approach is to estimate the AR parameters using
high-order
Yule Walker equations, fitting overdetermined least squares equations by
minimizing squared error. Filter the original signal with the inverse AR
filter to produce MA residuals. Estimate MA parameters of residuals ".

Do you mind if i send mine also ?

ARMA GUY


>Dave,
>
>You can send the code to me, I have also implemented something of such
>using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is 
>arma_ann at yahoo.com. The only disadvantage i saw in that is that the
>coefficients obtained for both AR and MA are not same with what i used
for
>generating the system especially when the variance of additive white
>gaussian noise is very high or the SNR is very low. 
>
>Rune, 
>
>Thanks for the suggestion on the model order. My own task is just on the
>model coefficients, I am presently evaluating the use of correct model
>order. The next step is to use different model orders for the AR and MA
>part respectively.
>
>
>ARMA GUY.
>>>Hi All,
>>>
>>>I am working on ARMA coefficient determination using a data sequence.
I
>>>have no idea of the underlying system from which the data was
generated
>>.
>>>What i only have is the data and thsi si as shown in the vector form
>>below.
>>>
>>>
>>>If the data is labeled
>>>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>>>d(45)]
>>>
>>>if the ARMA equation is 
>>>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>>
>>>and the main task is to determine the value of a(1),a(2), b(0) and
>b(1).
>>
>>>
>>>In order to do that, i am trying to model the data as the output of an
>>>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>>>
>>>My questions are
>>>
>>>1) Is it logical to assume that w (white noise) above is similar to x
>in
>>>the ARMA equation.
>>>
>>>2) If yes to question (1) above, for example can i assume that using
>the
>>>ARMA equation above, d(5) was generated by 
>>>
>>>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>>
>>>similarly
>>>
>>>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>>
>>>
>>> I have been told not to use any of the least square approach in
>>>determining my ARMA coefficents and not to use any of the sub optimal
>>>technique but to formulate it along that line.
>>>
>>>Thanks for anticipated response.
>>>ARMA-GUY
>>>
>>>
>>>
>>>
>>>
>>
>>ARMA-guy - 
>>
>>The w is the driving white noise process. Any system noise, etc.,
should
>>be modeled seperately.
>>
>>To solve for the AR and MA parameters simultaneously (optimal) you need
>to
>>solve nonlinear equations usng an iterative algorithm on the estimated
>>autocorrelation sequence. No convergence is garanteed (see text by Kay
>or
>>the Matlab system identification toolbox by Ljung). Or you can use a
>high
>>order AR approximation that involves linear equations (see text by
>>Marple).
>>
>>A bootstrap method is the system identification approach; estimate
>>parameters using linear system identification equations, filter to
>produce
>>residuals, use residuals to drive estimated ARMA filter, iterate...This
>is
>>also not garanteed to converge.
>>
>>The sub-optimal approach is to estimate the AR parameters using
>high-order
>>Yule Walker equations, fitting overdetermined least squares equations
by
>>minimizing squared error. Filter the originalsignal with the inverse AR
>>filter to produce MA residuals. Estimate MA parameters of residuals.
>>
>>I hope this helps. I do have Matlab code for this if you are in need.
>Let
>>me know.
>>
>>Dave
>>
>>
>>
>

ARMA-guy

Which code do you want? I mentioned several methods. I am not interested
in sending you all of the code that i have for parameteric modeling. From
reading you posts it seems that you need to study this problem further
before blindly using code that is handed to you (It will probably not
help). The built in function to Matlab will be useful for learning. You
should read carefully the last post from Rune about the realities of data
analysis in real aaplications. Also, carefully study the model assumptions
and read through the parameteric model chapters in the texts by both Kay
and Marple. Both books also include code you can use. Then try writing your
own matlab functions so solve for the AR and MA coefficients, etc. Make
sure you understand the direct, or brute force, algorithms and then move on
to the practical, or fast, algorithms (levinson, etc.). For example, can
you explain why your AR and MA coefficients are not the same when when your
SNR is low, etc.? When would they be the same? Read and ponder my
friend...then experiment...then ask for help on specific areas that you do
not understand. At this point my best answer to your questions is to read
chapters 5-10 in Modern Spectral Estimation by Steven Kay. It would take me
a very long time to reproduce his work in this discussion group. 

Dave   


>Dave,
>
>You can send the code to me, I have also implemented something of such
>using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is 
>arma_ann at yahoo.com. The only disadvantage i saw in that is that the
>coefficients obtained for both AR and MA are not same with what i used
for
>generating the system especially when the variance of additive white
>gaussian noise is very high or the SNR is very low. 
>
>Rune, 
>
>Thanks for the suggestion on the model order. My own task is just on the
>model coefficients, I am presently evaluating the use of correct model
>order. The next step is to use different model orders for the AR and MA
>part respectively.
>
>
>ARMA GUY.
>>>Hi All,
>>>
>>>I am working on ARMA coefficient determination using a data sequence.
I
>>>have no idea of the underlying system from which the data was
generated
>>.
>>>What i only have is the data and thsi si as shown in the vector form
>>below.
>>>
>>>
>>>If the data is labeled
>>>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>>>d(45)]
>>>
>>>if the ARMA equation is 
>>>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>>
>>>and the main task is to determine the value of a(1),a(2), b(0) and
>b(1).
>>
>>>
>>>In order to do that, i am trying to model the data as the output of an
>>>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>>>
>>>My questions are
>>>
>>>1) Is it logical to assume that w (white noise) above is similar to x
>in
>>>the ARMA equation.
>>>
>>>2) If yes to question (1) above, for example can i assume that using
>the
>>>ARMA equation above, d(5) was generated by 
>>>
>>>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>>
>>>similarly
>>>
>>>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>>
>>>
>>> I have been told not to use any of the least square approach in
>>>determining my ARMA coefficents and not to use any of the sub optimal
>>>technique but to formulate it along that line.
>>>
>>>Thanks for anticipated response.
>>>ARMA-GUY
>>>
>>>
>>>
>>>
>>>
>>
>>ARMA-guy - 
>>
>>The w is the driving white noise process. Any system noise, etc.,
should
>>be modeled seperately.
>>
>>To solve for the AR and MA parameters simultaneously (optimal) you need
>to
>>solve nonlinear equations usng an iterative algorithm on the estimated
>>autocorrelation sequence. No convergence is garanteed (see text by Kay
>or
>>the Matlab system identification toolbox by Ljung). Or you can use a
>high
>>order AR approximation that involves linear equations (see text by
>>Marple).
>>
>>A bootstrap method is the system identification approach; estimate
>>parameters using linear system identification equations, filter to
>produce
>>residuals, use residuals to drive estimated ARMA filter, iterate...This
>is
>>also not garanteed to converge.
>>
>>The sub-optimal approach is to estimate the AR parameters using
>high-order
>>Yule Walker equations, fitting overdetermined least squares equations
by
>>minimizing squared error. Filter the originalsignal with the inverse AR
>>filter to produce MA residuals. Estimate MA parameters of residuals.
>>
>>I hope this helps. I do have Matlab code for this if you are in need.
>Let
>>me know.
>>
>>Dave
>>
>>
>>
>

On 28 Jan, 11:07, "ARMA GUY" <arma_...@yahoo.com> wrote:

>  The only disadvantage i saw in that is that the
> coefficients obtained for both AR and MA are not same with what i used for
> generating the system especially when the variance of additive white
> gaussian noise is very high or the SNR is very low.

Welcome to the world of data analysis. Did you use the correct
orders for the AR and MA steps? What do you think will happen
when you don't know the true orders and parameters before you
compute your estimates? What are the risks involved in getting
those numbers wrong? That is, what consequences will such an
error have for the user of your program? If the answer is
"it doesn't matter to the user" - which might well be the
case - why use the elaborate ARMA model at all? Why not settle
for a simpler, more robust and well-behaved AR model?

Rune

Dave,

You can send the code to me, I have also implemented something of such
using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is 
arma_ann at yahoo.com. The only disadvantage i saw in that is that the
coefficients obtained for both AR and MA are not same with what i used for
generating the system especially when the variance of additive white
gaussian noise is very high or the SNR is very low. 

Rune, 

Thanks for the suggestion on the model order. My own task is just on the
model coefficients, I am presently evaluating the use of correct model
order. The next step is to use different model orders for the AR and MA
part respectively.


ARMA GUY.
>>Hi All,
>>
>>I am working on ARMA coefficient determination using a data sequence. I
>>have no idea of the underlying system from which the data was generated
>.
>>What i only have is the data and thsi si as shown in the vector form
>below.
>>
>>
>>If the data is labeled
>>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>>d(45)]
>>
>>if the ARMA equation is 
>>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>
>>and the main task is to determine the value of a(1),a(2), b(0) and
b(1).
>
>>
>>In order to do that, i am trying to model the data as the output of an
>>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>>
>>My questions are
>>
>>1) Is it logical to assume that w (white noise) above is similar to x
in
>>the ARMA equation.
>>
>>2) If yes to question (1) above, for example can i assume that using
the
>>ARMA equation above, d(5) was generated by 
>>
>>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>
>>similarly
>>
>>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>
>>
>> I have been told not to use any of the least square approach in
>>determining my ARMA coefficents and not to use any of the sub optimal
>>technique but to formulate it along that line.
>>
>>Thanks for anticipated response.
>>ARMA-GUY
>>
>>
>>
>>
>>
>
>ARMA-guy - 
>
>The w is the driving white noise process. Any system noise, etc., should
>be modeled seperately.
>
>To solve for the AR and MA parameters simultaneously (optimal) you need
to
>solve nonlinear equations usng an iterative algorithm on the estimated
>autocorrelation sequence. No convergence is garanteed (see text by Kay
or
>the Matlab system identification toolbox by Ljung). Or you can use a
high
>order AR approximation that involves linear equations (see text by
>Marple).
>
>A bootstrap method is the system identification approach; estimate
>parameters using linear system identification equations, filter to
produce
>residuals, use residuals to drive estimated ARMA filter, iterate...This
is
>also not garanteed to converge.
>
>The sub-optimal approach is to estimate the AR parameters using
high-order
>Yule Walker equations, fitting overdetermined least squares equations by
>minimizing squared error. Filter the originalsignal with the inverse AR
>filter to produce MA residuals. Estimate MA parameters of residuals.
>
>I hope this helps. I do have Matlab code for this if you are in need.
Let
>me know.
>
>Dave
>
>
>

Dave,

You can send the code to me, I have also implemented something of such
using MATLAB inbuilt functions (ARYULE, lpc, filter etc). My email is 
arma_ann at yahoo.com. The only disadvantage i saw in that is that the
coefficients obtained for both AR and MA are not same with what i used for
generating the system especially when the variance of additive white
gaussian noise is very high or the SNR is very low. 

Rune, 

Thanks for the suggestion on the model order. My own task is just on the
model coefficients, I am presently evaluating the use of correct model
order. The next step is to use different model orders for the AR and MA
part respectively.


ARMA GUY.
>>Hi All,
>>
>>I am working on ARMA coefficient determination using a data sequence. I
>>have no idea of the underlying system from which the data was generated
>.
>>What i only have is the data and thsi si as shown in the vector form
>below.
>>
>>
>>If the data is labeled
>>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>>d(45)]
>>
>>if the ARMA equation is 
>>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>>
>>and the main task is to determine the value of a(1),a(2), b(0) and
b(1).
>
>>
>>In order to do that, i am trying to model the data as the output of an
>>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>>
>>My questions are
>>
>>1) Is it logical to assume that w (white noise) above is similar to x
in
>>the ARMA equation.
>>
>>2) If yes to question (1) above, for example can i assume that using
the
>>ARMA equation above, d(5) was generated by 
>>
>>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>>
>>similarly
>>
>>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>>
>>
>> I have been told not to use any of the least square approach in
>>determining my ARMA coefficents and not to use any of the sub optimal
>>technique but to formulate it along that line.
>>
>>Thanks for anticipated response.
>>ARMA-GUY
>>
>>
>>
>>
>>
>
>ARMA-guy - 
>
>The w is the driving white noise process. Any system noise, etc., should
>be modeled seperately.
>
>To solve for the AR and MA parameters simultaneously (optimal) you need
to
>solve nonlinear equations usng an iterative algorithm on the estimated
>autocorrelation sequence. No convergence is garanteed (see text by Kay
or
>the Matlab system identification toolbox by Ljung). Or you can use a
high
>order AR approximation that involves linear equations (see text by
>Marple).
>
>A bootstrap method is the system identification approach; estimate
>parameters using linear system identification equations, filter to
produce
>residuals, use residuals to drive estimated ARMA filter, iterate...This
is
>also not garanteed to converge.
>
>The sub-optimal approach is to estimate the AR parameters using
high-order
>Yule Walker equations, fitting overdetermined least squares equations by
>minimizing squared error. Filter the originalsignal with the inverse AR
>filter to produce MA residuals. Estimate MA parameters of residuals.
>
>I hope this helps. I do have Matlab code for this if you are in need.
Let
>me know.
>
>Dave
>
>
>

On 27 Jan, 00:52, "dohm" <d....@comcast.net> wrote:

> To solve for the AR and MA parameters simultaneously (optimal) you need to...

Whatever method you use, just keep in mind the order estimation
problem. It's not sufficient to know that a system is ARMA. You
also need to come up with estimates for the orders p and q of
the AR and MA terms. If a system actually is ARMA(p,q) and you
only can find parameters for an ARMA(r,s) where at least one
of the conditions

r =/= p
s =/= q

hold, you basically have spent a lot of effort to come up with
an *approximation* to the true system. Solving nonlinear systems
can require substantial amounts of work, so the questions

1) Can I estimate the orders p and q with a reasonable effort?
2) If 'no', is the extra work needed to find merely an approximate
   ARMA(r,s) model (compared to finding an approximate AR(t) model)
   worth the extra effort?

need to be considered very carefully before you go on.

Rune

>Hi All,
>
>I am working on ARMA coefficient determination using a data sequence. I
>have no idea of the underlying system from which the data was generated
.
>What i only have is the data and thsi si as shown in the vector form
below.
>
>
>If the data is labeled
>d = [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
>d(45)]
>
>if the ARMA equation is 
>y(n) = a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>
>and the main task is to determine the value of a(1),a(2), b(0) and b(1).

>
>In order to do that, i am trying to model the data as the output of an
>ARMA system driving by white noise, w= w(1),w(2),w(3),w(4)...w(n).
>
>My questions are
>
>1) Is it logical to assume that w (white noise) above is similar to x in
>the ARMA equation.
>
>2) If yes to question (1) above, for example can i assume that using the
>ARMA equation above, d(5) was generated by 
>
>d(5) = a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>
>similarly
>
>d(25) = a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>
>
> I have been told not to use any of the least square approach in
>determining my ARMA coefficents and not to use any of the sub optimal
>technique but to formulate it along that line.
>
>Thanks for anticipated response.
>ARMA-GUY
>
>
>
>
>

ARMA-guy - 

The w is the driving white noise process. Any system noise, etc., should
be modeled seperately.

To solve for the AR and MA parameters simultaneously (optimal) you need to
solve nonlinear equations usng an iterative algorithm on the estimated
autocorrelation sequence. No convergence is garanteed (see text by Kay or
the Matlab system identification toolbox by Ljung). Or you can use a high
order AR approximation that involves linear equations (see text by
Marple).

A bootstrap method is the system identification approach; estimate
parameters using linear system identification equations, filter to produce
residuals, use residuals to drive estimated ARMA filter, iterate...This is
also not garanteed to converge.

The sub-optimal approach is to estimate the AR parameters using high-order
Yule Walker equations, fitting overdetermined least squares equations by
minimizing squared error. Filter the originalsignal with the inverse AR
filter to produce MA residuals. Estimate MA parameters of residuals.

I hope this helps. I do have Matlab code for this if you are in need. Let
me know.

Dave

On Jan 26, 4:05&#4294967295;pm, "ARMA GUY" <arma_...@yahoo.com> wrote:
> Hi All,
>
> I have asked my boss before now, he said , that determining the AR section
> from an ARMA using Yuke-Walker (and its variants) method is a linear
> approach while determining the MA coefficient is a non linear approach and
> that is the reason for using the sub optimal technique. This fact is
> contained in so many textbooks (e.g Digital; Signal and Image Processing
> using MATLAB, Page 316, " Finally, remember that estimating the
> coefficients of an MA model usually is not simple, since the relations 8.49
> between the model coefficients and the covariances are not linear, whereas
> they are for an AR model..." , also in Biomedical signal processing and
> signal modeling by Eugene N. Bruce pg 381 " ... the coefficients b(k) are
> not linearly related to the auto correlation values as was the case for &#4294967295;AR
> process"
>
> Based on this, he said there should be another method of determining the
> MA coefficients.
>
> So from your earlier response, the w or x in ARMA equation simply means
> the input signal.
>
> ARMA GUY.
>
>
>
> >On 25 Jan, 22:50, "ARMA GUY" <arma_...@yahoo.com> wrote:
> >> Hi All,
>
> >> I am working on ARMA coefficient determination using a data sequence.
> I
> >> have no idea of the underlying system from which the data was generated
> .
> >> What i only have is the data and thsi si as shown in the vector form
> belo=
> >w.
>
> >> If the data is labeled
> >> d =3D [d(1),d(2),d(3),d(4),d(5),d(6),d(7),d(8),d(9),d(10),d(11),......
> >> d(45)]
>
> >> if the ARMA equation is
> >> y(n) =3D a(1)y(n-1) + a(2)y(n-2) + b(0)w(n) + b(1)w(n-1)
>
> >> and the main task is to determine the value of a(1),a(2), b(0) and
> b(1).
>
> >> In order to do that, i am trying to model the data as the output of an
> >> ARMA system driving by white noise, w=3D w(1),w(2),w(3),w(4)...w(n).
>
> >> My questions are
>
> >> 1) Is it logical to assume that w (white noise) above is similar to x
> in
> >> the ARMA equation.
>
> >The noise w seems to be the input signal. Whether you call it 'x' or
> >'w'
> >doesn't matter.
>
> >> 2) If yes to question (1) above, for example can i assume that using
> the
> >> ARMA equation above, d(5) was generated by
>
> >> d(5) =3D a(1)d(4) + a(2)d(3) + b(0)w(5) + b(1)w(4)
>
> >> similarly
>
> >> d(25) =3D a(1)d(24) + a(2)d(23) + b(0)w(25) + b(1)w(24)
>
> >> =A0I have been told not to use any of the least square approach in
> >> determining my ARMA coefficents and not to use any of the sub optimal
> >> technique but to formulate it along that line.
>
> >And your question is...? The first approach would be to
> >ask your boss or supervisor exactly why you are not allowed
> >to use the sub-optimum LMS approaces. Mind you, there is a
> >chance that this is a 'trick question' so judge the answer
> >you get very carefully.
>
> >In the case this is *not* a trick question and there exists an
> >optimum approach that this boss or supervisor knows about, he
> >or she ought to tell you about it. If the boss doesn't know
> >of an optimum non-LMS approach, you ought to review the sub-
> >optimum LMS methods anyway to find out why people use them,
> >and not the optimum non-LMS approaches.
>
> >Basically, your stated task seems a bit dodgy, so everything
> >boils down to whether you trust that your boss knows what
> >he or she is doing. In that case this is a trick question
> >which has a trivial answer where no computations are
> >required.
>
> >Rune

Easiest way is by long division of the AR parameters into 1. This
gives an infinite MA  series whose values get smaller progressively.
It must be a convergent series otherwise the original model will be
unstable. There are no "exacts" in any case since this is all
statistical analysis.

or are you interested in an ARMA model? If so you can use recursive
maximum likelihood or extended least-squares. Also the instrumental
variables method.


Hardy

Hi,

I am not asking you for the solution to the assignment. Perhaps go through
the main question to see my question.

The main question have been answered before now. That is, the x(n) or w(n)
in the equation is the white noise input. 

nevertheless, thanks for answering the main question regarding x(n) or
w(n).

Bye.

>On 26 Jan, 04:05, "ARMA GUY" <arma_...@yahoo.com> wrote:
>
>> Based on this, he said there should be another method of determining
the
>> MA coefficients.
>
>So your task is to find this optimum way of estimating the MA
>parameters? If so, don't expect help here.
>
>First of all, such a method might not be known (if it was,
>the textbooks would mention it). Even if the method exists
>and somebody here knwos about it, they wouldn't do your
>homework for you.
>
>The best you can hope for is a pointer to the literature, if
>anything has been published. So all in all, your best choise
>is to do a literature search yourself.
>
>Rune
>

On 26 Jan, 04:05, "ARMA GUY" <arma_...@yahoo.com> wrote:

> Based on this, he said there should be another method of determining the
> MA coefficients.

So your task is to find this optimum way of estimating the MA
parameters? If so, don't expect help here.

First of all, such a method might not be known (if it was,
the textbooks would mention it). Even if the method exists
and somebody here knwos about it, they wouldn't do your
homework for you.

The best you can hope for is a pointer to the literature, if
anything has been published. So all in all, your best choise
is to do a literature search yourself.

Rune