Estimate Observation Noise Variance

Can anyone give some pointers on estimating observation noise variance
from an observed set.

I have an process (ARMA) that I am estimating the parameters of (blind
estimation, I don't know the input sequence, I only see the output),
it would be useful to be able to have an estimate of the variance of
the added white Gaussian observation noise.

My understanding was that the following would work, but my
understanding of this is sparse (can't find the reference for this
anymore):

Estimate the covariance matrix of the signal
Ryy = 1/N sum over i (y(i)*y(i)'), i=1 to N, for many N
for some block of observed data y(i), where the length of y(i) is
longer than the AR and MA filters of the original process.

then take the SVD: USV' = Ryy, and my understanding is that the
variance of the added noise should come out in the lower singular
values, but despite tinkering with this, I haven't had any luck.

Am I on the right track? Can anyone suggest any other method, I'll
take anything that gets me the noise variance?

Thanks,

Chris

On 23 Jun, 19:35, Chris Maryan <kmar...@gmail.com> wrote:
> Can anyone give some pointers on estimating observation noise variance
> from an observed set.
>
> I have an process (ARMA) that I am estimating the parameters of (blind
> estimation, I don't know the input sequence, I only see the output),
> it would be useful to be able to have an estimate of the variance of
> the added white Gaussian observation noise.
>
> My understanding was that the following would work, but my
> understanding of this is sparse (can't find the reference for this
> anymore):
>
> Estimate the covariance matrix of the signal
> Ryy = 1/N sum over i (y(i)*y(i)'), i=1 to N, for many N
> for some block of observed data y(i), where the length of y(i) is
> longer than the AR and MA filters of the original process.
>
> then take the SVD: USV' = Ryy, and my understanding is that the
> variance of the added noise should come out in the lower singular
> values, but despite tinkering with this, I haven't had any luck.
>
> Am I on the right track?

You are certainly on the right track, at least for AR
and sum-of-sine models. This is the basis for MUSIC,
ESPRIT and those sorts of methods.

One problem with most derivations of MUSIC & al
is that they ignore certain 1/N scaling terms.
If you find a good derivation of MUSIC and
retain the scaling terms, you might be able
to estimate the noise variance.

> Can anyone suggest any other method, I'll
> take anything that gets me the noise variance?

Another method would be the Levinson recursion
to find the prediction filter for the AR process.
The prediction residual will be the noise. Square
it to find the variance.

Rune

On Jun 24, 5:35 am, Chris Maryan <kmar...@gmail.com> wrote:
> Can anyone give some pointers on estimating observation noise variance
> from an observed set.
>
> I have an process (ARMA) that I am estimating the parameters of (blind
> estimation, I don't know the input sequence, I only see the output),
> it would be useful to be able to have an estimate of the variance of
> the added white Gaussian observation noise.
>
> My understanding was that the following would work, but my
> understanding of this is sparse (can't find the reference for this
> anymore):
>
> Estimate the covariance matrix of the signal
> Ryy = 1/N sum over i (y(i)*y(i)'), i=1 to N, for many N
> for some block of observed data y(i), where the length of y(i) is
> longer than the AR and MA filters of the original process.
>
> then take the SVD: USV' = Ryy, and my understanding is that the
> variance of the added noise should come out in the lower singular
> values, but despite tinkering with this, I haven't had any luck.
>
> Am I on the right track? Can anyone suggest any other method, I'll
> take anything that gets me the noise variance?
>
> Thanks,
>
> Chris

I am not convinced. When you add (say) white noise to an ARM or AR
process you get another process called an innovations model driven by
the innovations sequency. You cannot isolate the measurement noise
variance from this since it is combined via a spectral factorisation
equation of polynomials. You can however estimate the ratio of
measurement noise variance to innovation noise variance. Suppose we
have an AR process

a(z^-1)Y(k)=u(k) where u has variance n

adding white noise to Y(k) = (1/a)u(k) gives

s(k)=Y(k)+V(k) where V is the noise

then we get

s(k)=(d/a)eps(k)

where eps(k) is innovations sequence and

dd*re=aa*r+n is a spectral factorization.

and re is innovations variance, r is measurement noise variance,n is
driving noise variance.

* is such that a(z^-1)* =a(z).
You can show

dn/an=r/re

where dn and an are the nth element of the polynomials in a and d. I
know this is vague but this stuff is well known.


K.



K.