Can anyone give some pointers on estimating observation noise variance from an observed set. I have an process (ARMA) that I am estimating the parameters of (blind estimation, I don't know the input sequence, I only see the output), it would be useful to be able to have an estimate of the variance of the added white Gaussian observation noise. My understanding was that the following would work, but my understanding of this is sparse (can't find the reference for this anymore): Estimate the covariance matrix of the signal Ryy = 1/N sum over i (y(i)*y(i)'), i=1 to N, for many N for some block of observed data y(i), where the length of y(i) is longer than the AR and MA filters of the original process. then take the SVD: USV' = Ryy, and my understanding is that the variance of the added noise should come out in the lower singular values, but despite tinkering with this, I haven't had any luck. Am I on the right track? Can anyone suggest any other method, I'll take anything that gets me the noise variance? Thanks, Chris

# Estimate Observation Noise Variance

Started by ●June 23, 2008

Reply by ●June 23, 20082008-06-23

On 23 Jun, 19:35, Chris Maryan <kmar...@gmail.com> wrote:> Can anyone give some pointers on estimating observation noise variance > from an observed set. > > I have an process (ARMA) that I am estimating the parameters of (blind > estimation, I don't know the input sequence, I only see the output), > it would be useful to be able to have an estimate of the variance of > the added white Gaussian observation noise. > > My understanding was that the following would work, but my > understanding of this is sparse (can't find the reference for this > anymore): > > Estimate the covariance matrix of the signal > Ryy = 1/N sum over i (y(i)*y(i)'), i=1 to N, for many N > for some block of observed data y(i), where the length of y(i) is > longer than the AR and MA filters of the original process. > > then take the SVD: USV' = Ryy, and my understanding is that the > variance of the added noise should come out in the lower singular > values, but despite tinkering with this, I haven't had any luck. > > Am I on the right track?You are certainly on the right track, at least for AR and sum-of-sine models. This is the basis for MUSIC, ESPRIT and those sorts of methods. One problem with most derivations of MUSIC & al is that they ignore certain 1/N scaling terms. If you find a good derivation of MUSIC and retain the scaling terms, you might be able to estimate the noise variance.> Can anyone suggest any other method, I'll > take anything that gets me the noise variance?Another method would be the Levinson recursion to find the prediction filter for the AR process. The prediction residual will be the noise. Square it to find the variance. Rune

Reply by ●June 23, 20082008-06-23

On Jun 24, 5:35 am, Chris Maryan <kmar...@gmail.com> wrote:> Can anyone give some pointers on estimating observation noise variance > from an observed set. > > I have an process (ARMA) that I am estimating the parameters of (blind > estimation, I don't know the input sequence, I only see the output), > it would be useful to be able to have an estimate of the variance of > the added white Gaussian observation noise. > > My understanding was that the following would work, but my > understanding of this is sparse (can't find the reference for this > anymore): > > Estimate the covariance matrix of the signal > Ryy = 1/N sum over i (y(i)*y(i)'), i=1 to N, for many N > for some block of observed data y(i), where the length of y(i) is > longer than the AR and MA filters of the original process. > > then take the SVD: USV' = Ryy, and my understanding is that the > variance of the added noise should come out in the lower singular > values, but despite tinkering with this, I haven't had any luck. > > Am I on the right track? Can anyone suggest any other method, I'll > take anything that gets me the noise variance? > > Thanks, > > ChrisI am not convinced. When you add (say) white noise to an ARM or AR process you get another process called an innovations model driven by the innovations sequency. You cannot isolate the measurement noise variance from this since it is combined via a spectral factorisation equation of polynomials. You can however estimate the ratio of measurement noise variance to innovation noise variance. Suppose we have an AR process a(z^-1)Y(k)=u(k) where u has variance n adding white noise to Y(k) = (1/a)u(k) gives s(k)=Y(k)+V(k) where V is the noise then we get s(k)=(d/a)eps(k) where eps(k) is innovations sequence and dd*re=aa*r+n is a spectral factorization. and re is innovations variance, r is measurement noise variance,n is driving noise variance. * is such that a(z^-1)* =a(z). You can show dn/an=r/re where dn and an are the nth element of the polynomials in a and d. I know this is vague but this stuff is well known. K. K.