sinusoidal parameter estimation

On 19 Aug, 05:52, Andor <andor.bari...@gmail.com> wrote:
> On 18 Aug., 20:39, Dan <danluong...@gmail.com> wrote:
>
> > @Jerry, Andor: I implemented the least squares estimation, and it
> > looks like something with potential, although the matrix inversion may
> > become an issue in implementation in the fixed point DSP down the
> > line.
>
> It may not be necessary to invert a matrix (or solve a linear equation
> system) to find the least-squares estimator using a DSP. Consider
> again your problem: you have measurements y(t) which you assume arise
> from the model
>
> y(t) = a cos(w t) + b sin(w t) + c + measurement_noise(t)
>
> where w is a known frequency, t are the sampling times and the
> additive measurement noise is white. You wish to find (a,b,c).

With just a couple of parameters like the above, one might
get away with a statement that 'no matrix inversion is needed'.

Once the parameters either grow in numbers, or the basis
vectors  become non-orthogonal, like above, the matrix
inversion is needed.

Given the problem of expressing the column vector y in
terms of the columns of X,

Xa = y,

where a is unknown, the LMS solution is formally given as

a = (X'X)^{-1}X'y

where X' means (conjugate) transpose of X. Of course,
there are numerically well-behaved ways of solving such
systems that do not formally count as inversions, but
these methods are computationally at least as expensive
as naive inversion schemes.

Rune

On 19 Aug., 09:23, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 19 Aug, 05:52, Andor <andor.bari...@gmail.com> wrote:
>
>
>
>
>
> > On 18 Aug., 20:39, Dan <danluong...@gmail.com> wrote:
>
> > > @Jerry, Andor: I implemented the least squares estimation, and it
> > > looks like something with potential, although the matrix inversion ma=
y
> > > become an issue in implementation in the fixed point DSP down the
> > > line.
>
> > It may not be necessary to invert a matrix (or solve a linear equation
> > system) to find the least-squares estimator using a DSP. Consider
> > again your problem: you have measurements y(t) which you assume arise
> > from the model
>
> > y(t) =3D a cos(w t) + b sin(w t) + c + measurement_noise(t)
>
> > where w is a known frequency, t are the sampling times and the
> > additive measurement noise is white. You wish to find (a,b,c).
>
> With just a couple of parameters like the above, one might
> get away with a statement that 'no matrix inversion is needed'.
>
> Once the parameters either grow in numbers, or the basis
> vectors =A0become non-orthogonal, like above, the matrix
> inversion is needed.
>
> Given the problem of expressing the column vector y in
> terms of the columns of X,
>
> Xa =3D y,
>
> where a is unknown, the LMS solution is formally given as
>
> a =3D (X'X)^{-1}X'y
>
> where X' means (conjugate) transpose of X. Of course,
> there are numerically well-behaved ways of solving such
> systems that do not formally count as inversions, but
> these methods are computationally at least as expensive
> as naive inversion schemes.

The point is that one can compute the matrix in advance (using a
numerically robust method) then use standard convolution algorithms in
a fixed-point DSP for computing the coefficients a online.

Regards,
Andor

On 19 Aug, 11:31, Andor <andor.bari...@gmail.com> wrote:
> On 19 Aug., 09:23, Rune Allnor <all...@tele.ntnu.no> wrote:
>
>
>
>
>
> > On 19 Aug, 05:52, Andor <andor.bari...@gmail.com> wrote:
>
> > > On 18 Aug., 20:39, Dan <danluong...@gmail.com> wrote:
>
> > > > @Jerry, Andor: I implemented the least squares estimation, and it
> > > > looks like something with potential, although the matrix inversion =
may
> > > > become an issue in implementation in the fixed point DSP down the
> > > > line.
>
> > > It may not be necessary to invert a matrix (or solve a linear equatio=
n
> > > system) to find the least-squares estimator using a DSP. Consider
> > > again your problem: you have measurements y(t) which you assume arise
> > > from the model
>
> > > y(t) =3D a cos(w t) + b sin(w t) + c + measurement_noise(t)
>
> > > where w is a known frequency, t are the sampling times and the
> > > additive measurement noise is white. You wish to find (a,b,c).
>
> > With just a couple of parameters like the above, one might
> > get away with a statement that 'no matrix inversion is needed'.
>
> > Once the parameters either grow in numbers, or the basis
> > vectors =A0become non-orthogonal, like above, the matrix
> > inversion is needed.
>
> > Given the problem of expressing the column vector y in
> > terms of the columns of X,
>
> > Xa =3D y,
>
> > where a is unknown, the LMS solution is formally given as
>
> > a =3D (X'X)^{-1}X'y
>
> > where X' means (conjugate) transpose of X. Of course,
> > there are numerically well-behaved ways of solving such
> > systems that do not formally count as inversions, but
> > these methods are computationally at least as expensive
> > as naive inversion schemes.
>
> The point is that one can compute the matrix in advance (using a
> numerically robust method) then use standard convolution algorithms in
> a fixed-point DSP for computing the coefficients a online.

If you assume that both the frequency and the magnitudes
are known, there is no need to do any computations whatsoever.

Rune

On 19 Aug., 11:42, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 19 Aug, 11:31, Andor <andor.bari...@gmail.com> wrote:
>
>
>
>
>
> > On 19 Aug., 09:23, Rune Allnor <all...@tele.ntnu.no> wrote:
>
> > > On 19 Aug, 05:52, Andor <andor.bari...@gmail.com> wrote:
>
> > > > On 18 Aug., 20:39, Dan <danluong...@gmail.com> wrote:
>
> > > > > @Jerry, Andor: I implemented the least squares estimation, and it
> > > > > looks like something with potential, although the matrix inversio=
n may
> > > > > become an issue in implementation in the fixed point DSP down the
> > > > > line.
>
> > > > It may not be necessary to invert a matrix (or solve a linear equat=
ion
> > > > system) to find the least-squares estimator using a DSP. Consider
> > > > again your problem: you have measurements y(t) which you assume ari=
se
> > > > from the model
>
> > > > y(t) =3D a cos(w t) + b sin(w t) + c + measurement_noise(t)
>
> > > > where w is a known frequency, t are the sampling times and the
> > > > additive measurement noise is white. You wish to find (a,b,c).
>
> > > With just a couple of parameters like the above, one might
> > > get away with a statement that 'no matrix inversion is needed'.
>
> > > Once the parameters either grow in numbers, or the basis
> > > vectors =A0become non-orthogonal, like above, the matrix
> > > inversion is needed.
>
> > > Given the problem of expressing the column vector y in
> > > terms of the columns of X,
>
> > > Xa =3D y,
>
> > > where a is unknown, the LMS solution is formally given as
>
> > > a =3D (X'X)^{-1}X'y
>
> > > where X' means (conjugate) transpose of X. Of course,
> > > there are numerically well-behaved ways of solving such
> > > systems that do not formally count as inversions, but
> > > these methods are computationally at least as expensive
> > > as naive inversion schemes.
>
> > The point is that one can compute the matrix in advance (using a
> > numerically robust method) then use standard convolution algorithms in
> > a fixed-point DSP for computing the coefficients a online.
>
> If you assume that both the frequency and the magnitudes
> are known, there is no need to do any computations whatsoever.

Only the frequency is known. The magnitudes and the offset are
computed online via convolution.

On 19 Aug, 12:03, Andor <andor.bari...@gmail.com> wrote:
> On 19 Aug., 11:42, Rune Allnor <all...@tele.ntnu.no> wrote:

> > If you assume that both the frequency and the magnitudes
> > are known, there is no need to do any computations whatsoever.
>
> Only the frequency is known. The magnitudes and the offset are
> computed online via convolution.

Even if that was true (I can't find where the OP stated
that the frequency is known), you can only get approximate
values for amplitudes of general sinusoidals from convolution.

If you want amplitudes for the non-orthogonal sinusoidals,
you have to solve the LMS problem.

Rune

On 19 Aug., 12:13, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 19 Aug, 12:03, Andor <andor.bari...@gmail.com> wrote:
>
> > On 19 Aug., 11:42, Rune Allnor <all...@tele.ntnu.no> wrote:
> > > If you assume that both the frequency and the magnitudes
> > > are known, there is no need to do any computations whatsoever.
>
> > Only the frequency is known. The magnitudes and the offset are
> > computed online via convolution.
>
> Even if that was true (I can't find where the OP stated
> that the frequency is known),

Dan's post from Datum: Fri, 14 Aug 2009 14:20:54 -0700 (PDT):

"
I see what you are getting at, and yes frequency shift is not likely
in the system I am talking about. The input sinusoid frequency could
be taken as given, and the the problem just becomes one about
calculating the other parameters (amplitude, phase, and offset).
"


> you can only get approximate
> values for amplitudes of general sinusoidals from convolution.

Any estimation technique yields only approximate values.

>
> If you want amplitudes for the non-orthogonal sinusoidals,
> you have to solve the LMS problem.

That's what I'm talking about: solving the LMS problem can be coined
as a convolution of the measurement vector y with the rows in the
Moore-Penrose pseudo-inverse

(X'X)^{-1}X'

if the design matrix X is known, and therefore can be precomputed,
beforehand (which it is, given we know the frequency w).

Regards,
Andor