"Rune Allnor" <allnor@tele.ntnu.no> wrote in message
news:1103704995.680277.113000@f14g2000cwb.googlegroups.com...
>
> Country_Chiel wrote:
> > This is something I am not clear about
> >
> > suppose I have the quadratic function (where ' denotes transpose)
> >
> > J=X'AX
> >
> > and I wish to differentiate this (as in LMS problems) wrt X
> >
> > I know dJ/dX = 2AX (I think) assuming A is symmetric.
> >
> >
> > But how do I diferentiate
> >
> >
> > J=GAG'  ie dJ/dG where G is a matrix and A is symmetric like before?
> >
> > Thanks
>
> You probably have to write out the differentiation in the nitty
> gritty details. For a real-valued vector v and parameter vector a
> we have
>
> dv/da = [dv_1/da_1,dv_2/da_2,...,dv_N,da_N]^T.              [1]
>
> So
>
> d(v'w)/dw = d(w'v)/dw = v.                                  [2]
>
> So try to write the expression out in terms of the columns of J,
> and differentiate each such espression with respect to the
> columns of G'. I haven't tried these sorts of computations
> myself, but seeing the symmetry in [2] above, one might hope
> that the computation ends up with something like
> dJ/dG' = 2AG'.                                              [3]
>
> Rune
>
Yes I think that is a good guess. They appear to call it vec(matrix) for an
individual vector but it gets complicated with Kroneker products etc.

Country Chiel

Country_Chiel wrote:
> This is something I am not clear about
>
> suppose I have the quadratic function (where ' denotes transpose)
>
> J=X'AX
>
> and I wish to differentiate this (as in LMS problems) wrt X
>
> I know dJ/dX = 2AX (I think) assuming A is symmetric.
>
>
> But how do I diferentiate
>
>
> J=GAG'  ie dJ/dG where G is a matrix and A is symmetric like before?
>
> Thanks

You probably have to write out the differentiation in the nitty
gritty details. For a real-valued vector v and parameter vector a
we have

dv/da = [dv_1/da_1,dv_2/da_2,...,dv_N,da_N]^T.              [1]

So

d(v'w)/dw = d(w'v)/dw = v.                                  [2]

So try to write the expression out in terms of the columns of J,
and differentiate each such espression with respect to the
columns of G'. I haven't tried these sorts of computations
myself, but seeing the symmetry in [2] above, one might hope
that the computation ends up with something like
dJ/dG' = 2AG'.                                              [3]

Rune

This is something I am not clear about

suppose I have the quadratic function (where ' denotes transpose)

J=X'AX

and I wish to differentiate this (as in LMS problems) wrt X

I know dJ/dX = 2AX (I think) assuming A is symmetric.


But how do I diferentiate


J=GAG'  ie dJ/dG where G is a matrix and A is symmetric like before?

Thanks