This is something I am not clear about suppose I have the quadratic function (where ' denotes transpose) J=X'AX and I wish to differentiate this (as in LMS problems) wrt X I know dJ/dX = 2AX (I think) assuming A is symmetric. But how do I diferentiate J=GAG' ie dJ/dG where G is a matrix and A is symmetric like before? Thanks
Vector differentiation problem - LMS type
Started by ●December 21, 2004
Reply by ●December 22, 20042004-12-22
Country_Chiel wrote:> This is something I am not clear about > > suppose I have the quadratic function (where ' denotes transpose) > > J=X'AX > > and I wish to differentiate this (as in LMS problems) wrt X > > I know dJ/dX = 2AX (I think) assuming A is symmetric. > > > But how do I diferentiate > > > J=GAG' ie dJ/dG where G is a matrix and A is symmetric like before? > > ThanksYou probably have to write out the differentiation in the nitty gritty details. For a real-valued vector v and parameter vector a we have dv/da = [dv_1/da_1,dv_2/da_2,...,dv_N,da_N]^T. [1] So d(v'w)/dw = d(w'v)/dw = v. [2] So try to write the expression out in terms of the columns of J, and differentiate each such espression with respect to the columns of G'. I haven't tried these sorts of computations myself, but seeing the symmetry in [2] above, one might hope that the computation ends up with something like dJ/dG' = 2AG'. [3] Rune
Reply by ●December 22, 20042004-12-22
"Rune Allnor" <allnor@tele.ntnu.no> wrote in message news:1103704995.680277.113000@f14g2000cwb.googlegroups.com...> > Country_Chiel wrote: > > This is something I am not clear about > > > > suppose I have the quadratic function (where ' denotes transpose) > > > > J=X'AX > > > > and I wish to differentiate this (as in LMS problems) wrt X > > > > I know dJ/dX = 2AX (I think) assuming A is symmetric. > > > > > > But how do I diferentiate > > > > > > J=GAG' ie dJ/dG where G is a matrix and A is symmetric like before? > > > > Thanks > > You probably have to write out the differentiation in the nitty > gritty details. For a real-valued vector v and parameter vector a > we have > > dv/da = [dv_1/da_1,dv_2/da_2,...,dv_N,da_N]^T. [1] > > So > > d(v'w)/dw = d(w'v)/dw = v. [2] > > So try to write the expression out in terms of the columns of J, > and differentiate each such espression with respect to the > columns of G'. I haven't tried these sorts of computations > myself, but seeing the symmetry in [2] above, one might hope > that the computation ends up with something like > dJ/dG' = 2AG'. [3] > > Rune >Yes I think that is a good guess. They appear to call it vec(matrix) for an individual vector but it gets complicated with Kroneker products etc. Country Chiel