Confused with Notation

So some of the papers I am looking at represent convolution in 'matrix'
form in one of two ways:

y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
and
y[n] = H^t * X where H^t = transpose of filter H

Why is the first formula conjugated? 

To get a better sense of what I mean..incase my notation above is confusing
you.

I am trying to understand why on wikipedia the formula for the error of the
RLS algorithm (http://en.wikipedia.org/wiki/Recursive_least_squares_filter
- located at end under "RLS SUMMARY" section) is
e(n) = d(n) - H(n-1)^t * x(n)

while here
(http://www.researchusa.org/uploads/avi/pVPGuUnUhxGgcpr0UdAn.pdf) on page 4
the formula is
e(n) = d(n) - H(n-1)^H * x(n)

The equations in these two papers seem to be the exact same..but they
differ just on whether H should be conjugated..anyone know why?

On Apr 17, 7:09&#4294967295;pm, "firepluswater" <ampierce@n_o_s_p_a_m.gmail.com>
wrote:
> So some of the papers I am looking at represent convolution in 'matrix'
> form in one of two ways:
>
> y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
> and
> y[n] = H^t * X where H^t = transpose of filter H
>
> Why is the first formula conjugated?
>
> To get a better sense of what I mean..incase my notation above is confusing
> you.
>
> I am trying to understand why on wikipedia the formula for the error of the
> RLS algorithm (http://en.wikipedia.org/wiki/Recursive_least_squares_filter
> - located at end under "RLS SUMMARY" section) is
> e(n) = d(n) - H(n-1)^t * x(n)
>
> while here
> (http://www.researchusa.org/uploads/avi/pVPGuUnUhxGgcpr0UdAn.pdf) on page 4
> the formula is
> e(n) = d(n) - H(n-1)^H * x(n)
>
> The equations in these two papers seem to be the exact same..but they
> differ just on whether H should be conjugated..anyone know why?

If it is complex then you use conjugate transpose. To be honest, a lot
of poser engineer/maths types put Hermitian
when it is not actually needed but try and be general. For real data
you don't need Hermitian. In some comms problems however
you have I and q data so the problem becomes complex. For the rest of
us who use real data then it is an unnescessary complication!


Hardy

On 17 apr, 09:09, "firepluswater" <ampierce@n_o_s_p_a_m.gmail.com>
wrote:
> So some of the papers I am looking at represent convolution in 'matrix'
> form in one of two ways:
>
> y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
> and
> y[n] = H^t * X where H^t = transpose of filter H
>
> Why is the first formula conjugated?
>
> To get a better sense of what I mean..incase my notation above is confusing
> you.

The notation is standard.

You need the conjugate if you work with complex-valued data.
It's a property of the inner product between complex-valued
vectors. For real-valued X the Hermitian X^H and the
transpose X^T are equal.

Unfortunately, a lot of authors, both on DSP and on linear
algebra, only write the real-valued form when they derive
results on matrix form. There are cases where one needs to
treat the complex-valued case somewhat differently than the
real-valued case, that thus are missing from the textbooks.

In case you need to 'complexify' a real-valued expression,
you might want to be aware that there might be more to it
than merely substituting superscript H for superscript T.

Rune

HardySpicer wrote:
> On Apr 17, 7:09 pm, "firepluswater" <ampierce@n_o_s_p_a_m.gmail.com>
> wrote:
>> So some of the papers I am looking at represent convolution in 'matrix'
>> form in one of two ways:
>>
>> y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
>> and
>> y[n] = H^t * X where H^t = transpose of filter H
>>
>> Why is the first formula conjugated?
>>
>> To get a better sense of what I mean..incase my notation above is confusing
>> you.
>>
>> I am trying to understand why on wikipedia the formula for the error of the
>> RLS algorithm (http://en.wikipedia.org/wiki/Recursive_least_squares_filter
>> - located at end under "RLS SUMMARY" section) is
>> e(n) = d(n) - H(n-1)^t * x(n)
>>
>> while here
>> (http://www.researchusa.org/uploads/avi/pVPGuUnUhxGgcpr0UdAn.pdf) on page 4
>> the formula is
>> e(n) = d(n) - H(n-1)^H * x(n)
>>
>> The equations in these two papers seem to be the exact same..but they
>> differ just on whether H should be conjugated..anyone know why?
> 
> If it is complex then you use conjugate transpose. To be honest, a lot
> of poser engineer/maths types put Hermitian
> when it is not actually needed but try and be general. For real data
> you don't need Hermitian. In some comms problems however
> you have I and q data so the problem becomes complex. For the rest of
> us who use real data then it is an unnescessary complication!

Is it poser or poseur?  Would I be a poser to say 'poseur' without 
knowing French?  Or would I be a poseur to say 'poser' without being a 
rocker?

(I have listened to Doug & Dave -- does that mean I can say 'poser'?)

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com

On Apr 17, 10:01&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 17 apr, 09:09, "firepluswater" <ampierce@n_o_s_p_a_m.gmail.com>
> wrote:
>
> > So some of the papers I am looking at represent convolution in 'matrix'
> > form in one of two ways:
>
> > y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
> > and
> > y[n] = H^t * X where H^t = transpose of filter H
>
> > Why is the first formula conjugated?
>
> > To get a better sense of what I mean..incase my notation above is confusing
> > you.
>
> The notation is standard.
>
> You need the conjugate if you work with complex-valued data.
> It's a property of the inner product between complex-valued
> vectors. For real-valued X the Hermitian X^H and the
> transpose X^T are equal.
>
> Unfortunately, a lot of authors, both on DSP and on linear
> algebra, only write the real-valued form when they derive
> results on matrix form. There are cases where one needs to
> treat the complex-valued case somewhat differently than the
> real-valued case, that thus are missing from the textbooks.
>
> In case you need to 'complexify' a real-valued expression,
> you might want to be aware that there might be more to it
> than merely substituting superscript H for superscript T.
>
> Rune

All true but for begiiners books it doesn't help one biy. In control
theory we used to use
'*' to denote Hermition transpose. Looked a little simpler than
raising to the power H!

Hardy

On 18 apr, 04:56, HardySpicer <gyansor...@gmail.com> wrote:
> On Apr 17, 10:01&#4294967295;pm, Rune Allnor <all...@tele.ntnu.no> wrote:
>
>
>
>
>
> > On 17 apr, 09:09, "firepluswater" <ampierce@n_o_s_p_a_m.gmail.com>
> > wrote:
>
> > > So some of the papers I am looking at represent convolution in 'matrix'
> > > form in one of two ways:
>
> > > y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
> > > and
> > > y[n] = H^t * X where H^t = transpose of filter H
>
> > > Why is the first formula conjugated?
>
> > > To get a better sense of what I mean..incase my notation above is confusing
> > > you.
>
> > The notation is standard.
>
> > You need the conjugate if you work with complex-valued data.
> > It's a property of the inner product between complex-valued
> > vectors. For real-valued X the Hermitian X^H and the
> > transpose X^T are equal.
>
> > Unfortunately, a lot of authors, both on DSP and on linear
> > algebra, only write the real-valued form when they derive
> > results on matrix form. There are cases where one needs to
> > treat the complex-valued case somewhat differently than the
> > real-valued case, that thus are missing from the textbooks.
>
> > In case you need to 'complexify' a real-valued expression,
> > you might want to be aware that there might be more to it
> > than merely substituting superscript H for superscript T.
>
> > Rune
>
> All true but for begiiners books it doesn't help one biy. In control
> theory we used to use
> '*' to denote Hermition transpose. Looked a little simpler than
> raising to the power H!

The key is to know the difference between the transposed and
the conjugate transposed. As for the way to express the difference
in print, this is not the worst. The Newton vs Leibnitz ways
of expressing differentials is far more important. Newton's
way of expressing differentials in terms of primes and dots might
seem simple at first, but does not generalize very well.

And it confuses beginners. I remember a physics class in high school,
where we were given some function

f = abc

and asked to find its derivative f'.

I had no idea how to do that, since I had no idea what variable to
differentiate against. A couple of years later I learned the
notation

df / da

and realized that the task above would have been trivial.

Don't underestimate your students. They only *know* less than
yourself; their intellectual capacities might be way beyond yours.

Rune

Rune Allnor <allnor@tele.ntnu.no> wrote:
(snip, someone wrote)

>> All true but for begiiners books it doesn't help one biy. In control
>> theory we used to use
>> '*' to denote Hermition transpose. Looked a little simpler than
>> raising to the power H!
 
> The key is to know the difference between the transposed and
> the conjugate transposed. As for the way to express the difference
> in print, this is not the worst. The Newton vs Leibnitz ways
> of expressing differentials is far more important. Newton's
> way of expressing differentials in terms of primes and dots might
> seem simple at first, but does not generalize very well.

They are often used in physics for the cases where the
generalization isn't needed.  In many problems only time
derivatives are needed, and the dot notation works well.
 
> And it confuses beginners. I remember a physics class in high 
> school, where we were given some function
 
> f = abc
 
> and asked to find its derivative f'.
 
> I had no idea how to do that, since I had no idea what variable to
> differentiate against. A couple of years later I learned the
> notation
 
> df / da
 
> and realized that the task above would have been trivial.

Somehow this reminds me of exactly when I understood partial
derivatives.  I had heard somewhere along the way what they
were, but hadn't actually needed them.  Then I had a physics
quiz problem, which was to show that f(x-vt) and g(x+vt)
are solutions to the wave equation, as usual written with
partial derivatives.  So, having never done a partial
derivative before, I figured out how to get it to work.

I even still remember the pink paper that the quizzes were
printed on.  

So, yes there are many problems that are easier to do with 
the df/da notation, such as separation of variables problems.

Otherwise, you can always write

f'=abc'+ab'c+a'bc, where the ' means derivative with respect
to whichever variable is needed.  Then again, maybe

df=a b dc + a db c + da b c   

isn't so bad, either.  Though one does have to be careful 
to indicate which is multiplication.
 
> Don't underestimate your students. They only *know* less than
> yourself; their intellectual capacities might be way beyond yours.

-- glen

On Apr 18, 9:45&#4294967295;pm, glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
> Rune Allnor <all...@tele.ntnu.no> wrote:
>
> (snip, someone wrote)
>
> >> All true but for begiiners books it doesn't help one biy. In control
> >> theory we used to use
> >> '*' to denote Hermition transpose. Looked a little simpler than
> >> raising to the power H!
> > The key is to know the difference between the transposed and
> > the conjugate transposed. As for the way to express the difference
> > in print, this is not the worst. The Newton vs Leibnitz ways
> > of expressing differentials is far more important. Newton's
> > way of expressing differentials in terms of primes and dots might
> > seem simple at first, but does not generalize very well.
>
> They are often used in physics for the cases where the
> generalization isn't needed. &#4294967295;In many problems only time
> derivatives are needed, and the dot notation works well.
>
> > And it confuses beginners. I remember a physics class in high
> > school, where we were given some function
> > f = abc
> > and asked to find its derivative f'.
> > I had no idea how to do that, since I had no idea what variable to
> > differentiate against. A couple of years later I learned the
> > notation
> > df / da
> > and realized that the task above would have been trivial.
>
> Somehow this reminds me of exactly when I understood partial
> derivatives. &#4294967295;I had heard somewhere along the way what they
> were, but hadn't actually needed them. &#4294967295;Then I had a physics
> quiz problem, which was to show that f(x-vt) and g(x+vt)
> are solutions to the wave equation, as usual written with
> partial derivatives. &#4294967295;So, having never done a partial
> derivative before, I figured out how to get it to work.
>
> I even still remember the pink paper that the quizzes were
> printed on. &#4294967295;
>
> So, yes there are many problems that are easier to do with
> the df/da notation, such as separation of variables problems.
>
> Otherwise, you can always write
>
> f'=abc'+ab'c+a'bc, where the ' means derivative with respect
> to whichever variable is needed. &#4294967295;Then again, maybe
>
> df=a b dc + a db c + da b c &#4294967295;
>
> isn't so bad, either. &#4294967295;Though one does have to be careful
> to indicate which is multiplication.
>
> > Don't underestimate your students. They only *know* less than
> > yourself; their intellectual capacities might be way beyond yours.
>
> -- glen

Now try differentiating a matrix expression wrt a matrix!!
This is where it gets interesting. There are a few methods.

eg F=ABC

where A,B,C are all matrices.

find dF/dB

You get a matrix of matrices..

Hardy

On Apr 17, 3:43&#4294967295;pm, Tim Wescott <t...@seemywebsite.now> wrote:
> HardySpicer wrote:
> > On Apr 17, 7:09 pm, "firepluswater" <ampierce@n_o_s_p_a_m.gmail.com>
> > wrote:
> >> So some of the papers I am looking at represent convolution in 'matrix'
> >> form in one of two ways:
>
> >> y[n] = H^H * X where H^H = hermitian (conj transpose) of filter H
> >> and
> >> y[n] = H^t * X where H^t = transpose of filter H
>
> >> Why is the first formula conjugated?
>
> >> To get a better sense of what I mean..incase my notation above is confusing
> >> you.
>
> >> I am trying to understand why on wikipedia the formula for the error of the
> >> RLS algorithm (http://en.wikipedia.org/wiki/Recursive_least_squares_filter
> >> - located at end under "RLS SUMMARY" section) is
> >> e(n) = d(n) - H(n-1)^t * x(n)
>
> >> while here
> >> (http://www.researchusa.org/uploads/avi/pVPGuUnUhxGgcpr0UdAn.pdf) on page 4
> >> the formula is
> >> e(n) = d(n) - H(n-1)^H * x(n)
>
> >> The equations in these two papers seem to be the exact same..but they
> >> differ just on whether H should be conjugated..anyone know why?
>
> > If it is complex then you use conjugate transpose. To be honest, a lot
> > of poser engineer/maths types put Hermitian
> > when it is not actually needed but try and be general. For real data
> > you don't need Hermitian. In some comms problems however
> > you have I and q data so the problem becomes complex. For the rest of
> > us who use real data then it is an unnescessary complication!
>
> Is it poser or poseur? &#4294967295;Would I be a poser to say 'poseur' without
> knowing French? &#4294967295;Or would I be a poseur to say 'poser' without being a
> rocker?
>
> (I have listened to Doug & Dave -- does that mean I can say 'poser'?)
>
> --
> Tim Wescott
> Control system and signal processing consultingwww.wescottdesign.com- Hide quoted text -
>
> - Show quoted text -

I thought they were waiting to have their portraits made ;-)

Clay