LMS Algorithm

Question 1:

eq. 1)  W(k+1) = W(k) + 2*u*e(k)*X(k)

eq. 2)  W(k+1) = W(k) + u*e(k)*X(k)

I've seen the LMS algorithm written both ways. Which one is correct?
(the difference between 1 & 2 above is multiplying e(k)*X(k) by 2) 

Question 2:

Is there a formula for calculating u, the step size?

"Stacy" <stacy2003_young@yahoo.com> wrote in message 
news:LvOdnZdLmNtsOSfbnZ2dnUVZ_jmdnZ2d@giganews.com...
> Question 1:
>
> eq. 1)  W(k+1) = W(k) + 2*u*e(k)*X(k)
>
> eq. 2)  W(k+1) = W(k) + u*e(k)*X(k)
>
> I've seen the LMS algorithm written both ways. Which one is correct?
> (the difference between 1 & 2 above is multiplying e(k)*X(k) by 2)
>
> Question 2:
>
> Is there a formula for calculating u, the step size?

It's been a long time but ... I think that "u" is a gain factor and not a 
step size as such.  So, whether you decide to multiply by 2 or not is a 
choice you make and may have something to do with details of other 
formulations as a matter of convenience - but probably not necessity.

"u" or "2*u" determines how much of e(k)*X(k) is going to be applied at each 
time step.  The smaller it is, the more time it will take the filter to 
converge and the less resonsive to changes in the input / situation.  If 
it's too large, then the filter may be too time-varying to suit your 
purposes or may not converge.

There are lots of papers written on the subject.

Fred 

>It's been a long time but ... I think that "u" is a gain factor and not a

>step size as such.
> ...
>Fred 

Fred, thank-you for your timely response. 
According to the below excerpt from mathworks they refer to "u" as a step
size rather that a gain factor. Though I understand what you mean.(The
rest of your comment I agree with.)

http://www.mathworks.com/access/helpdesk/help/toolbox/filterdesign/index.html?/access/helpdesk/help

<quote>
Set the Adapting Filter Step Size
The first output of maxstep is the value needed for the mean of the
coefficients to converge while the second is the value needed for the mean
squared coefficients to converge. Choosing a large step size often causes
large variations from the convergence values, so choose smaller step sizes
generally.

halms.stepsize = mumaxmselms/30; % Can be set graphically. 
inspect(halms) % Opens the Property Inspector in MATLAB.
hanlms.stepsize = mumaxmsenlms/20;
inspect(hanlms)If you know the step size to use, you can set the step size
value with the step input argument when you create your filter. 

halms = adaptfilt.lms(n,step); Adds the step input argument.
</quote>


>> eq. 1)  W(k+1) = W(k) + 2*u*e(k)*X(k)
>>
>> eq. 2)  W(k+1) = W(k) + u*e(k)*X(k)
>>
>> I've seen the LMS algorithm written both ways. Which one is correct?
>> (the difference between 1 & 2 above is multiplying e(k)*X(k) by 2)

Granted, one could calculate a step size one half of what is desired and
use eq.1. What I would like to know is how "u" is determined in practice
by people who have done so?

On Aug 9, 3:52 am, "Stacy" <stacy2003_yo...@yahoo.com> wrote:

> Granted, one could calculate a step size one half of what is desired and
> use eq.1. What I would like to know is how "u" is determined in practice
> by people who have done so?

1. Both equations can be right. The step size "u" is an arbitrary
unitless parameter, so you can absorb the 2 into it. You could put any
factor you want in front of that term and just absorb it into the step
size.

2. You can calculate suitable values (and actually the optimum value)
of the step size if you have the correlation matrix of the input
signal. This obviously isn't likely, but an upper bound on the step
size to guarantee convergence can be found if you can find the
eigenvalues of the matrix. Otherwise, I think you might have to
determine it empirically, or use an adaptive-step-size version of the
algorithm.

Jason

I think the factor 2 is introduced here purely for mathematical
convenience.  You will have to derive it from the very beginning to see
the difference but nevertheless it does not carry any physical meaning
here.  I'd suggest you to look at 2*u just as u, which is a scaling
factor.  If you use eq (2), to ensure the convergence, your step size u
has to be within 0 and 2/(maximum eigenvalue of data covariance matrix). 
Many people simply use 2/(trace of data covariance matrix) as the upper
limit.  As long as you are within this interval, larger step size gives
faster convergence but it may not locate the minimum as precise as the
smaller step size.  So many people just use larger step at the beginning
and then turn to smaller step sizes when it approaches the minimum.

HTH,

>Question 1:
>
>eq. 1)  W(k+1) = W(k) + 2*u*e(k)*X(k)
>
>eq. 2)  W(k+1) = W(k) + u*e(k)*X(k)
>
>I've seen the LMS algorithm written both ways. Which one is correct?
>(the difference between 1 & 2 above is multiplying e(k)*X(k) by 2) 
>
>Question 2:
>
>Is there a formula for calculating u, the step size?
>

Stacy wrote:

>
>>>eq. 1)  W(k+1) = W(k) + 2*u*e(k)*X(k)

> Granted, one could calculate a step size one half of what is desired and
> use eq.1. What I would like to know is how "u" is determined in practice
> by people who have done so?

The optimum value of u (which is commonly referred as Beta) depends on 
many factors, and it can be made adaptive also. In practice, it is 
determined by trial and error. The usual numbers are in 1e-2...1e-4 range.

Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

On Aug 9, 5:24 pm, "Stacy" <stacy2003_yo...@yahoo.com> wrote:
> Question 1:
>
> eq. 1)  W(k+1) = W(k) + 2*u*e(k)*X(k)
>
> eq. 2)  W(k+1) = W(k) + u*e(k)*X(k)
>
> I've seen the LMS algorithm written both ways. Which one is correct?
> (the difference between 1 & 2 above is multiplying e(k)*X(k) by 2)
>
> Question 2:
>
> Is there a formula for calculating u, the step size?

Don't use LMS, it's no good for real problems. Instead use normalised
LMS. That way you don't need to worry so much about mu.