Hi all, w(n+1) = w(n) + mu*e(n)*x(n) Can anyone tell me how to determine the maximum mu? I read some materials that mu should be less than 2/lambda_max, where lambda_max is the maximum eigen value of the autocorrelation matrix of the input. Take the 50Hz noise cancellation filter in Widrow's paper as an example, "Adaptive Noise Cancelling : Principles and Applications". How to determine the maximum mu? Thanks, cfy30
LMS maximum step size
Started by ●July 12, 2010
Reply by ●July 14, 20102010-07-14
On Jul 12, 3:37�pm, "cfy30" <cfy30@n_o_s_p_a_m.yahoo.com> wrote:> Hi all, > > w(n+1) = w(n) + mu*e(n)*x(n) > > Can anyone tell me how to determine the maximum mu? I read some materials > that mu should be less than 2/lambda_max, where lambda_max is the maximum > eigen value of the autocorrelation matrix of the input. > > Take the 50Hz noise cancellation filter in Widrow's paper as an example, > "Adaptive Noise Cancelling : Principles and Applications". How to determine > the maximum mu? > > Thanks, > cfy30I could tell you but then I would have to shoot you. Best to use NLMS. Hardy
Reply by ●July 14, 20102010-07-14
On Jul 13, 11:10�pm, HardySpicer <gyansor...@gmail.com> wrote:> > I could tell you but then I would have to shoot you....> Best to use NLMS.so now are you gonna shoot him? r b-j
Reply by ●July 14, 20102010-07-14
On Jul 11, 10:37�pm, "cfy30" <cfy30@n_o_s_p_a_m.yahoo.com> wrote:> Hi all, > > w(n+1) = w(n) + mu*e(n)*x(n) > > Can anyone tell me how to determine the maximum mu? I read some materials > that mu should be less than 2/lambda_max, where lambda_max is the maximum > eigen value of the autocorrelation matrix of the input. > > Take the 50Hz noise cancellation filter in Widrow's paper as an example, > "Adaptive Noise Cancelling : Principles and Applications". How to determine > the maximum mu? > > Thanks, > cfy30For the LMS algorithm: 1. Define a weight-error vector 2. Write the weight-error vector as a recursive algorithm in terms of the auto-correlation matrix of the input 3. Perform a unitary-similarity transformation on the auto-correlation matrix 4. Rotate the weight-error coordinate system using the eigenvector matrix of the unitarity-similarity transformation 5. Observe what must occur for convergence 6. QED Or, you could spend some time and look it up in one of several books: [1] Clarkson, Peter M. "Optimal and Adaptive Signal Processing" [2] Widrow, Bernard, and Stearns, Samuel, "Adaptive Signal Processing" [3] Hatken, Simon, "Adaptive Filter Theory" [4] Graupe, daniel, "Time Series Analysis" If you look at the normalized LMS (NLMS), then the maximum mu is limited to the sum of the number of filter coefficients. That proof I will leave to you. Maurice Givens
Reply by ●July 14, 20102010-07-14
maury wrote:> If you look at the normalized LMS (NLMS), then the maximum mu is > limited to the sum of the number of filter coefficients.IMO what does matter is optimim rather then maximum. You need to minimize the total error of the process. This error is because of imperfect adaptation, noisy gradients, nonlinearity, ambient noise and numeric artifacts. It also depends on the statistics and power of the reference signal. So the algorithm must adapt Mu in pseudo Kalman way. Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
Reply by ●July 14, 20102010-07-14
On Jul 14, 9:35�am, Vladimir Vassilevsky <nos...@nowhere.com> wrote:> maury wrote: > > If you look at the normalized LMS (NLMS), then the maximum mu is > > limited to the sum of the number of filter coefficients. > > IMO what does matter is optimim rather then maximum. You need to > minimize the total error of the process. This error is because of > imperfect adaptation, noisy gradients, nonlinearity, ambient noise and > numeric artifacts. It also depends on the statistics and power of the > reference signal. So the algorithm must adapt Mu in pseudo Kalman way. > > Vladimir Vassilevsky > DSP and Mixed Signal Design Consultanthttp://www.abvolt.comHi Vlad, With respect to the NLMS, the effect of the power of the reference depends on the implmentation. For example, in a system idemtification implementation, the power of the reference is not important since the normalization of the LMS is performed over the signal power. The algorithm is self-adjusting for imput power. However!!! The performance is very dependent on the model assumed. By nonlinearity, I take you to mean that a linear model will have errors if nonlinearities are present in the reference signal (the reference being some nonlinear inner-product of the signal and system vector). This just means the wrong model was chosen. If your model matches the system, then the squared error will not be great. No matter what the model, the algorithm will give the minimum squared error for the model. This doesn't mean that the coefficients you end up with are close to the desired coefficients. The other important parameter is the condition of the input auto- correlation matrix. An ill-conditioned matrix will result in not being able to determine an optimum estimate of the system impulse response. This is the problem with the example used by the OP. An input of 50 Hz (single signal) will result in an ill-conditioned input auto- correlartion matrix. There is NO optimum solution in this case. In fact, there are an infinite number of solutions to this problem. But, your point is well taken that the maximum mu used will be based on the optimum performance required from the problem. That performance may be minimum misadjustment noise, or maximum speed of convergence. Each will ditate a different "maximum" mu. What I think the OP was looking for, is how to determine the maximum mu for convergence (often equated with stability). This value is different for LMS versus NLMS. Maurice
Reply by ●July 14, 20102010-07-14
maury wrote:> On Jul 14, 9:35 am, Vladimir Vassilevsky <nos...@nowhere.com> wrote: > >>maury wrote: >> >>>If you look at the normalized LMS (NLMS), then the maximum mu is >>>limited to the sum of the number of filter coefficients. >> >>IMO what does matter is optimim rather then maximum. You need to >>minimize the total error of the process. This error is because of >>imperfect adaptation, noisy gradients, nonlinearity, ambient noise and >>numeric artifacts. It also depends on the statistics and power of the >>reference signal. So the algorithm must adapt Mu in pseudo Kalman way. >> >>Vladimir Vassilevsky >>DSP and Mixed Signal Design Consultanthttp://www.abvolt.com > > > Hi Vlad, > With respect to the NLMS, the effect of the power of the reference > depends on the implmentation. For example, in a system idemtification > implementation, the power of the reference is not important since the > normalization of the LMS is performed over the signal power. The > algorithm is self-adjusting for imput power.Yes, but the input is catching the ambient noise and double talk besides the reflected signal. Hence if the input power is low, the algorithm should not crank Mu all the way up to infinity, as simple NLMS would do, but do the exactly the opposite.> However!!! The performance is very dependent on the model assumed. By > nonlinearity, I take you to mean that a linear model will have errors > if nonlinearities are present in the reference signal (the reference > being some nonlinear inner-product of the signal and system vector). > This just means the wrong model was chosen. If your model matches the > system, then the squared error will not be great.System is given to you. Your algorithm has to work with whatever you got. Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
Reply by ●July 14, 20102010-07-14
On Jul 14, 2:07�pm, Vladimir Vassilevsky <nos...@nowhere.com> wrote:> System is given to you. Your algorithm has to work with whatever you got.Kobayashi maru solution!
Reply by ●July 14, 20102010-07-14
On 7/14/2010 4:38 PM, maury wrote:> On Jul 14, 2:07 pm, Vladimir Vassilevsky<nos...@nowhere.com> wrote: > >> System is given to you. Your algorithm has to work with whatever you got. > > Kobayashi maru solution!Hah! another Trekke! -- Engineering is the art of making what you want from things you can get.
Reply by ●July 14, 20102010-07-14
On Jul 14, 8:07�pm, Vladimir Vassilevsky <nos...@nowhere.com> wrote:> maury wrote: > > On Jul 14, 9:35 am, Vladimir Vassilevsky <nos...@nowhere.com> wrote: > > >>maury wrote: > > >>>If you look at the normalized LMS (NLMS), then the maximum mu is > >>>limited to the sum of the number of filter coefficients. > > >>IMO what does matter is optimim rather then maximum. You need to > >>minimize the total error of the process. This error is because of > >>imperfect adaptation, noisy gradients, nonlinearity, ambient noise and > >>numeric artifacts. It also depends on the statistics and power of the > >>reference signal. So the algorithm must adapt Mu in pseudo Kalman way. > > >>Vladimir Vassilevsky > >>DSP and Mixed Signal Design Consultanthttp://www.abvolt.com > > > Hi Vlad, > > With respect to the NLMS, the effect of the power of the reference > > depends on the implmentation. For example, in a system idemtification > > implementation, the power of the reference is not important since the > > normalization of the LMS is performed over the signal power. The > > algorithm is self-adjusting for imput power. > > Yes, but the input is catching the ambient noise and double talk besides > � the reflected signal. Hence if the input power is low, the algorithm > should not crank Mu all the way up to infinity, as simple NLMS would do, > but do the exactly the opposite. > > > However!!! �The performance is very dependent on the model assumed. By > > nonlinearity, I take you to mean that a linear model will have errors > > if nonlinearities are present in the reference signal (the reference > > being some nonlinear inner-product of the signal and system vector). > > This just means the wrong model was chosen. If your model matches the > > system, then the squared error will not be great. > > System is given to you. Your algorithm has to work with whatever you got.What if you discovered by sheer luck that doing something wanky like skipping a beat works wonders to your signal? Sure this is uncool, inelegant, and can't be analyzed, only characterized. Do you scrap all this and wake up next day pretending that nothing has happened? -Momo
