The Least Mean Squares (LMS) algorithm is an adaptive filter algorithm that iteratively updates filter coefficients to minimize the mean squared error between a filter's output and a desired reference signal. It belongs to the stochastic gradient descent family and is widely used in signal processing applications where the optimal filter coefficients are unknown or time-varying.
In practice
LMS is used in embedded DSP applications such as acoustic echo cancellation, active noise control, channel equalization, and system identification. The core update rule is: w(n+1) = w(n) + mu * e(n) * x(n), where w is the coefficient vector, mu is the step size, e(n) is the error signal (desired minus actual output), and x(n) is the input signal vector. (Some derivations include a factor of 2 that is absorbed into mu; the form above follows the convention most common in implementation references.) This arithmetic simplicity makes LMS attractive on resource-constrained hardware where more computationally demanding algorithms like RLS (Recursive Least Squares) are too expensive.
The step size mu is the most critical tuning parameter. Too large a value causes the filter to diverge or oscillate; too small a value produces slow convergence. For a filter with N taps and input signal power P, a commonly cited rule-of-thumb stability bound is 0 < mu < 1/(N*P), though this is an approximation and the exact bound depends on the eigenvalues of the input autocorrelation matrix and the specific LMS variant. On fixed-point MCUs and DSPs (such as the TI C2000 or Cortex-M4F targets), quantization of coefficients and the error signal introduces additional constraints on mu and coefficient word length that are not present in floating-point implementations.
Memory and compute costs scale with filter length. An N-tap LMS filter requires storing N input samples and N coefficients, and each iteration involves N multiply-accumulate operations for the filter output plus N multiply-accumulate operations for the coefficient update -- approximately 2N MACs total per sample, though the exact count depends on the architecture and how additions are folded into MAC instructions. On architectures with efficient multiply-accumulate support (for example, the ARM Cortex-M4F, which includes a single-cycle 32-bit MAC instruction and an FPU, or dedicated DSP cores like the TI C55x), this is manageable even at audio or moderate RF sample rates, but long adaptive filters (hundreds of taps) may require the frequency-domain LMS variant (FLMS/FDAF) to meet timing budgets.
A common pitfall is ignoring the correlation structure of the input signal. LMS convergence speed depends on the eigenvalue spread of the input autocorrelation matrix: highly colored or narrowband input signals slow convergence dramatically and may require normalized LMS (NLMS), which divides the update by the input power estimate, to maintain stable and consistent convergence speed across varying input conditions.
Discussed on DSPRelated
Frequently asked
What is the difference between LMS and NLMS?
Standard LMS uses a fixed step size mu, which means convergence speed and stability both depend on input signal power. Normalized LMS (NLMS) divides the update by the squared norm of the input vector plus a small stabilizing constant epsilon: w(n+1) = w(n) + (mu / (||x(n)||^2 + epsilon)) * e(n) * x(n). This makes the effective step size inversely proportional to input power, giving more consistent convergence behavior when the input level varies -- common in real-world audio and communications signals.
How do I choose the number of taps for an LMS adaptive filter?
Tap count determines how much of the signal's history the filter can model. For echo cancellation, the tap count must cover the full echo delay span: at 8 kHz with a 50 ms echo tail, you need at least 400 taps. For channel equalization, tap count is governed by the channel
impulse response length. More taps improve modeling accuracy but increase compute cost and slow LMS convergence, since the same total misadjustment noise is spread across more coefficients.
Can LMS run on an MCU without a floating-point unit?
Yes, but care is required. Fixed-point LMS implementations must handle coefficient and accumulator word lengths carefully to avoid coefficient quantization noise and overflow. A common approach on fixed-point DSPs (such as the TI C55x or C2000) is to maintain coefficients in a higher-precision accumulator format and round only on readback. The step size mu must also be representable in the chosen fixed-point format, which constrains the finest achievable granularity of adaptation.
Why does LMS converge slowly with a narrowband or correlated input?
LMS convergence rate is governed by the eigenvalues of the input
autocorrelation matrix R. When the input is narrowband or highly colored, R has a large eigenvalue spread (condition number). LMS uses a single global step size, so the step must be small enough for the largest eigenvalue to remain stable, which makes adaptation along the smallest-eigenvalue directions very slow. Techniques like NLMS, transform-domain LMS, or RLS address this at varying cost.
What is misadjustment in the context of LMS?
Misadjustment is the excess mean squared error that remains after the LMS filter has converged, expressed as a fraction of the minimum achievable MSE. Under the independence assumption and for small mu, it is approximately mu * N * P for a standard LMS filter with N taps and input power P; this is a rough approximation that becomes less accurate for larger step sizes or correlated inputs. There is a fundamental trade-off: a larger mu converges faster but produces higher steady-state misadjustment; a smaller mu converges slowly but tracks closer to the true optimum. The blog post 'DSP Algorithm Implementation: A Comprehensive Approach' discusses this kind of implementation-level trade-off in the broader context of DSP algorithm design.
Differentiators vs similar concepts
LMS is often confused with RLS (Recursive Least Squares). RLS uses the exact inverse
autocorrelation matrix to compute the optimal coefficient update at each step, giving much faster convergence and better performance with correlated inputs, but at O(N^2) cost per sample versus LMS's O(N). LMS is also distinct from
Wiener filter design: the Wiener filter computes the globally optimal fixed filter given known signal statistics, while LMS iteratively estimates and tracks those statistics in real time without requiring them to be known in advance.
