Kalman Filtering and Neural Networks
State-of-the-art coverage of Kalman filter methods for the design of neural networks This self-contained book consists of seven chapters by expert contributors that discuss Kalman filtering as applied to the training and use of neural networks. Although the traditional approach to the subject is almost always linear, this book recognizes and deals with the fact that real problems are most often nonlinear. The first chapter offers an introductory treatment of Kalman filters with an emphasis on basic Kalman filter theory, Rauch-Tung-Striebel smoother, and the extended Kalman filter. Other chapters cover:* An algorithm for the training of feedforward and recurrent multilayered perceptrons, based on the decoupled extended Kalman filter (DEKF)* Applications of the DEKF learning algorithm to the study of image sequences and the dynamic reconstruction of chaotic processes* The dual estimation problem* Stochastic nonlinear dynamics: the expectation-maximization (EM) algorithm and the extended Kalman smoothing (EKS) algorithm* The unscented Kalman filter Each chapter, with the exception of the introduction, includes illustrative applications of the learning algorithms described here, some of which involve the use of simulated and real-life data. Kalman Filtering and Neural Networks serves as an expert resource for researchers in neural networks and nonlinear dynamical systems. An Instructor's Manual presenting detailed solutions to all the problems in the book is available upon request from the Wiley Makerting Department.
Why Read This Book
You should read this book if you want a focused treatment of how Kalman-based estimation methods (EKF, DEKF and smoothing) can be used to train and deploy neural networks for nonlinear signal estimation and time-series tasks. It gives you algorithmic detail, theory, and application examples that bridge classical statistical estimation and neural-network learning.
Who Will Benefit
Researchers and engineers working on adaptive signal processing, system identification, or nonlinear estimation who want to apply Kalman-filter methods to train and analyze feedforward and recurrent neural networks.
Level: Advanced — Prerequisites: Solid linear algebra and probability, familiarity with basic Kalman filter concepts and with neural-network fundamentals (backpropagation, MLPs and basic recurrent networks).
Key Takeaways
- Implement the extended Kalman filter (EKF) and Rauch-Tung-Striebel smoother for state estimation in nonlinear models.
- Apply the decoupled extended Kalman filter (DEKF) to train feedforward and recurrent neural networks efficiently.
- Formulate neural-network training as a state-space estimation problem and compare Kalman-based learning with gradient-based methods.
- Analyze computational complexity, numerical stability, and convergence properties of Kalman-based learning algorithms.
- Use Kalman filter variants for online time-series prediction, system identification, and adaptive filtering with nonlinear models.
Topics Covered
- Introduction to Kalman Filtering and Statistical Estimation
- Rauch-Tung-Striebel Smoothing and Smoothers for Nonlinear Systems
- The Extended Kalman Filter: Theory and Practical Issues
- Formulating Neural Networks in State-Space Form
- The Decoupled Extended Kalman Filter (DEKF) for Feedforward Networks
- DEKF and Variants for Recurrent Networks
- Comparisons with Backpropagation, RTRL and Other Training Methods
- Computational Complexity, Numerical Stability, and Implementation Strategies
- Applications: Time-Series Prediction and System Identification
- Applications: Signal Processing Case Studies (communications, control, speech/time-series)
- Practical Considerations, Initialization and Regularization
- Concluding Remarks and Future Directions
Languages, Platforms & Tools
How It Compares
More application-focused on Kalman-based NN training than Haykin's general 'Neural Networks and Learning Machines'; for pure Kalman theory use Grewal & Andrews' 'Kalman Filtering: Theory and Practice', which is deeper on estimation but lacks NN-specific algorithms like DEKF.
