Stochastic Systems: Estimation, Identification, and Adaptive Control (Classics in Applied Mathematics, Series Number 75)
Since its origins in the 1940s, the subject of decision making under uncertainty has grown into a diversified area with application in several branches of engineering and in those areas of the social sciences concerned with policy analysis and prescription. These approaches required a computing capacity too expensive for the time, until the ability to collect and process huge quantities of data engendered an explosion of work in the area.
This book provides succinct and rigorous treatment of the foundations of stochastic control; a unified approach to filtering, estimation, prediction, and stochastic and adaptive control; and the conceptual framework necessary to understand current trends in stochastic control, data mining, machine learning, and robotics.
Audience: This book is recommended for those who have been introduced to probability theory and stochastic processes and want to learn more about decision making under uncertainty. It can be used as a one- or two-semester course textbook for advanced undergrad or first-year graduate students.
Contents: Chapter 1: Introduction; Chapter 2: State space models; Chapter 3: Properties of linear stochastic systems; Chapter 4: Controlled Markov chain model; Chapter 5: Input output models; Chapter 6: Dynamic programming; Chapter 7: Linear systems: estimation and control; Chapter 8: Infinite horizon dynamic programming; Chapter 9: Introduction to system identification; Chapter 10: Linear system identification; Chapter 11: Bayesian adaptive control; Chapter 12: Non-Bayesian adaptive control; Chapter 13: Self-tuning regulators for linear systems
Why Read This Book
You will gain a rigorous, unified foundation in estimation, identification, and adaptive control that connects classical stochastic control theory to modern data-driven signal processing and learning. The book emphasizes mathematical clarity and conceptual frameworks so you can apply Kalman filtering, adaptive algorithms, and stochastic control principles across DSP, communications, radar, and robotics problems.
Who Will Benefit
Engineers and researchers with a solid mathematical background who need a principled, unifying treatment of stochastic estimation and control to apply in DSP, communications, radar, or robotics.
Level: Advanced — Prerequisites: Probability theory and stochastic processes at the level of graduate coursework (measure-theoretic probability helpful), linear systems and signals, linear algebra, basic optimization, and familiarity with differential/difference equations.
Key Takeaways
- Derive and implement linear estimators and the Kalman filter family, understanding assumptions and optimality conditions
- Formulate and solve stochastic optimal control problems using dynamic programming and understand their connections to estimation
- Design and analyze adaptive filters and adaptive control laws, including convergence and stability properties
- Perform system identification from stochastic input–output data and connect identification methods to estimator design
- Apply spectral analysis and statistical signal-processing concepts to understand noise, prediction, and frequency-domain estimators
- Analyze stochastic algorithms rigorously to assess performance limits and guide practical implementations in DSP and communications systems
Topics Covered
- 1. Introduction: Decision Making Under Uncertainty
- 2. Probability and Stochastic Processes for Systems
- 3. Linear State-Space Models and Basic Properties
- 4. Estimation and the Kalman Filter: Derivation and Properties
- 5. Prediction, Smoothing, and Spectral Representations
- 6. System Identification: Stochastic Models and Parameter Estimation
- 7. Adaptive Filtering and Adaptive Control Algorithms
- 8. Stochastic Optimal Control and Dynamic Programming
- 9. Nonlinear and Non-Gaussian Filtering (concepts and extensions)
- 10. Applications: Communications, Radar, Audio/Speech, and DSP Connections
- Appendices: Mathematical Background and Useful Tools
Languages, Platforms & Tools
How It Compares
Compared with S. M. Kay's Fundamentals of Statistical Signal Processing, Kumar's book provides a broader stochastic-control and unified systems perspective rather than focusing solely on estimator derivations; compared with Åström & Wittenmark's Adaptive Control, it is more rigorous on stochastic estimation and unified theory than on hands-on controller tuning.
