Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications
Probability, Random Variables, and Random Processes is a comprehensive textbook on probability theory for engineers that provides a more rigorous mathematical framework than is usually encountered in undergraduate courses. It is intended for first-year graduate students who have some familiarity with probability and random variables, though not necessarily of random processes and systems that operate on random signals. It is also appropriate for advanced undergraduate students who have a strong mathematical background.
The book has the following features:
- Several appendices include related material on integration, important inequalities and identities, frequency-domain transforms, and linear algebra. These topics have been included so that the book is relatively self-contained. One appendix contains an extensive summary of 33 random variables and their properties such as moments, characteristic functions, and entropy.
- Unlike most books on probability, numerous figures have been included to clarify and expand upon important points. Over 600 illustrations and MATLAB plots have been designed to reinforce the material and illustrate the various characterizations and properties of random quantities.
- Sufficient statistics are covered in detail, as is their connection to parameter estimation techniques. These include classical Bayesian estimation and several optimality criteria: mean-square error, mean-absolute error, maximum likelihood, method of moments, and least squares.
- The last four chapters provide an introduction to several topics usually studied in subsequent engineering courses: communication systems and information theory; optimal filtering (Wiener and Kalman); adaptive filtering (FIR and IIR); and antenna beamforming, channel equalization, and direction finding. This material is available electronically at the companion website.
Probability, Random Variables, and Random Processes is the only textbook on probability for engineers that includes relevant background material, provides extensive summaries of key results, and extends various statistical techniques to a range of applications in signal processing.
Why Read This Book
You should read this book if you want a rigorous, engineer-focused foundation in probability and random processes that directly supports modern DSP work — from spectral analysis and FFT-based methods to filtering and communications. You will learn how to model, analyze, and manipulate stochastic signals with mathematical clarity, and apply that theory to practical problems in audio/speech, radar, and communications systems.
Who Will Benefit
Ideal for first-year graduate students and practicing engineers (DSP, communications, radar, audio) with solid math skills who need a rigorous treatment of random processes and their signal-processing applications.
Level: Advanced — Prerequisites: Undergraduate calculus (including multivariable integration), linear algebra, basic probability and random variables, and familiarity with signals & systems (Fourier transforms and LTI systems) — familiarity with MATLAB or Python is helpful but not required.
Key Takeaways
- Formulate and analyze stationarity, ergodicity, and moments of random processes used to model signals in communications, radar, and audio.
- Derive and compute autocorrelation functions and power spectral densities and use them for spectral analysis and filtering design.
- Apply linear systems theory to random inputs to predict output statistics and understand PSD propagation through LTI systems.
- Design and analyze optimal linear estimators and Wiener filters for mean-square error criteria, and understand basic adaptive filtering principles.
- Use limit theorems and transform-domain tools (Fourier/DTFT, Laplace, z-transform) to perform practical signal-processing calculations and spectral estimation.
Topics Covered
- 1. Review of Probability and Random Variables
- 2. Mathematical Tools: Integration, Inequalities, and Transforms (appendices expanded)
- 3. Random Vectors and Multivariate Distributions
- 4. Convergence, Laws of Large Numbers, and Central Limit Theorems
- 5. Random Processes: Definitions, Moments, and Stationarity
- 6. Ergodicity, Gaussian Processes, and Markov Processes
- 7. Linear Systems Driven by Random Signals
- 8. Spectral Analysis: Autocorrelation, PSD, and the Wiener–Khinchin Theorem
- 9. Discrete-Time Processes, z-Transform, and FFT-based Spectral Methods
- 10. Linear Estimation and Wiener Filtering (MMSE) and Basics of Adaptive Filtering
- 11. Applications to Communications, Radar, and Audio/Speech Signal Models
- 12. Appendices: Frequency-Domain Transforms and Linear Algebra
Languages, Platforms & Tools
How It Compares
Covers similar ground to Papoulis & Pillai's classical text but places more emphasis on signal-processing applications; compared with Kay's 'Fundamentals of Statistical Signal Processing,' Shynk provides more foundational probability and random-process coverage rather than a narrow focus on estimator algorithms.
