Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares
This groundbreaking textbook combines straightforward explanations with a wealth of practical examples to offer an innovative approach to teaching linear algebra. Requiring no prior knowledge of the subject, it covers the aspects of linear algebra - vectors, matrices, and least squares - that are needed for engineering applications, discussing examples across data science, machine learning and artificial intelligence, signal and image processing, tomography, navigation, control, and finance. The numerous practical exercises throughout allow students to test their understanding and translate their knowledge into solving real-world problems, with lecture slides, additional computational exercises in Julia and MATLAB®, and data sets accompanying the book online. Suitable for both one-semester and one-quarter courses, as well as self-study, this self-contained text provides beginning students with the foundation they need to progress to more advanced study.
Why Read This Book
You will learn the core linear-algebra tools used every day in signal processing, communications, and machine learning, with an emphasis on practical computation and real examples. The book teaches vectors, matrices, and least squares through applications — so you can directly map concepts to DSP tasks like filtering, spectral analysis, denoising, and system identification using MATLAB/Julia exercises.
Who Will Benefit
Engineers and graduate students with basic math who want a practical, application-first grounding in linear algebra to apply to DSP, audio/speech, radar, and communications problems.
Level: Beginner — Prerequisites: High-school algebra and basic calculus; no prior linear algebra required (familiarity with basic programming is helpful for computational exercises).
Key Takeaways
- Apply matrix and vector notation to represent convolution, linear filtering, and linear measurement models used in DSP
- Formulate and solve least-squares problems for system identification, spectral estimation, and deconvolution
- Use SVD and PCA to perform denoising, low-rank approximation, and feature extraction on audio, speech, and radar data
- Model FFT and spectral transforms as linear operators and exploit circulant/Toeplitz structure for efficient computation
- Design regularized inverse solutions (Tikhonov) and assess numerical stability and conditioning for robust signal recovery
- Implement QR-, SVD-, and projection-based algorithms in MATLAB/Julia to solve real-world signal-processing problems
Topics Covered
- 1. Vectors, norms, and geometric intuition
- 2. Matrices as linear maps and basic matrix algebra
- 3. Linear systems and matrix inverses
- 4. Orthogonality, projections, and the geometry of least squares
- 5. Matrix factorizations: LU and QR decompositions
- 6. The singular value decomposition and low-rank approximation
- 7. Eigenvalues, eigenvectors, and diagonalization
- 8. Norms, conditioning, and numerical stability
- 9. Circulant and Toeplitz matrices: convolution and FFT as linear maps
- 10. Regularization, Tikhonov methods, and inverse problems
- 11. Applications: spectral analysis, filter design, audio/speech denoising, radar and communications examples
- 12. Numerical methods, computational exercises (MATLAB/Julia), and datasets
Languages, Platforms & Tools
How It Compares
More application-focused and computational than Gilbert Strang's classical treatments, and more accessible for practitioners than Golub & Van Loan's exhaustive numerical-algorithms text.
