Hidden Linear Algebra in DSP
Linear algebra (LA) is usually thought of as a blunt theoretical subject. However, LA is found hidden in many DSP algorithms used widely in practice.
An obvious clue in finding LA in DSP is the linearity assumption used in theoretical analysis of systems for modelling or design. A standard modelling example for this case would be linear time invariant (LTI) systems. LTI are usually used to model flat wireless communication channels. LTI systems are also used in the design of digital filter such as FIR and IIR filters.
Linear systems are often introduced in standard system theory textbooks, informally, as systems that preserve additive inputs up to a scaling factor. The emphasis, however, is usually on the impulse response, which completely characterizes the system, and convolution that is used to find the output to any input signal. This emphasis is justly in its place, but let us deviate from the conventional treatment and look at it from LA point of view. If we are examining a linear system, LA says that we can represent it in matrix form, i.e. if we have a (Nx1) input vector signal (x) and we desire a (Mx1) vector output signal (y), the system can be characterized by the standard LA form (y = A*x) , where (A) is a (MxN) matrix. In case of LTI systems the matrix is Toeplitz, a structure repeatedly seen in DSP e.g. in wide sense stationary covariance matrices. What is interesting is that LA gives the advantage of geometric visualization of the system. We can think of the input signal (x) as a vector in N dimensional space and of the system matrix (A) as a linear transformation operator., so the system is just transforming one vector in space, x, to another vector, y. LA also provides further insight into the system by inspecting intrinsic matrix characteristics such as eigenvalues and eigenvectors.
This approach is of interest by itself (if you get in touch with your mathematical side) or can be of practical value in the future. In both cases, using the vast and well developed area of LA in DSP will surely have significant benefits.
- Comments
- Write a Comment Select to add a comment
for those starting out - or wanting a refresh and beautiful 'real time' visualisations of anything geometric/LA, then I don't believe there is any better resource than first watching and working through 3b1b's first course on LA:
it is simply excellent.
To post reply to a comment, click on the 'reply' button attached to each comment. To post a new comment (not a reply to a comment) check out the 'Write a Comment' tab at the top of the comments.
Please login (on the right) if you already have an account on this platform.
Otherwise, please use this form to register (free) an join one of the largest online community for Electrical/Embedded/DSP/FPGA/ML engineers:
