Matrix Multiplication

Let $\mathbf{A}^{\!\hbox{\tiny T}}$ be a general $M\times L$ matrix and let $\mathbf{B}$ denote a general $L\times N$ matrix. Denote the matrix product by $\mathbf{C}=\mathbf{A}^{\!\hbox{\tiny T}}\, \mathbf{B}$ . Then matrix multiplication is carried out by computing the inner product of every row of $\mathbf{A}^{\!\hbox{\tiny T}}$ with every column of $\mathbf{B}$ . Let the th row of $\mathbf{A}^{\!\hbox{\tiny T}}$ be denoted by $\underline{a}^{\hbox{\tiny T}}_i$ , $i=1, 2,\ldots,M$ , and the th column of $\mathbf{B}$ by $\underline{b}_j$ , $j=1,2,\ldots,N$ . Then the matrix product $\mathbf{C}=\mathbf{A}^{\!\hbox{\tiny T}}\, \mathbf{B}$ is defined as

$\displaystyle \mathbf{C}= \mathbf{A}^{\!\hbox{\tiny T}}\, \mathbf{B}= \left[\be... ...cdots & <\underline{a}^{\hbox{\tiny T}}_M,\underline{b}_N> \end{array}\right].$

This definition can be extended to complex matrices by using a definition of inner product which does not conjugate its second argument.^H.2

Examples:

$\displaystyle \left[\begin{array}{cc} a & b \\ c & d \\ e & f \end{array}\right... ...gamma & c\beta+d\delta \\ e\alpha+f\gamma & e\beta+f\delta \end{array}\right]$

$\displaystyle \left[\begin{array}{cc} \alpha & \beta \\ \gamma & \delta \end{ar... ...ma a + \delta b & \gamma c + \delta d & \gamma e + \delta f \end{array}\right]$

$\displaystyle \left[\begin{array}{c} \alpha \\ \beta \end{array}\right] \cdot \... ...pha a & \alpha b & \alpha c \\ \beta a & \beta b & \beta c \end{array}\right]$

$\displaystyle \left[\begin{array}{ccc} a & b & c \end{array}\right] \cdot \left... ...} \alpha \\ \beta \\ \gamma \end{array}\right] = a \alpha + b \beta + c \gamma$

An $M\times L$ matrix $\mathbf{A}$ can be multiplied on the right by an $L\times N$ matrix, where is any positive integer. An $L\times N$ matrix $\mathbf{A}$ can be multiplied on the left by a $M\times L$ matrix, where is any positive integer. Thus, the number of columns in the matrix on the left must equal the number of rows in the matrix on the right.

Matrix multiplication is non-commutative, in general. That is, normally $\mathbf{A}\,\mathbf{B}\neq \mathbf{B}\,\mathbf{A}$ even when both products are defined (such as when the matrices are square.)

The transpose of a matrix product is the product of the transposes in reverse order:

$\displaystyle (\mathbf{A}\mathbf{B})^{\hbox{\tiny T}} = \mathbf{B}^{\hbox{\tiny T}} \mathbf{A}^{\!\hbox{\tiny T}}$

The identity matrix is denoted by $\mathbf{I}$ and is defined as

$\displaystyle \mathbf{I}\isdef \left[\begin{array}{ccccc} 1 & 0 & 0 & \cdots &... ...dots & \vdots & \cdots & \vdots \\ 0 & 0 & 0 & \cdots & 1 \end{array}\right]$

Identity matrices are always square. The $N\times N$ identity matrix $\mathbf{I}$ , sometimes denoted as $\mathbf{I}_N$ , satisfies $\mathbf{A}\cdot \mathbf{I}_N =\mathbf{A}$ for every $M\times N$ matrix $\mathbf{A}$ . Similarly, $\mathbf{I}_M\cdot \mathbf{A}=\mathbf{A}$ , for every $M\times N$ matrix $\mathbf{A}$ .

As a special case, a matrix $\mathbf{A}^{\!\hbox{\tiny T}}$ times a vector $\underline{x}$ produces a new vector $\underline{y}= \mathbf{A}^{\!\hbox{\tiny T}}\underline{x}$ which consists of the inner product of every row of $\mathbf{A}^{\!\hbox{\tiny T}}$ with $\underline{x}$

$\displaystyle \mathbf{A}^{\!\hbox{\tiny T}}\underline{x}= \left[\begin{array}{c... ...vdots \\ <\underline{a}^{\hbox{\tiny T}}_M,\underline{x}> \end{array}\right].$

A matrix $\mathbf{A}^{\!\hbox{\tiny T}}$ times a vector $\underline{x}$ defines a linear transformation of $\underline{x}$ . In fact, every linear function of a vector $\underline{x}$ can be expressed as a matrix multiply. In particular, every linear filtering operation can be expressed as a matrix multiply applied to the input signal. As a special case, every linear, time-invariant (LTI) filtering operation can be expressed as a matrix multiply in which the matrix is Toeplitz, i.e., $\mathbf{A}^{\!\hbox{\tiny T}}[i,j] = \mathbf{A}^{\!\hbox{\tiny T}}[i-j]$ (constant along diagonals).

As a further special case, a row vector on the left may be multiplied by a column vector on the right to form a single inner product:

$\displaystyle \underline{y}^{\ast }{\underline{x}} = \langle \underline{x},\underline{y}\rangle % \ip brackets huge due to y-underbar$

Order a Hardcopy of Mathematics of the DFT

Previous: Matrices
Next: Solving Linear Equations Using Matrices

written by Julius Orion Smith III
Julius Smith's background is in electrical engineering (BS Rice 1975, PhD Stanford 1983). He is presently Professor of Music and Associate Professor (by courtesy) of Electrical Engineering at Stanford's Center for Computer Research in Music and Acoustics (CCRMA), teaching courses and pursuing research related to signal processing applied to music and audio systems. See http://ccrma.stanford.edu/~jos/ for details.

Sign in

Search Online Books

Free Online Books

Sponsor

Chapters

See Also

Mathematics of the DFT
Matrices
Matrix Multiplication