Math Core

In the last unit a matrix was mostly a compact way to store a linear system. From here on you will treat matrices as objects in their own right: you can add them, scale them and, most importantly, multiply them. Matrix multiplication is defined the way it is for a reason: it records what happens when you apply one linear map after another.

Notation

An $m \times n$ matrix has $m$ rows and $n$ columns. The entry in row $i$ and column $j$ of $A$ is written $a_{ij}$ , and you will often write $A$ by its columns:

A = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n \end{bmatrix}, \qquad \mathbf{a}_j \in \mathbb{R}^m.

The entries $a_{11}, a_{22}, a_{33}, \dots$ form the main diagonal. A square matrix whose off-diagonal entries are all $0$ is diagonal, and the diagonal matrix with $1$ s on the diagonal is the identity matrix $I_n$ . The zero matrix $0$ has every entry equal to $0$ .

Sums and scalar multiples

Two matrices are equal when they have the same size and the same entries. If $A$ and $B$ are both $m \times n$ , then $A + B$ is the $m \times n$ matrix whose entries are $a_{ij} + b_{ij}$ . For a scalar $r$ , the matrix $rA$ has entries $r a_{ij}$ . A sum of matrices of different sizes is simply undefined.

These operations behave exactly like vector addition and scaling, because an $m \times n$ matrix is really just a list of $mn$ numbers arranged in a grid. So $A + B = B + A$ , $(A + B) + C = A + (B + C)$ , $A + 0 = A$ , $r(A + B) = rA + rB$ and $(r + s)A = rA + sA$ .

Worked example: A linear combination of matrices

Let $A = \begin{bmatrix} 2 & -1 & 3 \\ 0 & 4 & 1 \end{bmatrix}$ and $B = \begin{bmatrix} 1 & 5 & -2 \\ 3 & -1 & 0 \end{bmatrix}$ . Find $2A - B$ .

Double every entry of $A$ , then subtract the matching entry of $B$ :

2A - B = \begin{bmatrix} 4 - 1 & -2 - 5 & 6 + 2 \\ 0 - 3 & 8 + 1 & 2 - 0 \end{bmatrix} = \begin{bmatrix} 3 & -7 & 8 \\ -3 & 9 & 2 \end{bmatrix}.

Matrix multiplication

When you multiply a vector $\mathbf{x}$ by $B$ , you get $B\mathbf{x}$ . If you then multiply by $A$ , you get $A(B\mathbf{x})$ . The product $AB$ is defined so that a single matrix does both steps at once: $(AB)\mathbf{x} = A(B\mathbf{x})$ for every $\mathbf{x}$ .

Write $B = \begin{bmatrix} \mathbf{b}_1 & \cdots & \mathbf{b}_p \end{bmatrix}$ and $\mathbf{x} = (x_1, \dots, x_p)$ . Then $B\mathbf{x} = x_1\mathbf{b}_1 + \cdots + x_p\mathbf{b}_p$ , and because multiplying by $A$ distributes over sums and pulls out scalars,

A(B\mathbf{x}) = x_1 A\mathbf{b}_1 + \cdots + x_p A\mathbf{b}_p = \begin{bmatrix} A\mathbf{b}_1 & \cdots & A\mathbf{b}_p \end{bmatrix}\mathbf{x}.

That tells you what $AB$ has to be.

Definition

Matrix product

If $A$ is $m \times n$ and $B$ is $n \times p$ with columns $\mathbf{b}_1, \dots, \mathbf{b}_p$ , then $AB$ is the $m \times p$ matrix

AB = \begin{bmatrix} A\mathbf{b}_1 & A\mathbf{b}_2 & \cdots & A\mathbf{b}_p \end{bmatrix}.

Each column of $AB$ is a linear combination of the columns of $A$ , with weights taken from the matching column of $B$ .

The sizes must fit: the number of columns of $A$ must equal the number of rows of $B$ . A handy picture is $(m \times \mathbf{n})(\mathbf{n} \times p) \to m \times p$ : the inside numbers must match, and the outside numbers give the size of the product.

For hand computation, use the row–column rule. The $(i, j)$ entry of $AB$ is the dot product of row $i$ of $A$ with column $j$ of $B$ :

(AB)_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{in}b_{nj}.

Worked example: Computing a product

Let $A = \begin{bmatrix} 1 & 2 & -1 \\ 3 & 0 & 4 \end{bmatrix}$ and $B = \begin{bmatrix} 2 & 1 \\ 0 & -3 \\ 5 & 2 \end{bmatrix}$ . Find $AB$ .

$A$ is $2 \times 3$ and $B$ is $3 \times 2$ , so $AB$ is defined and is $2 \times 2$ . Use row $i$ of $A$ against column $j$ of $B$ :

\begin{aligned} (AB)_{11} &= 1(2) + 2(0) + (-1)(5) = -3, \\ (AB)_{12} &= 1(1) + 2(-3) + (-1)(2) = -7, \\ (AB)_{21} &= 3(2) + 0(0) + 4(5) = 26, \\ (AB)_{22} &= 3(1) + 0(-3) + 4(2) = 11. \end{aligned}

So $AB = \begin{bmatrix} -3 & -7 \\ 26 & 11 \end{bmatrix}$ . Notice that $BA$ is also defined, but it is $3 \times 3$ , so it cannot possibly equal $AB$ .

Properties, and the ones that fail

For matrices of sizes that make the products defined, and any scalar $r$ :

$A(BC) = (AB)C$ (associative law),
$A(B + C) = AB + AC$ and $(B + C)A = BA + CA$ (distributive laws),
$r(AB) = (rA)B = A(rB)$ ,
$I_m A = A = A I_n$ when $A$ is $m \times n$ .

Associativity is not a coincidence: both sides represent "apply $C$ , then $B$ , then $A$ ."

Common mistake

Matrix multiplication is not commutative. With $A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}$ and $B = \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix}$ ,

AB = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}, \qquad BA = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}.

Two other habits from ordinary algebra also break. Cancellation fails: $AB = AC$ does not force $B = C$ . And a product can be zero without either factor being zero: $\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}\begin{bmatrix} 2 & -4 \\ -1 & 2 \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}$ . In particular, $(A + B)^2 = A^2 + AB + BA + B^2$ , which is generally not $A^2 + 2AB + B^2$ .

Powers

For a square matrix $A$ and a positive integer $k$ , $A^k$ means $A$ multiplied by itself $k$ times, and $A^0 = I$ . Powers of $A$ show up whenever a process is repeated, such as a population model applied year after year.

The transpose

The transpose $A^T$ of an $m \times n$ matrix $A$ is the $n \times m$ matrix whose columns are the rows of $A$ . In symbols, $(A^T)_{ij} = a_{ji}$ .

Transpose rules

(A^T)^T = A, \qquad (A + B)^T = A^T + B^T, \qquad (rA)^T = rA^T, \qquad (AB)^T = B^T A^T.

The transpose of a product is the product of the transposes in reverse order.

The reversal makes sense by size alone: if $A$ is $2 \times 3$ and $B$ is $3 \times 4$ , then $(AB)^T$ is $4 \times 2$ , and $B^T A^T$ is $(4 \times 3)(3 \times 2)$ , which is also $4 \times 2$ . The product $A^T B^T$ would be $(3 \times 2)(4 \times 3)$ , which is not even defined.

Worked example: A product with a transpose

Let $A = \begin{bmatrix} 1 & 2 \\ 3 & -1 \\ 0 & 4 \end{bmatrix}$ . Find $A^T A$ .

$A^T$ is $2 \times 3$ , so $A^T A$ is $2 \times 2$ . The rows of $A^T$ are the columns of $A$ , so each entry of $A^TA$ is a dot product of two columns of $A$ :

A^T A = \begin{bmatrix} 1 + 9 + 0 & 2 - 3 + 0 \\ 2 - 3 + 0 & 4 + 1 + 16 \end{bmatrix} = \begin{bmatrix} 10 & -1 \\ -1 & 21 \end{bmatrix}.

The result equals its own transpose. That is always true, since $(A^TA)^T = A^T (A^T)^T = A^TA$ .

Tip

Before multiplying, write the sizes side by side. If the inside numbers don't match, stop: the product is undefined. If they do, the outside numbers tell you the shape of the answer, which is a quick check on your work.

Practice

Practice 1

$A$ is $3 \times 5$ and $B$ is $5 \times 2$ . Which statement is true?

Practice 2

Let $A = \begin{bmatrix} 3 & -2 \\ 1 & 4 \end{bmatrix}$ and $B = \begin{bmatrix} 0 & 5 \\ -1 & 2 \end{bmatrix}$ . What is the entry in row 2, column 1 of $3A - 2B$ ?