Math Core

You have now seen the same problem in two costumes: a system of equations and a vector equation. A third notation, the matrix equation $A\mathbf{x} = \mathbf{b}$ , packages both into a single compact statement. It also reframes the question "does this system have a solution?" as "which vectors $\mathbf{b}$ can the matrix $A$ produce?"

The product of a matrix and a vector

Definition

Matrix-vector product

If $A$ is an $m \times n$ matrix with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ , and $\mathbf{x}$ is a vector in $\mathbb{R}^n$ , then

A\mathbf{x} = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n.

$A\mathbf{x}$ is defined only when the number of columns of $A$ equals the number of entries in $\mathbf{x}$ , and the result lies in $\mathbb{R}^m$ .

So $A\mathbf{x}$ is a linear combination of the columns of $A$ , with the entries of $\mathbf{x}$ as weights. For example,

\begin{bmatrix} 1 & 2 & -1 \\ 0 & -5 & 3 \end{bmatrix}\begin{bmatrix} 4 \\ 3 \\ 7 \end{bmatrix} = 4\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 3\begin{bmatrix} 2 \\ -5 \end{bmatrix} + 7\begin{bmatrix} -1 \\ 3 \end{bmatrix} = \begin{bmatrix} 3 \\ 6 \end{bmatrix}.

For hand computation there is a faster way to organize the same arithmetic. Entry $i$ of $A\mathbf{x}$ collects the $i$ -th entry of each column times its weight, which is the sum of products of row $i$ of $A$ with the entries of $\mathbf{x}$ :

Row-vector rule

The $i$ -th entry of $A\mathbf{x}$ is $a_{i1}x_1 + a_{i2}x_2 + \cdots + a_{in}x_n$ , the sum of the products of the entries in row $i$ of $A$ with the corresponding entries of $\mathbf{x}$ .

In the example, row 1 gives $1(4) + 2(3) - 1(7) = 3$ and row 2 gives $0(4) - 5(3) + 3(7) = 6$ . Keep both viewpoints: the row rule for computing, the column rule for thinking.

Three equivalent formulations

With the definition in hand, the system

\begin{aligned} x_1 + 2x_2 - x_3 &= 4 \\ -5x_2 + 3x_3 &= 1 \end{aligned}

is the vector equation $x_1\begin{bmatrix} 1 \\ 0 \end{bmatrix} + x_2\begin{bmatrix} 2 \\ -5 \end{bmatrix} + x_3\begin{bmatrix} -1 \\ 3 \end{bmatrix} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}$ , which is the matrix equation

\begin{bmatrix} 1 & 2 & -1 \\ 0 & -5 & 3 \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}.

One problem, three notations

If $A$ is $m \times n$ with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ and $\mathbf{b}$ is in $\mathbb{R}^m$ , then the matrix equation $A\mathbf{x} = \mathbf{b}$ , the vector equation $x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$ , and the linear system with augmented matrix $[\,\mathbf{a}_1 \;\cdots\; \mathbf{a}_n \mid \mathbf{b}\,]$ all have the same solution set.

Consequently, $A\mathbf{x} = \mathbf{b}$ has a solution if and only if $\mathbf{b}$ is a linear combination of the columns of $A$ .

Worked example: Solving a matrix equation

Solve $A\mathbf{x} = \mathbf{b}$ for $A = \begin{bmatrix} 1 & 1 & 0 \\ 0 & 1 & 1 \\ 1 & 0 & 1 \end{bmatrix}$ and $\mathbf{b} = \begin{bmatrix} 3 \\ 5 \\ 4 \end{bmatrix}$ .

Row reduce $[\,A \mid \mathbf{b}\,]$ : $R_3 - R_1$ gives $[\,0 \;\; {-1} \;\; 1 \mid 1\,]$ , then $R_3 + R_2$ gives $[\,0 \;\; 0 \;\; 2 \mid 6\,]$ :

\left[\begin{array}{ccc|c} 1 & 1 & 0 & 3 \\ 0 & 1 & 1 & 5 \\ 0 & 0 & 2 & 6 \end{array}\right].

So $x_3 = 3$ , $x_2 = 5 - 3 = 2$ and $x_1 = 3 - 2 = 1$ : $\mathbf{x} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$ . Check with the row rule: $A\mathbf{x} = \begin{bmatrix} 1 + 2 \\ 2 + 3 \\ 1 + 3 \end{bmatrix} = \begin{bmatrix} 3 \\ 5 \\ 4 \end{bmatrix}$ .

When is $A\mathbf{x} = \mathbf{b}$ solvable for every $\mathbf{b}$ ?

For a fixed matrix $A$ , some right-hand sides may be reachable and others not. The set of reachable $\mathbf{b}$ is exactly $\operatorname{Span}\{\mathbf{a}_1, \dots, \mathbf{a}_n\}$ . The best possible situation is that this span is all of $\mathbb{R}^m$ .

Worked example: A matrix whose columns do not span

Let $A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 5 & 7 \\ 1 & 3 & 4 \end{bmatrix}$ . For which $\mathbf{b}$ is $A\mathbf{x} = \mathbf{b}$ consistent?

Row reduce $[\,A \mid \mathbf{b}\,]$ with $R_2 - 2R_1$ , $R_3 - R_1$ , then $R_3 - R_2$ :

\left[\begin{array}{ccc|c} 1 & 2 & 3 & b_1 \\ 0 & 1 & 1 & b_2 - 2b_1 \\ 0 & 1 & 1 & b_3 - b_1 \end{array}\right] \to \left[\begin{array}{ccc|c} 1 & 2 & 3 & b_1 \\ 0 & 1 & 1 & b_2 - 2b_1 \\ 0 & 0 & 0 & b_1 - b_2 + b_3 \end{array}\right].

The system is consistent exactly when $b_1 - b_2 + b_3 = 0$ . That is the equation of a plane through the origin in $\mathbb{R}^3$ , so the columns of $A$ span only a plane, not all of $\mathbb{R}^3$ . For example $\mathbf{b} = (1, 0, 0)$ is unreachable.

The trouble in that example came from the zero row in the coefficient part: it left the last entry as a condition on $\mathbf{b}$ . If every row of $A$ had a pivot, no such condition could arise.

Spanning theorem

Let $A$ be an $m \times n$ matrix. The following statements are either all true or all false.

For each $\mathbf{b}$ in $\mathbb{R}^m$ , the equation $A\mathbf{x} = \mathbf{b}$ has a solution.
Each $\mathbf{b}$ in $\mathbb{R}^m$ is a linear combination of the columns of $A$ .
The columns of $A$ span $\mathbb{R}^m$ .
$A$ has a pivot position in every row.

A quick consequence: an $m \times n$ matrix has at most $n$ pivots (one per column), so if $n$ is less than $m$ the columns cannot span $\mathbb{R}^m$ . Two vectors can never span $\mathbb{R}^3$ .

Common mistake

Statement 4 is about the coefficient matrix $A$ , not the augmented matrix $[\,A \mid \mathbf{b}\,]$ . An augmented matrix can have a pivot in every row precisely because its last column is a pivot column, which means the system is inconsistent, the opposite of what you want.

Linearity of $A\mathbf{x}$

Directly from the column definition, for vectors $\mathbf{u}, \mathbf{v}$ in $\mathbb{R}^n$ and a scalar $c$ ,

A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v} \qquad\text{and}\qquad A(c\mathbf{u}) = c\,A\mathbf{u}.

For instance, $A(\mathbf{u} + \mathbf{v}) = (u_1 + v_1)\mathbf{a}_1 + \cdots + (u_n + v_n)\mathbf{a}_n$ , and regrouping gives $A\mathbf{u} + A\mathbf{v}$ . These two rules will drive the next lesson's description of solution sets, and later the whole theory of linear transformations.

Tip

To check a solution of $A\mathbf{x} = \mathbf{b}$ , multiply $A\mathbf{x}$ with the row-vector rule and compare with $\mathbf{b}$ . It takes seconds and doesn't depend on your row reduction being right.

Practice

Practice 1

Compute $A\mathbf{x}$ for $A = \begin{bmatrix} 1 & 2 & -1 \\ 0 & 3 & 4 \end{bmatrix}$ and $\mathbf{x} = \begin{bmatrix} 2 \\ 1 \\ 3 \end{bmatrix}$ . Give the result as an ordered pair.

Enter a point like (2, -3)

Practice 2

Which matrix equation represents the system $3x_1 - x_2 = 2$ , $\;x_1 + 4x_3 = 0$ ?

Practice 3

Suppose $A\mathbf{u} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$ and $A\mathbf{v} = \begin{bmatrix} 3 \\ -1 \end{bmatrix}$ . Find $A(2\mathbf{u} - \mathbf{v})$ as an ordered pair.

Enter a point like (2, -3)

Practice 4

Solve $A\mathbf{x} = \mathbf{b}$ for $A = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 3 \\ 2 & 3 & 0 \end{bmatrix}$ and $\mathbf{b} = \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}$ . Give $\mathbf{x}$ as $(x_1, x_2, x_3)$ .

Enter a point like (2, -3)

Practice 5

Let $A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 5 & 7 \\ 1 & 3 & 4 \end{bmatrix}$ (the matrix from the example above) and $\mathbf{b} = \begin{bmatrix} 1 \\ 3 \\ b_3 \end{bmatrix}$ . For what value of $b_3$ is $A\mathbf{x} = \mathbf{b}$ consistent?

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Practice 6

Do the columns of $B = \begin{bmatrix} 1 & 0 & 2 \\ 2 & 1 & 3 \\ 0 & 1 & 1 \end{bmatrix}$ span $\mathbb{R}^3$ ?

Practice 7

$A$ is a $5 \times 3$ matrix. What is the largest number of pivot positions $A$ can have? (Use your answer to decide whether the columns of $A$ can span $\mathbb{R}^5$ .)

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

The product of a matrix and a vector

Three equivalent formulations

When is Ax=bA\mathbf{x} = \mathbf{b}Ax=b solvable for every b\mathbf{b}b?

Linearity of AxA\mathbf{x}Ax

Practice

When is $A\mathbf{x} = \mathbf{b}$ solvable for every $\mathbf{b}$ ?

Linearity of $A\mathbf{x}$