Math Core

A matrix has columns and rows, and each set spans its own subspace. Remarkably, those two subspaces always have the same dimension, and that single number, the rank, controls how many solutions $A\mathbf{x} = \mathbf{b}$ can have. The Rank Theorem ties it all together in one equation.

The row space

Think of each row of an $m \times n$ matrix $A$ as a vector in $\mathbb{R}^n$ .

Definition

Row space

The row space of $A$ , written $\operatorname{Row} A$ , is the set of all linear combinations of the rows of $A$ . It is a subspace of $\mathbb{R}^n$ . Since the rows of $A$ are the columns of $A^T$ , $\operatorname{Row} A = \operatorname{Col} A^T$ .

Row operations interact with the row space very differently than with the column space. Each row operation replaces rows by combinations of rows, so the new rows lie in the old row space. Each operation is also reversible, so the old rows lie in the new row space. Therefore:

Row space and row reduction

If $A$ and $B$ are row equivalent, then $\operatorname{Row} A = \operatorname{Row} B$ . If $B$ is in echelon form, the nonzero rows of $B$ form a basis for $\operatorname{Row} A$ (and for $\operatorname{Row} B$ ).

The nonzero rows of an echelon form are independent because each has a leading entry in a column where all the rows below it have zeros.

Worked example: Bases for all three subspaces

Find bases for $\operatorname{Row} A$ , $\operatorname{Col} A$ and $\operatorname{Nul} A$ , where

A = \begin{bmatrix} 1 & 2 & -1 & 3 \\ 2 & 4 & 0 & 2 \\ 3 & 6 & -1 & 5 \end{bmatrix}.

$R_2 - 2R_1$ gives $(0, 0, 2, -4)$ and $R_3 - 3R_1$ gives $(0, 0, 2, -4)$ , then $R_3 - R_2$ gives a zero row:

A \sim B = \begin{bmatrix} 1 & 2 & -1 & 3 \\ 0 & 0 & 2 & -4 \\ 0 & 0 & 0 & 0 \end{bmatrix}.

Row space: the nonzero rows of $B$ , namely $(1, 2, -1, 3)$ and $(0, 0, 2, -4)$ .

Column space: pivots are in columns $1$ and $3$ , so use columns $1$ and $3$ of $A$ : $(1, 2, 3)$ and $(-1, 0, -1)$ .

Null space: scale row $2$ of $B$ by $\tfrac{1}{2}$ and add it to row $1$ to get the reduced form with rows $(1, 2, 0, 1)$ and $(0, 0, 1, -2)$ . So $x_1 = -2x_2 - x_4$ and $x_3 = 2x_4$ , giving the basis $(-2, 1, 0, 0)$ and $(-1, 0, 2, 1)$ .

All three bases have size $2$ , $2$ and $2$ , and $2 + 2 = 4$ , the number of columns.

Common mistake

For the row space you use the rows of the echelon form; for the column space you use the columns of the original matrix. Mixing these up is the most common error here. (The rows of $A$ itself also span $\operatorname{Row} A$ , but they may be dependent, as the third row of $A$ above is the sum of the first two.)

Rank and the Rank Theorem

Definition

Rank

The rank of $A$ is the dimension of its column space: $\operatorname{rank} A = \dim \operatorname{Col} A$ .

Both $\dim \operatorname{Col} A$ and $\dim \operatorname{Row} A$ equal the number of pivots in an echelon form of $A$ : the pivot columns give a basis for $\operatorname{Col} A$ , and each nonzero row of the echelon form contains exactly one pivot. Meanwhile, each non-pivot column gives one free variable.

The Rank Theorem

For an $m \times n$ matrix $A$ ,

\dim \operatorname{Col} A = \dim \operatorname{Row} A = \operatorname{rank} A \qquad\text{and}\qquad \operatorname{rank} A + \dim \operatorname{Nul} A = n.

In words: (pivot columns) + (free columns) = (total columns). The quantity $\dim \operatorname{Nul} A$ is often called the nullity of $A$ .

Two consequences come up constantly:

$\operatorname{rank} A \le \min(m, n)$ , since pivots need distinct rows and distinct columns.
$\operatorname{rank} A^T = \operatorname{rank} A$ , because $\operatorname{Col} A^T = \operatorname{Row} A$ .

Using the Rank Theorem

The theorem lets you answer questions about solutions without doing any row reduction.

Worked example: Information from dimensions alone

(a) A $7 \times 9$ matrix $A$ has a 2-dimensional null space. Is $A\mathbf{x} = \mathbf{b}$ consistent for every $\mathbf{b}$ in $\mathbb{R}^7$ ?

$\operatorname{rank} A = 9 - 2 = 7$ . So $\operatorname{Col} A$ is a 7-dimensional subspace of $\mathbb{R}^7$ , which must be all of $\mathbb{R}^7$ . Yes, every equation $A\mathbf{x} = \mathbf{b}$ is consistent.

(b) Can a $6 \times 9$ matrix have a 2-dimensional null space?

That would force $\operatorname{rank} A = 9 - 2 = 7$ . But the rank is at most $6$ , the number of rows. So no: the null space of a $6 \times 9$ matrix has dimension at least $3$ .

Worked example: Choosing a parameter to control the rank

For what value of $h$ does $A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & h & 6 \\ 1 & 2 & 4 \end{bmatrix}$ have rank $2$ ?

Expanding along the first row,

\det A = 1(4h - 12) - 2(8 - 6) + 3(4 - h) = h - 4.

If $h \ne 4$ , $A$ is invertible with rank $3$ . If $h = 4$ , row $2$ is twice row $1$ , and row $3$ is not a multiple of row $1$ , so exactly two rows are independent and the rank is $2$ . The answer is $h = 4$ .

Rank and invertibility

For a square $n \times n$ matrix, the Rank Theorem adds new entries to the Invertible Matrix Theorem. Each of the following is equivalent to $A$ being invertible:

The columns of $A$ form a basis of $\mathbb{R}^n$ .
$\operatorname{Col} A = \mathbb{R}^n$ .
$\operatorname{rank} A = n$ .
$\operatorname{Nul} A = \lbrace \mathbf{0} \rbrace$ , that is, $\dim \operatorname{Nul} A = 0$ .

Tip

To sanity-check any rank calculation, add the rank to the number of free variables. If the total isn't the number of columns, something went wrong.