Math Core

In the last lesson, the homogeneous equation $A\mathbf{x} = \mathbf{0}$ had either only the trivial solution or infinitely many. Read in terms of the columns of $A$ , that dichotomy answers a question about redundancy: does every vector in a set contribute a genuinely new direction, or can one of them be built from the others? That question is linear independence, one of the central ideas of the whole course.

Definition

Linear independence

A set of vectors $\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ in $\mathbb{R}^n$ is linearly independent if the vector equation

x_1\mathbf{v}_1 + x_2\mathbf{v}_2 + \cdots + x_p\mathbf{v}_p = \mathbf{0}

has only the trivial solution $x_1 = x_2 = \cdots = x_p = 0$ . The set is linearly dependent if there are weights $c_1, \dots, c_p$ , not all zero, with $c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$ . Such an equation is called a linear dependence relation.

Because the vector equation above is the matrix equation $A\mathbf{x} = \mathbf{0}$ with $A = [\,\mathbf{v}_1 \;\cdots\; \mathbf{v}_p\,]$ , everything you know about homogeneous systems applies at once.

Independence and pivots

The columns of a matrix $A$ are linearly independent if and only if $A\mathbf{x} = \mathbf{0}$ has only the trivial solution, which happens if and only if $A$ has a pivot position in every column (no free variables).

Compare this with the spanning theorem from two lessons ago: spanning $\mathbb{R}^m$ needs a pivot in every row, independence needs a pivot in every column.

Worked example: Testing a set and finding a relation

Determine whether $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$ , $\mathbf{v}_2 = \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}$ , $\mathbf{v}_3 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$ are independent. If not, find a dependence relation.

Row reduce $[\,\mathbf{v}_1 \;\; \mathbf{v}_2 \;\; \mathbf{v}_3 \mid \mathbf{0}\,]$ . $R_2 - 2R_1$ and $R_3 - 3R_1$ give rows $[\,0 \;\; {-3} \;\; {-3} \mid 0\,]$ and $[\,0 \;\; {-6} \;\; {-6} \mid 0\,]$ ; then $R_3 - 2R_2$ zeros out row 3. Scaling row 2 by $-\tfrac{1}{3}$ and clearing above it gives the RREF

\left[\begin{array}{ccc|c} 1 & 0 & -2 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right].

Column 3 has no pivot, so $x_3$ is free and the set is dependent. From the RREF, $x_1 = 2x_3$ and $x_2 = -x_3$ . Taking $x_3 = 1$ gives $x_1 = 2$ , $x_2 = -1$ :

2\mathbf{v}_1 - \mathbf{v}_2 + \mathbf{v}_3 = \mathbf{0}.

Check the first entries: $2(1) - 4 + 2 = 0$ . Each vector can be solved for in terms of the others; for instance $\mathbf{v}_3 = \mathbf{v}_2 - 2\mathbf{v}_1$ .

Small sets and special cases

Several cases can be decided without row reduction.

One vector. $\{\mathbf{v}\}$ is independent if and only if $\mathbf{v} \ne \mathbf{0}$ , since $x\mathbf{v} = \mathbf{0}$ with $\mathbf{v} \ne \mathbf{0}$ forces $x = 0$ .
Two vectors. $\{\mathbf{u}, \mathbf{v}\}$ is dependent if and only if one is a scalar multiple of the other. Geometrically, the two vectors are dependent exactly when they lie on a common line through the origin.
A set containing $\mathbf{0}$ is always dependent: $1\cdot\mathbf{0} + 0\mathbf{v}_2 + \cdots = \mathbf{0}$ is a nontrivial relation.
Too many vectors. If $p$ is greater than $n$ , any set of $p$ vectors in $\mathbb{R}^n$ is dependent. The matrix $[\,\mathbf{v}_1 \;\cdots\; \mathbf{v}_p\,]$ is $n \times p$ , so it has at most $n$ pivots and at least one column without a pivot.

u and −2u lie on one line through the origin, so {u, −2u} is dependent. w is not a multiple of u, so {u, w} is independent.Open in grapher →

Characterization of dependent sets

An indexed set $\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ of two or more vectors is linearly dependent if and only if at least one of the vectors is a linear combination of the others.

Why? If $c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$ with some $c_j \ne 0$ , divide by $c_j$ and solve for $\mathbf{v}_j$ . Conversely, if $\mathbf{v}_j = \sum_{i \ne j} d_i\mathbf{v}_i$ , then moving $\mathbf{v}_j$ to the other side gives a relation whose coefficient on $\mathbf{v}_j$ is $-1 \ne 0$ .

In terms of span: a set is dependent exactly when some vector lies in the span of the others, so removing it does not shrink the span. Independent vectors are the ones with no dead weight.

Common mistake

"Dependent" means some vector is a combination of the others, not every vector. In $\{(1, 0), (2, 0), (0, 1)\}$ the first two are multiples of each other, so the set is dependent, yet $(0, 1)$ is not a combination of $(1, 0)$ and $(2, 0)$ . Also, checking that no two vectors are multiples of each other proves independence only for sets of two vectors. Three vectors can be pairwise non-parallel and still dependent, as in the example above.

Worked example: A parameter that forces dependence

For which $h$ is $\left\{\begin{bmatrix} 1 \\ 2 \\ -1 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 3 \end{bmatrix}, \begin{bmatrix} 2 \\ 5 \\ h \end{bmatrix}\right\}$ linearly dependent?

Row reduce the matrix with these columns. $R_2 - 2R_1$ and $R_3 + R_1$ give

\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 1 \\ 0 & 3 & h + 2 \end{bmatrix} \xrightarrow{R_3 - 3R_2} \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 1 \\ 0 & 0 & h - 1 \end{bmatrix}.

Column 3 has a pivot unless $h - 1 = 0$ . So the set is dependent exactly when $h = 1$ , and then the third vector equals $2\mathbf{v}_1 + \mathbf{v}_2 = (2, 5, 1)$ .

Tip

For $n$ vectors in $\mathbb{R}^n$ (a square matrix), independence and spanning happen together: a pivot in every column of an $n \times n$ matrix is the same as a pivot in every row. You will see this again as part of the invertible matrix theorem.

Practice

Practice 1

Is the set $\left\{\begin{bmatrix} 1 \\ 2 \end{bmatrix}, \begin{bmatrix} 3 \\ 6 \end{bmatrix}\right\}$ linearly independent?

Practice 2

Which set is guaranteed to be linearly dependent without any computation?

Practice 3

How many pivot columns does the matrix with columns $\begin{bmatrix} 1 \\ 0 \\ 2 \end{bmatrix}$ , $\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}$ , $\begin{bmatrix} 3 \\ -2 \\ 4 \end{bmatrix}$ have? (Then decide whether the columns are independent.)

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Practice 4

For the vectors in the previous problem, find $c_1$ and $c_2$ so that $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \mathbf{v}_3 = \mathbf{0}$ . Give $(c_1, c_2)$ .

Enter a point like (2, -3)

Practice 5

Which set is linearly independent?

Practice 6

For what value of $h$ are the vectors $\begin{bmatrix} 1 \\ 2 \\ -1 \end{bmatrix}$ , $\begin{bmatrix} 0 \\ 1 \\ 3 \end{bmatrix}$ , $\begin{bmatrix} 2 \\ 5 \\ h \end{bmatrix}$ linearly dependent?

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Practice 7

Find a value of $h$ for which $\left\{\begin{bmatrix} 1 \\ h \end{bmatrix}, \begin{bmatrix} h \\ 4 \end{bmatrix}\right\}$ is linearly dependent. (There are two; give either.)

Enter a number. Fractions like 3/4 and sqrt(2) are OK.