Math Core

A linear system can be read row by row, as a list of equations, or column by column, as a single question about vectors: can a target vector be built out of some given vectors? The column view is where most of the geometry of linear algebra lives, and it leads directly to the idea of a span.

Vectors in $\mathbb{R}^n$

A vector in $\mathbb{R}^n$ is an ordered list of $n$ real numbers, written as a column:

\mathbf{u} = \begin{bmatrix} 3 \\ -1 \end{bmatrix} \in \mathbb{R}^2, \qquad \mathbf{v} = \begin{bmatrix} 1 \\ 0 \\ 4 \end{bmatrix} \in \mathbb{R}^3.

Two vectors are equal when their corresponding entries are equal. The two basic operations act entry by entry: the sum $\mathbf{u} + \mathbf{v}$ adds matching entries, and the scalar multiple $c\mathbf{u}$ multiplies every entry by the number $c$ (a scalar). The zero vector $\mathbf{0}$ has every entry equal to $0$ .

\begin{bmatrix} 1 \\ 2 \end{bmatrix} + \begin{bmatrix} 3 \\ -1 \end{bmatrix} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}, \qquad -2\begin{bmatrix} 3 \\ -1 \end{bmatrix} = \begin{bmatrix} -6 \\ 2 \end{bmatrix}.

These operations obey the familiar algebraic rules: addition is commutative and associative, $\mathbf{u} + \mathbf{0} = \mathbf{u}$ , $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$ , $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$ , $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$ , $c(d\mathbf{u}) = (cd)\mathbf{u}$ and $1\mathbf{u} = \mathbf{u}$ . Each follows from the corresponding rule for real numbers, applied one entry at a time.

Geometry in the plane

Picture $\begin{bmatrix} a \\ b \end{bmatrix}$ as an arrow from the origin to the point $(a, b)$ . Scalar multiplication stretches or shrinks the arrow (and reverses it when $c$ is negative). Addition follows the parallelogram rule: $\mathbf{u} + \mathbf{v}$ is the fourth corner of the parallelogram with sides $\mathbf{u}$ and $\mathbf{v}$ .

The parallelogram rule: u = (1, 2), v = (3, −1) and u + v = (4, 1).Open in grapher →

Linear combinations

Definition

Linear combination

Given vectors $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_p$ in $\mathbb{R}^n$ and scalars $c_1, c_2, \dots, c_p$ , the vector

\mathbf{y} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_p\mathbf{v}_p

is a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_p$ with weights $c_1, \dots, c_p$ . The weights can be any real numbers, including zero.

The central question is the reverse one: given $\mathbf{b}$ , can you find weights that produce it? Write $\mathbf{a}_1 = \begin{bmatrix} 1 \\ -2 \\ 3 \end{bmatrix}$ , $\mathbf{a}_2 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$ and ask whether $x_1\mathbf{a}_1 + x_2\mathbf{a}_2 = \mathbf{b}$ for some $x_1, x_2$ . Combining the left side into one vector,

\begin{bmatrix} x_1 + 2x_2 \\ -2x_1 + x_2 \\ 3x_1 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix},

and two vectors are equal exactly when their entries agree. So the vector equation is the same thing as a linear system, whose augmented matrix has the vectors $\mathbf{a}_1, \mathbf{a}_2, \mathbf{b}$ as its columns.

Vector equations are linear systems

The vector equation

x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n = \mathbf{b}

has the same solution set as the linear system whose augmented matrix is

\left[\begin{array}{cccc|c} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n & \mathbf{b} \end{array}\right].

In particular, $\mathbf{b}$ is a linear combination of $\mathbf{a}_1, \dots, \mathbf{a}_n$ if and only if this system is consistent.

Worked example: Finding the weights

Is $\mathbf{b} = \begin{bmatrix} 7 \\ 1 \\ 9 \end{bmatrix}$ a linear combination of $\mathbf{a}_1 = \begin{bmatrix} 1 \\ -2 \\ 3 \end{bmatrix}$ and $\mathbf{a}_2 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$ ?

Row reduce $[\,\mathbf{a}_1 \;\; \mathbf{a}_2 \mid \mathbf{b}\,]$ with $R_2 + 2R_1$ and $R_3 - 3R_1$ :

\left[\begin{array}{cc|c} 1 & 2 & 7 \\ -2 & 1 & 1 \\ 3 & 0 & 9 \end{array}\right] \to \left[\begin{array}{cc|c} 1 & 2 & 7 \\ 0 & 5 & 15 \\ 0 & -6 & -12 \end{array}\right].

Row 2 forces $x_2 = 3$ , but row 3 forces $x_2 = 2$ . Formally, $R_3 + \tfrac{6}{5}R_2$ gives $[\,0 \;\; 0 \mid 6\,]$ . The system is inconsistent, so $\mathbf{b}$ is not a linear combination of $\mathbf{a}_1$ and $\mathbf{a}_2$ .

Change the target to $\mathbf{b} = \begin{bmatrix} 7 \\ 1 \\ 3 \end{bmatrix}$ and the same steps give rows $[\,0 \;\; 5 \mid 15\,]$ and $[\,0 \;\; {-6} \mid {-18}\,]$ , both saying $x_2 = 3$ . Then $x_1 = 7 - 6 = 1$ , and indeed $1\mathbf{a}_1 + 3\mathbf{a}_2 = \begin{bmatrix} 7 \\ 1 \\ 3 \end{bmatrix}$ .

Span

Definition

Span

If $\mathbf{v}_1, \dots, \mathbf{v}_p$ are in $\mathbb{R}^n$ , then $\operatorname{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ is the set of all linear combinations of $\mathbf{v}_1, \dots, \mathbf{v}_p$ , that is, all vectors of the form $c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p$ with $c_1, \dots, c_p$ scalars.

Asking "is $\mathbf{b}$ in $\operatorname{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ ?" is the same as asking whether the system with augmented matrix $[\,\mathbf{v}_1 \;\cdots\; \mathbf{v}_p \mid \mathbf{b}\,]$ is consistent. Every span contains $\mathbf{0}$ (take all weights $0$ ) and every $c\mathbf{v}_i$ .

Geometrically:

If $\mathbf{v} \ne \mathbf{0}$ , then $\operatorname{Span}\{\mathbf{v}\}$ is the line through the origin in the direction of $\mathbf{v}$ .
If $\mathbf{u}$ and $\mathbf{v}$ are nonzero and neither is a multiple of the other, then $\operatorname{Span}\{\mathbf{u}, \mathbf{v}\}$ is the plane through the origin containing them. In $\mathbb{R}^2$ that plane is all of $\mathbb{R}^2$ .
If $\mathbf{v}$ is a multiple of $\mathbf{u}$ , adding it contributes nothing new, and the span is still just a line.

Span{v} for v = (2, 1) is the whole line y = x/2 through the origin. Every multiple of v, such as −2v, lies on it.Open in grapher →

Worked example: A span in R³

Describe $\operatorname{Span}\{\mathbf{u}, \mathbf{v}\}$ for $\mathbf{u} = \begin{bmatrix} 1 \\ 0 \\ 2 \end{bmatrix}$ and $\mathbf{v} = \begin{bmatrix} 0 \\ 1 \\ -1 \end{bmatrix}$ , and decide whether $\mathbf{b} = \begin{bmatrix} 3 \\ -2 \\ 8 \end{bmatrix}$ lies in it.

Neither vector is a multiple of the other, so the span is a plane through the origin. A general element is

c_1\mathbf{u} + c_2\mathbf{v} = \begin{bmatrix} c_1 \\ c_2 \\ 2c_1 - c_2 \end{bmatrix}.

Matching the first two entries of $\mathbf{b}$ forces $c_1 = 3$ and $c_2 = -2$ , and then the third entry must be $2(3) - (-2) = 8$ . It is, so $\mathbf{b} = 3\mathbf{u} - 2\mathbf{v}$ lies in the plane. (In fact the plane is exactly the set of vectors with $x_3 = 2x_1 - x_2$ .)

Common mistake

A span is a set of vectors, not a single vector. Writing " $\operatorname{Span}\{\mathbf{u}, \mathbf{v}\} = \mathbf{u} + \mathbf{v}$ " confuses one particular combination with all of them. Also remember that the weights may be negative or zero; the span of $\mathbf{v}$ is the entire line, not just the ray in the direction of $\mathbf{v}$ .

Practice

Practice 1

Let $\mathbf{u} = \begin{bmatrix} 1 \\ -2 \end{bmatrix}$ and $\mathbf{v} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}$ . Compute $3\mathbf{u} - 2\mathbf{v}$ and give it as an ordered pair.

Enter a point like (2, -3)

Practice 2

Find weights $x_1, x_2$ with $x_1\begin{bmatrix} 1 \\ 2 \end{bmatrix} + x_2\begin{bmatrix} 3 \\ -1 \end{bmatrix} = \begin{bmatrix} 5 \\ 3 \end{bmatrix}$ . Give $(x_1, x_2)$ .

Enter a point like (2, -3)

Practice 3

Is $\mathbf{b} = \begin{bmatrix} 1 \\ 2 \\ 4 \end{bmatrix}$ in $\operatorname{Span}\{\mathbf{a}_1, \mathbf{a}_2\}$ , where $\mathbf{a}_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}$ and $\mathbf{a}_2 = \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}$ ?

Practice 4

Let $\mathbf{a}_1 = \begin{bmatrix} 1 \\ 0 \\ 2 \end{bmatrix}$ , $\mathbf{a}_2 = \begin{bmatrix} 2 \\ 1 \\ 5 \end{bmatrix}$ and $\mathbf{b} = \begin{bmatrix} 3 \\ -1 \\ h \end{bmatrix}$ . For what value of $h$ is $\mathbf{b}$ in $\operatorname{Span}\{\mathbf{a}_1, \mathbf{a}_2\}$ ?

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Practice 5

Which best describes $\operatorname{Span}\left\{\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ 4 \\ 0 \end{bmatrix}\right\}$ in $\mathbb{R}^3$ ?

Practice 6

Write $\mathbf{b} = \begin{bmatrix} 5 \\ 1 \\ -2 \end{bmatrix}$ as $x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + x_3\mathbf{a}_3$ , where $\mathbf{a}_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}$ , $\mathbf{a}_2 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$ , $\mathbf{a}_3 = \begin{bmatrix} 0 \\ 1 \\ 3 \end{bmatrix}$ . Give $(x_1, x_2, x_3)$ .

Enter a point like (2, -3)

Practice 7

For what value of $k$ is $\begin{bmatrix} 3 \\ k \end{bmatrix}$ in $\operatorname{Span}\left\{\begin{bmatrix} -1 \\ 2 \end{bmatrix}\right\}$ ?

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Vectors in Rn\mathbb{R}^nRn

Geometry in the plane

Linear combinations

Span

Practice

Vectors in $\mathbb{R}^n$