Math Core

Two classes can have the same average test score and still be completely different: in one, everyone scored close to the average; in the other, scores were all over the place. To describe a data set honestly you need two numbers: one for its center (a typical value) and one for its spread (how much the values vary). This lesson reviews the tools you met in middle school and adds the one statisticians use most, the standard deviation.

Measures of center

The two main measures of center are the mean and the median.

The mean is the sum of the values divided by how many there are. For values $x_1, x_2, \dots, x_n$ it is written $\bar{x}$ ("x-bar"):

\bar{x} = \frac{x_1 + x_2 + \cdots + x_n}{n}.

The median is the middle value when the data are in order. With an even number of values, it is the mean of the two middle values.

The mean uses the actual size of every value, so one extreme value can drag it a long way. The median only cares about position, so it barely moves. Statisticians say the median is resistant to extreme values and the mean is not.

Worked example: One long commute

Eight students recorded how many minutes it takes them to get to school:

12, \ 15, \ 15, \ 18, \ 20, \ 22, \ 25, \ 61

Find the mean and the median. Which better describes a typical commute?

Mean. The sum is $12 + 15 + 15 + 18 + 20 + 22 + 25 + 61 = 188$ , so $\bar{x} = \dfrac{188}{8} = 23.5$ minutes.

Median. The two middle values are the 4th and 5th, $18$ and $20$ , so the median is $\dfrac{18 + 20}{2} = 19$ minutes.

Seven of the eight students take $25$ minutes or less, yet the mean is $23.5$ , higher than six of them. The single $61$ -minute commute pulled it up. The median, 19 minutes, is the better description of a typical commute.

Measures of spread

The range is maximum minus minimum. It is quick, but it depends only on the two most extreme values.

The interquartile range is $\text{IQR} = Q_3 - Q_1$ , the width of the middle half of the data. To find the quartiles, put the data in order and split it at the median. $Q_1$ is the median of the lower half and $Q_3$ is the median of the upper half. When the number of values is odd, leave the median itself out of both halves. Like the median, the IQR is resistant to extreme values.

Standard deviation

The IQR ignores the actual sizes of most values. The standard deviation uses every value. It measures how far the values typically are from the mean.

Definition

Standard deviation

The standard deviation of a data set is

\sigma = \sqrt{\frac{(x_1 - \bar{x})^2 + (x_2 - \bar{x})^2 + \cdots + (x_n - \bar{x})^2}{n}}.

Each $x - \bar{x}$ is a deviation: how far a value is above ( $+$ ) or below ( $-$ ) the mean. The symbol $\sigma$ is the Greek letter sigma. The quantity under the square root, the mean of the squared deviations, is called the variance.

Why square the deviations? Because the deviations always add up to $0$ . The positive ones cancel the negative ones exactly, so their plain average tells you nothing. Squaring makes every deviation positive, and the square root at the end brings the answer back to the original units.

Computing a standard deviation

Find the mean $\bar{x}$ .
Subtract the mean from each value to get the deviations. (Check: they add to $0$ .)
Square each deviation.
Find the mean of the squares: add them and divide by $n$ .
Take the square root.

Worked example: Standard deviation by hand

Find the standard deviation of $4, \ 7, \ 8, \ 10, \ 11$ .

The mean is $\bar{x} = \dfrac{40}{5} = 8$ . A table keeps the work organized:

$x$	$x - \bar{x}$	$(x - \bar{x})^2$
$4$	$-4$	$16$
$7$	$-1$	$1$
$8$	$0$	$0$
$10$	$2$	$4$
$11$	$3$	$9$
total	$0$	$30$

The deviations add to $0$ , as they should. The variance is $\dfrac{30}{5} = 6$ , so

\sigma = \sqrt{6} \approx 2.45.

The values are typically about $2.45$ units away from the mean of $8$ .

A standard deviation of $0$ means every value is the same. The more the values scatter away from the mean, the larger the standard deviation gets. For example, $49, 50, 51$ and $10, 50, 90$ both have mean $50$ , but the first has $\sigma \approx 0.82$ and the second has $\sigma \approx 32.7$ .

Tip

Graphing calculators and spreadsheets report two standard deviations. $\sigma_x$ divides by $n$ , as above; it describes the data you have. $s_x$ divides by $n - 1$ instead; statisticians use it when the data are a sample from a larger population. For the data in the example, $s_x = \sqrt{30/4} \approx 2.74$ . In this course, "standard deviation" means $\sigma$ unless a problem says otherwise.

Outliers and the 1.5 × IQR rule

An outlier is a value that is unusually far from the rest of the data. "Unusually far" needs a precise meaning, so statisticians use a standard test.

The 1.5 × IQR rule

Compute the fences

\text{lower fence} = Q_1 - 1.5 \cdot \text{IQR}, \qquad \text{upper fence} = Q_3 + 1.5 \cdot \text{IQR}.

A value is an outlier if it is below the lower fence or above the upper fence.

Worked example: Finding outliers

Here are the numbers of text messages ten friends sent in one hour:

3, \ 5, \ 6, \ 6, \ 7, \ 8, \ 9, \ 10, \ 11, \ 25

Are there any outliers?

Quartiles. There are $10$ values, so the median is $\dfrac{7 + 8}{2} = 7.5$ . The lower half is $3, 5, 6, 6, 7$ , so $Q_1 = 6$ . The upper half is $8, 9, 10, 11, 25$ , so $Q_3 = 10$ . The IQR is $10 - 6 = 4$ .

Fences. $1.5 \cdot 4 = 6$ . The lower fence is $6 - 6 = 0$ and the upper fence is $10 + 6 = 16$ .

Check. No value is below $0$ , but $25 > 16$ . So 25 is an outlier, and it is the only one.

Outliers deserve a second look. Sometimes they are recording mistakes (someone typed $25$ instead of $2.5$ ), and sometimes they are the most interesting values in the data. Don't delete one just because it's inconvenient.

Choosing the right summary

Because the mean and the standard deviation use every value, one outlier affects them both. The median and the IQR resist outliers. That gives a simple rule:

the data are…	describe center with	describe spread with
roughly symmetric, no outliers	mean	standard deviation
skewed, or have outliers	median	IQR

Changing every value

What happens to these measures if you change every value in the same way?

Adding a constant $k$ to every value shifts the whole data set. The mean and median go up by $k$ , but the values are just as spread out, so the range, IQR and standard deviation do not change.
Multiplying by a positive constant $k$ stretches the data set. The mean, median, range, IQR and standard deviation are all multiplied by $k$ .

Worked example: Curving a test

A test has a mean of $68$ points and a standard deviation of $9$ points.

The teacher adds $5$ points to every score. What are the new mean and standard deviation?
Instead, the teacher multiplies every original score by $1.25$ . What are the new mean and standard deviation?

Solutions.

Adding $5$ shifts everything: the new mean is $68 + 5 = 73$ . The spread is unchanged, so the standard deviation is still $9$ .
Multiplying by $1.25$ scales everything: the new mean is $1.25 \cdot 68 = 85$ and the new standard deviation is $1.25 \cdot 9 = 11.25$ .

Common mistake

Adding the same number to every value does not change the standard deviation. Students often add it to $\sigma$ as well. The distances between the values, and their distances from the mean, stay exactly the same.