Math Core

You can't ask every voter in the country who they'll vote for, or test every light bulb a factory makes (testing destroys them). Instead you study a small group and use it to draw conclusions about the whole. Whether those conclusions are any good depends almost entirely on how the group was chosen.

Populations and samples

Definition

Population and sample

The population is the entire group you want to know about. A sample is the part of the population you actually collect data from. Collecting data from every member of the population is called a census.

A number that describes a population is a parameter. A number that describes a sample is a statistic.

A helpful memory aid: population goes with parameter, sample goes with statistic. The whole point of sampling is to use a statistic (which you can compute) to estimate a parameter (which you usually can't).

For example, suppose a school of $1{,}800$ students wants to know the proportion who eat breakfast. The population is all $1{,}800$ students, and the true proportion is a parameter. If $150$ randomly chosen students are surveyed and $96$ say they eat breakfast, the sample proportion $\dfrac{96}{150} = 0.64$ is a statistic. The school would estimate that about $64\%$ of all students eat breakfast, or about $0.64 \times 1800 = 1152$ students.

Sampling methods

A good sample is representative: it looks like the population in the ways that matter. The most reliable way to get one is to let chance do the choosing.

Method	How it works
Simple random sample	Every group of $n$ members is equally likely to be chosen. Example: put every name in a hat, or number everyone and use a random number generator.
Stratified random sample	Split the population into groups (strata) that share a trait, such as grade level, then take a random sample from each stratum.
Cluster sample	Split the population into groups (clusters), such as homerooms, randomly choose some clusters, and survey everyone in those clusters.
Systematic sample	Choose a random starting point, then select every $k$ th member of a list.
Convenience sample	Survey whoever is easiest to reach.
Voluntary response sample	Invite everyone and let people decide whether to respond.

The first four use randomness and can produce trustworthy results. The last two usually do not: people who are easy to reach, or who choose to respond, often differ from the population in systematic ways.

Common mistake

Stratified and cluster samples are easy to mix up. In a stratified sample you take some members from every group. In a cluster sample you take all members from some groups.

Worked example: Identifying the method

A city wants to survey residents about a new park. Identify each method.

The city randomly selects $12$ of its $80$ neighborhoods and surveys every household in those $12$ .
A reporter stands outside the library and asks the first $50$ people who walk by.
The city randomly selects $100$ residents from each of its $5$ voting districts.
From an alphabetical list of residents, the city picks a random number from $1$ to $40$ , starts there, and chooses every $40$ th name.

Solution.

Cluster sample: all members of a few randomly chosen groups.
Convenience sample: easy to reach, not random. Library visitors may care more about public spaces than other residents.
Stratified random sample: a random sample from every district.
Systematic sample: every $40$ th name after a random start.

Sizing systematic and stratified samples

For a systematic sample of size $n$ from a list of $N$ members, choose every $k$ th member where $k = \dfrac{N}{n}$ .

For a stratified sample, the most common approach is proportional allocation: each stratum gets the same share of the sample as it has of the population.

Worked example: Proportional allocation

A high school has $1{,}200$ students: $400$ ninth graders, $300$ tenth graders, $300$ eleventh graders and $200$ twelfth graders. The principal wants a stratified sample of $60$ students. How many should come from each grade?

Solution. The sample is $\dfrac{60}{1200} = \dfrac{1}{20}$ of the population, so take $\dfrac{1}{20}$ of each grade:

\frac{400}{20} = 20, \qquad \frac{300}{20} = 15, \qquad \frac{300}{20} = 15, \qquad \frac{200}{20} = 10.

Check: $20 + 15 + 15 + 10 = 60$ .

Bias

A sampling method is biased if it tends to produce results that are off in the same direction every time. Bigger samples do not fix bias. A biased method with $10{,}000$ responses is still wrong, just more confidently wrong.

Common sources of bias

Undercoverage (selection bias): part of the population has little or no chance of being chosen. Example: a phone survey that calls only landlines.
Voluntary response bias: people with strong opinions, often negative ones, are much more likely to respond.
Nonresponse bias: many selected people can't be reached or refuse, and they differ from those who answer.
Response bias: people answer untruthfully, often to seem more socially acceptable, or because of how the question is asked.
Question wording bias: a leading or confusing question pushes people toward an answer.

Worked example: Spotting bias in a survey question

A survey asks: "Don't you agree that the town should stop wasting money on an unnecessary new stadium?" Describe the problem and rewrite the question.

Solution. The words "wasting" and "unnecessary" and the phrase "Don't you agree" lead people toward saying yes. This is question wording bias, and it will overestimate opposition to the stadium. A neutral version: "Do you support or oppose the town building a new stadium?"

Tip

When judging a survey, ask three questions: Who was chosen, and how? Who actually answered? What exactly were they asked? A problem at any step can ruin the results.

Practice

Practice 1

A school wants to survey students about lunch options. It randomly selects $6$ of its $40$ homerooms and surveys every student in those $6$ homerooms. What type of sample is this?

Practice 2

A news website posts an online poll: "Should the speed limit on highways be raised?" Of $5{,}000$ readers who chose to respond, $78\%$ said yes. What is the biggest problem with using this result to describe all drivers?

Practice 3

A researcher randomly selects $400$ of a state's registered voters and finds that $55\%$ support a proposal. Which statement is correct?

Practice 4

A store has a list of $2{,}400$ loyalty-card customers and wants a systematic sample of $80$ of them. After a random start, every $k$ th customer on the list is chosen. What is $k$ ?

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Practice 5

A company has $900$ employees: $540$ are hourly and $360$ are salaried. It wants a stratified sample of $50$ employees using proportional allocation. How many hourly employees should be in the sample?

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

Practice 6

Which survey method is most likely to give an unbiased estimate of the average number of hours per week that students at a large university spend studying?

Practice 7

A town has $12{,}000$ adult residents. In a simple random sample of $250$ adults, $95$ say they would use a new bike lane. Use the sample proportion to estimate the number of adult residents who would use the bike lane.

Enter a number. Fractions like 3/4 and sqrt(2) are OK.