Lesson 3.3 · Collecting Data
Observational studies and experiments
Headlines love to announce that coffee drinkers live longer or that students who eat breakfast earn higher grades. Whether a study like that can show that one thing causes another depends almost entirely on how the data were produced. This lesson separates the two main kinds of studies and explains why only one of them can establish cause and effect.
Explanatory and response variables
When a study looks at a relationship between two variables, it helps to name their roles.
- The explanatory variable is the one you think may explain or influence changes in the other. (You met this idea as in scatterplots.)
- The response variable measures an outcome of the study. (This was .)
In "Does daily breakfast improve test scores?", eating breakfast is the explanatory variable and test score is the response variable.
Observational studies versus experiments
Definition
Observational study and experiment
An observational study observes individuals and measures variables of interest but does not attempt to influence the responses.
An experiment deliberately imposes a treatment on individuals to measure their responses.
The question to ask is simple: did the researchers decide which individuals received which treatment? If they assigned it, the study is an experiment. If the individuals chose for themselves, or the explanatory variable is something like age or smoking history that the researchers merely recorded, it is an observational study.
Observational studies come in two types:
- A retrospective study looks backward, examining existing data or asking about past behavior. (For example, reviewing the medical records of 2,000 patients to compare past vitamin use.)
- A prospective study identifies a group of individuals and follows them into the future, recording what happens. (For example, enrolling 5,000 adults in 2026 and tracking their diets and health for 20 years.)
Confounding
Why can't observational studies establish causation? Because the groups being compared may differ in other ways besides the explanatory variable.
Definition
Confounding
Two variables are confounded when their effects on a response variable cannot be distinguished from each other. A confounding variable is related to the explanatory variable and also affects the response variable, making it impossible to tell which one is responsible for the observed association.
Suppose an observational study finds that students who eat breakfast every day have higher GPAs. Students who eat breakfast daily may also get more sleep, come from households with more regular routines, or have parents who pay closer attention to schoolwork. Any of those could raise GPA on its own. Sleep is a confounding variable: it is associated with eating breakfast (well-rested students have time for breakfast) and it affects GPA.
When you name a confounding variable on the AP exam, describe both links: how it relates to the explanatory variable and how it affects the response.
Association is not causation
A well-designed experiment can give good evidence that changes in the explanatory variable cause changes in the response. An observational study, no matter how large, can show only an association, because of possible confounding variables.
Observational studies are still valuable. Sometimes an experiment would be unethical (you can't assign people to smoke), impractical (you can't assign people to live in a city for 30 years) or impossible (you can't assign someone's age). Observational studies are also a good way to describe a population or to spot associations that are worth testing later.
The language of experiments
Experiments have their own vocabulary, and the AP exam expects you to use it precisely.
- Experimental units are the objects to which treatments are assigned. When the units are people, they are called subjects.
- A treatment is a specific condition applied to the units.
- A factor is an explanatory variable that is manipulated and may cause a change in the response.
- The levels of a factor are the different values it takes in the experiment.
With one factor, the treatments are just its levels. With two or more factors, each treatment is a combination of one level from each factor, so
Worked example: Observational study or experiment?
Decide whether each study is an observational study or an experiment. For any observational study, say whether it is retrospective or prospective.
- Researchers review hospital records from 2015 to 2024 to compare recovery times for patients who did and did not take a certain supplement.
- A teacher randomly assigns half of her 60 students to study with flashcards and half to study by rereading notes, then compares quiz scores.
- A city recruits 3,000 adults, records how many minutes each one walks per day, and tracks their blood pressure for the next ten years.
Solution.
- Observational, retrospective. No treatment was imposed, and the data describe the past.
- Experiment. The teacher imposed the study method on each student.
- Observational, prospective. The adults chose how much to walk; researchers only measured and followed them forward in time.
Worked example: Identifying a confounding variable
An observational study of 1,500 adults finds that those who own a dog have lower average blood pressure than those who do not. A newspaper concludes, "Getting a dog lowers your blood pressure." Explain why this conclusion is not justified and identify a possible confounding variable.
Solution. This is an observational study, so it can show an association but not causation. A possible confounding variable is amount of daily physical activity. People who are already active may be more likely to choose to own a dog (dogs need walks), and physical activity lowers blood pressure on its own. Because dog owners and non-owners may differ in activity level, we can't tell whether the dog or the exercise habits explain the lower blood pressure.
Worked example: Factors, levels and treatments
An agricultural scientist studies tomato yield. She tests 3 types of fertilizer (organic, synthetic, none) and 2 watering schedules (daily, every other day). She has 48 plots of land and wants each treatment to be used on the same number of plots.
(a) Identify the factors, their levels, and the number of treatments.
(b) How many plots receive each treatment?
Solution.
(a) There are two factors: fertilizer type with 3 levels and watering schedule with 2 levels. The experimental units are the plots, and the response variable is tomato yield. The number of treatments is (for example, organic fertilizer watered daily is one treatment).
(b) plots per treatment.
Common mistake
"The researchers compared two groups" does not make a study an experiment. Comparing people who chose to exercise with people who didn't is still observational. The deciding feature of an experiment is that the researchers assigned the treatments.
Practice
Researchers survey 900 high school students about how many hours per night they use their phones after 10 p.m. and compare the answers with the students' reported sleep quality. What type of study is this?
A study finds that children who take music lessons score higher on math tests than children who do not. Which variable is the most plausible confounding variable?
A food scientist tests how baking temperature (325°F, 350°F, 375°F) and baking time (10, 12, 14 or 16 minutes) affect the texture of a cookie. How many treatments are in the experiment?
Enter a number. Fractions like 3/4 and sqrt(2) are OK.
In 2026, researchers enroll 2,000 newly hired nurses, record whether each works mostly night shifts or day shifts, and plan to measure their weight gain every year for the next 8 years. What type of study is this?
A company tests whether a new website layout increases the time visitors spend on its site. Each visitor is randomly shown either the old layout or the new layout, and the number of minutes on the site is recorded. What are the explanatory and response variables?
An observational study of 10,000 adults found that people who drink green tea daily have a lower rate of heart disease. Why can't researchers conclude that green tea prevents heart disease?
A psychologist studies how background noise (silence, or white noise) and room lighting (dim, normal, bright) affect the number of words people can memorize. She wants 10 subjects to receive each treatment, and no subject will receive more than one treatment. How many subjects does she need?
Enter a number. Fractions like 3/4 and sqrt(2) are OK.