Significance tests for a proportion

A confidence interval asks "what values of $p$ are plausible?" A significance test asks a sharper question: "Is this particular claim about $p$ believable, given my data?" Tests are how researchers decide whether an effect is real or could just be sampling variability.

The logic of a significance test

A company claims that 30% of its customers use its mobile app. A manager suspects the true figure is higher. She takes a random sample of 200 customers and finds that 74 use the app, so $\hat{p} = 0.37$.

Is 0.37 convincing evidence that $p > 0.30$? Even if the true proportion were exactly 0.30, random samples would sometimes give $\hat{p}$ values as large as 0.37. The test measures how unusual 0.37 would be if the claim were true. If it would be very unusual, the claim looks doubtful.

This is proof by contradiction with probability: assume the claim is true, then ask whether the data are surprising under that assumption.

Hypotheses

Hypotheses are always about the parameter $p$, never about $\hat{p}$. You decide on $H_a$ from the research question before looking at the data. For the app example, $H_0: p = 0.30$ and $H_a: p > 0.30$, where $p$ is the proportion of all the company's customers who use the app.

The test statistic

To measure how far $\hat{p}$ is from $p_0$, standardize it. Because you are assuming $H_0$ is true, use $p_0$ (not $\hat{p}$) in the standard deviation:

$$z = \frac{\hat{p} - p_0}{\sqrt{\dfrac{p_0(1-p_0)}{n}}}.$$

The test statistic tells you how many standard deviations $\hat{p}$ is from the null value.

The P-value

• $H_a: p > p_0$: P-value $= P(Z \ge z)$.
• $H_a: p < p_0$: P-value $= P(Z \le z)$.
• $H_a: p \ne p_0$: P-value $= 2P(Z \ge |z|)$.

A small P-value means the observed result would rarely happen by chance alone if $H_0$ were true, which is evidence against $H_0$. Here is an excerpt of a standard Normal table, giving the area to the left of $z$:

$z$	1.80	1.90	2.00	2.09	2.16	2.30	2.32
Area left	0.9641	0.9713	0.9772	0.9817	0.9846	0.9893	0.9898

By symmetry, the area to the left of $-z$ equals the area to the right of $z$, which is $1$ minus the table value.

Making a decision

Before collecting data, choose a significance level $\alpha$, commonly 0.05. Then compare:

Conditions and the four steps

The conditions match those for intervals, with one change: check Large Counts using $p_0$, because the test assumes $H_0$ is true.

1. Random: random sample or randomized experiment.
2. 10%: $n \le 0.10N$ when sampling without replacement.
3. Large Counts: $np_0 \ge 10$ and $n(1 - p_0) \ge 10$.

The four steps become:

• State: hypotheses, parameter in context, and $\alpha$.
• Plan: name the one-sample $z$-test for $p$ and check conditions.
• Do: compute $\hat{p}$, $z$, and the P-value.
• Conclude: compare the P-value to $\alpha$, make a decision, and state it in context.

Interpreting a P-value

On the AP exam you may be asked to interpret a P-value in context. Use this template: "Assuming [$H_0$ in context] is true, there is a [P-value] probability of getting a sample proportion of [$\hat{p}$] or [more extreme direction] by chance alone." For the app example: assuming 30% of customers use the app, there is about a 0.0154 probability of getting a sample proportion of 0.37 or higher in a random sample of 200.

Tests and intervals agree

A two-sided test at significance level $\alpha$ and a $C = 1 - \alpha$ confidence interval tell a consistent story: if $p_0$ lies outside the interval, a two-sided test would reject $H_0: p = p_0$; if $p_0$ lies inside, the test would fail to reject. The interval gives extra information, a whole range of plausible values, so it is often worth reporting both.

Practice

Practice 1

A city once found that 45% of residents recycle regularly. After a new education campaign, officials want to know if the proportion has increased. Which hypotheses are appropriate?

A$H_0: \hat{p} = 0.45$, $H_a: \hat{p} > 0.45$B$H_0: p > 0.45$, $H_a: p = 0.45$C$H_0: p = 0.45$, $H_a: p > 0.45$D$H_0: p = 0.45$, $H_a: p \ne 0.45$

CheckShow hint

Practice 2

A website says 30% of visitors click on its daily deal. After a redesign, a random sample of 150 visitors shows that 58 clicked. For testing $H_0: p = 0.30$ versus $H_a: p > 0.30$, compute the test statistic $z$. Round to two decimal places.

Your answer

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

CheckShow hint

Practice 3

Continuing the previous problem ($z \approx 2.32$, $H_a: p > 0.30$), find the P-value. Round to four decimal places.

Your answer

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

CheckShow hint

Practice 4

For the website test ($H_0: p = 0.30$, $H_a: p > 0.30$, P-value $\approx 0.010$), which conclusion is correct at $\alpha = 0.05$?

AReject $H_0$. There is convincing evidence that more than 30% of visitors click on the daily deal after the redesign.BReject $H_0$. The redesign has been proven to cause exactly 38.7% of visitors to click.CFail to reject $H_0$. There is convincing evidence that 30% of visitors click on the daily deal.DFail to reject $H_0$. The P-value is smaller than $\alpha$.

Check

Practice 5

A coin-flipping app claims to be fair. In 800 simulated flips it produced 412 heads. Test $H_0: p = 0.5$ versus $H_a: p \ne 0.5$, where $p$ is the long-run proportion of heads. Find the P-value, rounded to three decimal places.

Your answer

Enter a number. Fractions like 3/4 and sqrt(2) are OK.

CheckShow hint (1/2)

Practice 6

A study tested $H_0: p = 0.6$ against $H_a: p < 0.6$, where $p$ is the proportion of adults who exercise weekly, and got $\hat{p} = 0.55$ with a P-value of 0.04. Which is the correct interpretation of the P-value?

AThere is a 0.04 probability that the true proportion is 0.6.BAssuming the true proportion is 0.6, there is a 0.04 probability of getting a sample proportion of 0.55 or lower by chance alone.CThere is a 0.04 probability that the alternative hypothesis is true.DAssuming the true proportion is 0.55, there is a 0.04 probability of getting a sample proportion of 0.6 or lower.

Check

Practice 7

A national survey found that 20% of adults had donated blood in the past five years. A random sample of 250 adults in one county found 38 donors ($\hat{p} = 0.152$). For $H_0: p = 0.20$ versus $H_a: p < 0.20$, the test gives $z \approx -1.90$ and P-value $\approx 0.029$. What is the correct decision at $\alpha = 0.01$?

AReject $H_0$, because the P-value is less than 0.05.BReject $H_0$, because the sample proportion is less than 0.20.CFail to reject $H_0$; this proves the county's donation rate is 20%.DFail to reject $H_0$; there is not convincing evidence that fewer than 20% of the county's adults donated blood.

CheckShow hint

Practice 8

A 95% confidence interval for the proportion of voters who favor a ballot measure is $(0.46,\ 0.53)$. Based on this interval, what can you conclude about the two-sided test of $H_0: p = 0.5$ at $\alpha = 0.05$?

AReject $H_0$, because 0.5 is near the center of the interval.BReject $H_0$, because the interval includes values below 0.5.CFail to reject $H_0$, because 0.5 is a plausible value inside the interval.DNo conclusion is possible without the P-value.

Check