SAT Math Review · Problem-Solving and Data Analysis
Evaluating statistical claims: samples, experiments and causation
Four conclusions are offered and three of them say more than the study can support. The whole item is deciding how far the result is allowed to travel.
Which conclusion is most appropriate?
To estimate the mean number of hours per week that residents of a town exercise, a researcher surveyed 200 residents selected at random from the membership list of the town's recreation center. The mean for the sample was 6.4 hours per week.
Random selection lets a result generalize to the group the sample was drawn from, and that group here is the recreation center's members, not the town. A treats "random" as a magic word: the selection was random within the wrong list, and members exercise more than residents in general. B drops the uncertainty and reports a sample mean as the population's exact mean. D over-corrects — the sampling was genuinely random, so the results do reach the membership; they simply stop there.
1
Every study item is two questions wearing one coat.
How far does this result reach? And is one thing shown to have caused the other? They have different answers, and each one has its own piece of the study description.
random selection → how far it reachesrandom assignment → whether it caused
selectionwho got into the study"200 chosen at random from…"
assignmentwho got which treatment"randomly assigned to two groups"
A study can have both, one, or neither. Read the description once for each word before you look at a single choice.
2
The list the sample came from is the largest population you may talk about.
Not the population the researcher was curious about — the one the names were actually drawn from.
The sample
Largest population
400 chosen at random from a company's 6,000 employees
all 6,000 employees
220 chosen at random from a photography club's members
that club's members — not everyone who takes photographs
300 shoppers who agreed to be stopped outside one store
nobody beyond those 300
80 volunteers who answered an online advertisement
nobody beyond those 80
Two ways the choices push you off the list
Upwards: a sample of club members offered as evidence about "all adults in the country". Downwards: a genuinely random sample described as applying only to the people surveyed. A random sample reaches its whole list, and stops there.
3
Only random assignment licenses the word "causes".
When a researcher decides who gets the treatment by chance, the two groups start out alike in every respect — age, fitness, motivation, everything nobody thought to measure. Any difference at the end has only the treatment left to explain it.
Study design
Generalises to
Supports "causes"?
Random sample, nothing assigned (observational)
that population
no — association only
Volunteers, nothing assigned
only those people
no — association only
Volunteers, randomly assigned to treatments
only those people
yes, for those people
Random sample, randomly assigned
that population
yes
The two questions are answered by two different sentences in the stem, so the correct choice is often a half-and-half: "the treatment likely caused the difference for these participants, but the result cannot be generalized beyond them."
A group that chose its own behaviour — who eats breakfast, who takes the course, who sleeps seven hours — was never randomly assigned, no matter how large the study or how the participants were found.
4
An association has three other explanations.
"Students who use the tutoring center have lower grades than students who do not" is a real pattern in real data. Here is everything it could mean.
causetutoring lowers gradespossible, but not shown
reverselow grades send students to tutoringthe arrow runs the other way
third variablea harder course load does botha confounding variable
chancethis sample happened to differwhy a margin of error is reported
Observational data cannot tell these four apart, which is the whole content of "correlation does not imply causation". The correct choice usually says "there is an association … but a cause-and-effect relationship cannot be concluded".
Do not over-correct either
"There is no association between tutoring and grades" is just as wrong: an association was measured and it is real. And "no conclusion can be drawn" is wrong when the sample was random — the association itself generalises perfectly well.
5
A confounding variable moves with both of the things being compared.
A company reports that the employees who take its optional wellness class have fewer sick days than the employees who do not. The employees who opt in are also, on average, younger.
the class group is younger
the groups differed before the class began
younger employees take fewer sick days anyway
age alone would produce the same pattern
so the class and age are tangled
the study cannot separate them
What would have fixed it
Assign the employees to take the class or not by chance. Then the two groups have similar ages, similar jobs and similar everything-else to begin with, and the confounding variable is gone — not measured away, but designed away. That is why the SAT treats random assignment as the thing that licenses a causal claim.
6
Score the four choices instead of searching for the right one.
Three of them will overstate something. Run each choice past these three tests and the survivor is the answer.
Test
Reject the choice if…
Population
it names a group wider than the list the sample came from
Causation
it says causes, improves, leads to or reduces, with no random assignment
Certainty
it says exactly, all, every, always, proves or impossible
A fourth pattern is worth knowing: a choice that denies the association altogether ("there is no relationship because the study was not an experiment") is always wrong. The association was measured; only its explanation is in doubt.
The correct choice is usually the cautious one
It tends to be the longest, and it tends to contain a "but": an association but no cause, a cause for these participants but no wider claim. That is not a trick — it is what a careful reading of the design actually supports.
✓
Check: two words, two verdicts.
the openerrandom selection from the member listmembers yes, town no
no assignmentnobody was told to exerciseso no causal claim was on offer
the wellness classvolunteers, age confoundedassociation only ✓
Remember the flow
Selection sets the reach · Assignment sets the cause · Everything else is an association
If nobody was assigned anything, no choice containing "causes" can be right.
Now you try
Practice problem
Which conclusion is most appropriate?
A study of 1,500 adults selected at random from a country found that the adults who ate breakfast every day had a lower body mass index, on average, than the adults who did not. The researchers also reported that the adults who ate breakfast every day exercised more, on average, than those who did not.
The adults were selected at random from the country, so the association generalizes to adults in the country; but nobody was assigned a breakfast habit, and exercise moves with both variables, so exercise is a confounding variable and no cause can be claimed. A reads a cause out of an observational study.B goes too far the other way — the study cannot show that there is no effect either, only that it has not isolated one. D confuses the two questions: it is random assignment that is missing, and random selection, which is present, is what carries the result to the country's adults.
Hint: answer "how far?" and "caused?" separately before you read the choices, then find the one choice that gives both of your answers.
