Lessons and exercises follow the school system and stay in its language.Les leçons et les exercices suivent le système scolaire et restent dans sa langue.Lektionen und Übungen folgen dem Schulsystem und bleiben in seiner Sprache.

Collecting data

Print Algebra II only

The lessons of this topic taught in Algebra II, ready for paper.

To save a PDF, choose “Save as PDF” in the print dialog.

1. Observational studies, experiments and confounding

Warm-up

Three quick questions from earlier lessons.

  1. A researcher measures the sleep of 376 randomly chosen teenagers and finds a mean of 7.1 hours. What is 7.1?
    1. A statistic: it describes the sample
    2. A parameter: it describes all teenagers
    3. The population
  2. To learn how much US college students spend on books, a team surveys 671 students at 20 colleges. What is the population?
    1. All US college students
    2. The 671 students surveyed
    3. The 20 colleges
  3. Census records show that 52% of all households in a town own a car. What is 52%?
    1. A parameter: it describes the whole population
    2. A statistic
    3. A sample proportion

Discover

Cities with more firefighters at a fire have more damage. Do firefighters cause damage?

  1. No, bigger fires explain both
  2. Yes, the data show it

Key idea

Only experiments can show cause and effect.

In an observational study we record what happens without intervening; in an experiment we impose treatments. A confounding variable is related to both the explanatory and the response variable, so its effect cannot be separated. Observational studies show associations, not causation.

Worked example

Students who take music lessons have higher grades.

Observational study
Nobody assigned the lessons.
Possible confounder: family income
Affects both lessons and grades.
Conclusion: association only
To test cause, assign lessons at random.

Your turn, guided

Your turn: 80 volunteers are split at random, 40 per group. How many get the treatment?

4040
Half of 80.

Practice ★

  1. Researchers randomly assign 66 volunteers to drink coffee or water, then test their reaction times. What kind of study is it?
    1. An experiment: the researchers assign the treatment
    2. An observational study
    3. A sample survey
  1. Children with bigger shoes read better (16% higher scores). What is the most likely explanation?
    1. A confounding variable: age affects both shoe size and reading
    2. Bigger feet improve reading
    3. It must be pure chance

Watch out!

Practice ★★

  1. An observational study of 3416 adults finds that those who eat breakfast weigh less. Does this show that eating breakfast causes lower weight?
    1. No: other differences between the groups could explain it
    2. Yes, because the sample is large
    3. Yes, if the association is strong
  2. An observational study of 5700 adults finds that those who eat breakfast weigh less. Does this show that eating breakfast causes lower weight?
    1. No: other differences between the groups could explain it
    2. Yes, because the sample is large
    3. Yes, if the association is strong

Exit ticket

Three questions to check you have got it.

  1. Researchers randomly assign 66 volunteers to drink coffee or water, then test their reaction times. What kind of study is it?
    1. An experiment: the researchers assign the treatment
    2. An observational study
    3. A sample survey
  2. Researchers randomly assign 166 volunteers to drink coffee or water, then test their reaction times. What kind of study is it?
    1. An experiment: the researchers assign the treatment
    2. An observational study
    3. A sample survey
  3. Researchers randomly assign 46 volunteers to drink coffee or water, then test their reaction times. What kind of study is it?
    1. An experiment: the researchers assign the treatment
    2. An observational study
    3. A sample survey

Summary card

Studies

  • Observational: record. Experiment: assign.
  • Confounding hides cause in observational data.

Watch out: association is not causation.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. In an experiment, 200 randomly chosen people get a drug and 200 get a placebo. 73 and 58 people improve. By how many percentage points is the drug group's rate higher?

2. Designing experiments: randomization, replication, blocking

Warm-up

Three quick questions from earlier lessons.

  1. Researchers randomly assign 66 volunteers to drink coffee or water, then test their reaction times. What kind of study is it?
    1. An experiment: the researchers assign the treatment
    2. An observational study
    3. A sample survey
  2. Children with bigger shoes read better (16% higher scores). What is the most likely explanation?
    1. A confounding variable: age affects both shoe size and reading
    2. Bigger feet improve reading
    3. It must be pure chance
  3. An observational study of 3416 adults finds that those who eat breakfast weigh less. Does this show that eating breakfast causes lower weight?
    1. No: other differences between the groups could explain it
    2. Yes, because the sample is large
    3. Yes, if the association is strong

Discover

To compare two fertilizers, you put fertilizer A on the sunny half of a field and B on the shady half. A grows better. What is the flaw?

  1. Sunlight, not the fertilizer, could explain the difference
  2. There is no flaw

Key idea

Principles of experimental design.

Comparison of treatments (often with a control or placebo group); random assignment of units to treatments; replication on enough units; blocking groups similar units first and randomizes within each block. With several factors, the treatments are all combinations of their levels.

Worked example

Test 2 doses × 3 diets on 60 rats.

Treatments: 2×3=62 \times 3 = 6
Each combination.
10 rats per treatment
Replication.
Shuffle the rats into the 6 groups
Random assignment.

Your turn, guided

Your turn: 3 temperatures × 4 times. Number of treatments.

1212
3 × 4.

Practice ★

  1. Why are 196 subjects assigned to the treatments at random?
    1. To balance other variables between the groups, so differences can be blamed on the treatment
    2. So the results apply to the whole population
    3. To give every subject the same chance of being in the study
  1. An experiment tests 2 fertilizer types at 3 watering levels, with every combination used. How many treatments are there?
  2. An experiment tests 9 fertilizer types at 12 watering levels, with every combination used. How many treatments are there?

Watch out!

Practice ★★

  1. A test of two diets on 115 adults first splits them into men and women, then randomizes diets within each group. What is this design?
    1. A randomized block design, blocking by sex
    2. A stratified random sample
    3. A completely randomized design

Exit ticket

Three questions to check you have got it.

  1. An experiment tests 2 fertilizer types at 3 watering levels, with every combination used. How many treatments are there?
  2. An experiment tests 9 fertilizer types at 12 watering levels, with every combination used. How many treatments are there?
  3. An experiment tests 8 fertilizer types at 9 watering levels, with every combination used. How many treatments are there?

Summary card

Experimental design

Compare · randomize · replicate · block. Treatments = combinations of factor levels.

Watch out: blocks (experiments) vs strata (samples).

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. An experiment crosses 2 fertilizers with 2 watering levels, and each treatment is used on 3 plants. How many plants are needed?

3. History of experiments: Fisher at Rothamsted

Discover

At an agricultural station, a scientist wants to know if a new fertilizer works. Some plots get it, others don't. How should he choose which plots get it?

  1. At random
  2. Give it to the best-looking plots

Key idea

Experiments, from fields to medicine.

1840186018801900192019401837Rothamsted1837Rothamsted
  1. Step 11840186018801900192019401837Rothamsted1837Rothamsted1837: the Rothamsted station is founded to study soils and plant growth.
  2. Step 21840186018801900192019401837Rothamsted1890Fisher is born1837Rothamsted1890Fisher is bornRonald Fisher 1890–1962Barry Eagel, 1952. Barry Eagel. CC BY-SA 4.01890: Ronald Fisher is born; at Rothamsted he introduces randomization.
  3. Step 31840186018801900192019401837Rothamsted1890Fisher is born1935Fisher's The Design of Experiments1837Rothamsted1890Fisher is born1935Fisher's The Design of ExperimentsRonald Fisher 1890–1962Barry Eagel, 1952. Barry Eagel. CC BY-SA 4.01935: Fisher's The Design of Experiments.

At the Rothamsted Agricultural Experiment Station (founded 1837), Ronald Fisher (1890–1962) studied the design of experiments, introducing randomization and the analysis of variance. He collected his methods in The Design of Experiments (1935).

Worked example

20 plots, 10 to get the fertilizer. How many different random assignments are possible?

(2010)\dbinom{20}{10}
Choose 10 plots out of 20.
=184 756= 184\,756 possible assignments
Compute.

Practice ★

  1. What did Fisher introduce in the design of experiments at Rothamsted?
    1. Randomization and the analysis of variance
    2. Polls of voters
    3. The normal curve
  1. 4 plots are split at random into a treatment group and a control group of 2 each. How many assignments are possible?
  2. 6 plots are split at random into a treatment group and a control group of 3 each. How many assignments are possible?
  1. Like Fisher: 3 fertilizers are compared, each on 9 randomly chosen plots. How many plots are needed?

Summary card

Experiments today

  • Medical trials assign treatments at random.
  • Tech companies test website changes with random A/B splits.
  • Farmers still compare seeds and fertilizers on randomized plots.

Sources: MacTutor biography of Fisher (see "Sources and credits").

4. Randomized response

Discover

A survey asks 'Have you ever cheated on a test?'. Many people would lie. Here is a trick: each person flips a coin in secret. Heads: answer truthfully. Tails: just say 'yes'. Can we still estimate the true rate?

  1. Yes: we know half the 'yes' answers come from tails
  2. No: the coin destroys the information

Key idea

Randomized response.

With a fair coin (heads: truth, tails: 'yes'):

P(yes)=12p+12P(\text{yes}) = \tfrac{1}{2}p + \tfrac{1}{2}, \quad so p=2 P(yes)−1\quad p = 2\,P(\text{yes}) - 1,

where pp is the true proportion. Randomness protects each person but not the overall estimate.

Worked example

62 % of the answers are 'yes'.

p=2×0.62−1p = 2 \times 0.62 - 1
Use the formula.
p=0.24p = 0.24: about 24 %
Compute.

Your turn, guided

Your turn: 70 % 'yes'. Estimate pp.

0.40.4
2×0.7−12 \times 0.7 - 1.

Practice ★

  1. Randomized response with a fair coin: 71 % of the answers are 'yes'. Estimate the true proportion (as a decimal).
  2. Randomized response with a fair coin: 69 % of the answers are 'yes'. Estimate the true proportion (as a decimal).
  3. Randomized response with a fair coin: 94 % of the answers are 'yes'. Estimate the true proportion (as a decimal).

Practice ★★

  1. Randomized response: 500 people answer and 420 say 'yes'. About how many of them truly are 'yes' people?
  2. Randomized response: 400 people answer and 216 say 'yes'. About how many of them truly are 'yes' people?

Exit ticket

Two questions to check.

  1. Randomized response with a fair coin: 71 % of the answers are 'yes'. Estimate the true proportion (as a decimal).
  2. Randomized response with a fair coin: 69 % of the answers are 'yes'. Estimate the true proportion (as a decimal).

Summary card

Randomized response

  • p=2P(yes)−1p = 2P(\text{yes}) - 1 with a fair coin.
  • Privacy for each person, accuracy for the group.

Statistics offices use variants of this idea to protect people's privacy.