Conclusions, errors and power

Lessons and exercises follow the school system and stay in its language.Les leçons et les exercices suivent le système scolaire et restent dans sa langue.Lektionen und Übungen folgen dem Schulsystem und bleiben in seiner Sprache.

About this lesson

Statistics (AP-aligned) › Significance tests

To save a PDF, choose “Save as PDF” in the print dialog.

Sources and credits

Written with this source as a starting point:

sedifier rewrote the text and added its own examples and exercises.

Written with AI assistance; every answer is checked by our math engine. Not yet reviewed by a teacher.

This lesson: CC BY-SA 4.0, © sedifier · All credits

Conclusions, errors and power

Significance tests.

Warm-up

Three quick questions from earlier lessons.

  1. Test H₀: p = 0.5 with a random sample of 100 in which p̂ = 52/100. Find the z statistic.
  2. Test H₀: p = 0.2 with n = 400 and p̂ = 28/100. Find the z statistic.
  3. A test gives a P-value of 0.05. What does it mean?
    1. If H₀ were true, a result at least as extreme as ours would happen with probability 0.05
    2. There is a 0.05 probability that H₀ is true
    3. The result is due to chance with probability 0.05

Discover

At α = 0.05 a test gives P-value 0.30. Does this prove H₀ is true?

  1. No: we just lack evidence against it
  2. Yes

Key idea

Decide, and know what can go wrong.

If P-value ≤α\le \alpha, reject H₀ (convincing evidence for Hₐ); otherwise fail to reject H₀. Type I error: rejecting a true H₀ (probability α). Type II error: failing to reject a false H₀. Power = probability of correctly rejecting a false H₀; it grows with nn and with α.

Worked example

H₀: a new teaching method is no better.

Type I: adopting a method that is not better
Reject a true H₀.
Type II: keeping the old method when the new one is better
Miss a real improvement.
Higher power, fewer Type II errors
More students in the study.

Your turn, guided

Your turn: α = 0.01. What is the probability of a Type I error when H₀ is true?

0.010.01
It equals α.

Practice ★

  1. At significance level α = 0.05, a test gives P-value 0.04. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true
  1. A test of a new drug (sample of 388) has H₀: the drug has no effect. What is a Type I error?
    1. Concluding the drug works when it really has no effect
    2. Concluding there is no effect when the drug really works
    3. Making a mistake in the data

Watch out!

Practice ★★

  1. How can a researcher increase the power of a test that now uses 82 subjects?
    1. Use a larger sample
    2. Use a smaller α
    3. Use fewer subjects

Exit ticket

Three questions to check you have got it.

  1. At significance level α = 0.05, a test gives P-value 0.04. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true
  2. At significance level α = 0.05, a test gives P-value 0.03. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true
  3. At significance level α = 0.05, a test gives P-value 0.02. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true

Summary card

Conclusions and errors

  • P ≤ α: reject H₀. Otherwise: fail to reject.
  • Type I: reject true H₀. Type II: keep false H₀.
  • Power rises with nn.

Watch out: never "accept" or "prove" H₀.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. A test uses significance level α = 0.07. If H₀ is true, what is the probability of a Type I error?
Sources and credits

Written with this source as a starting point:

sedifier rewrote the text and added its own examples and exercises.

Written with AI assistance; every answer is checked by our math engine. Not yet reviewed by a teacher.

This lesson: CC BY-SA 4.0, © sedifier · All credits