Lessons and exercises follow the school system and stay in its language.Les leçons et les exercices suivent le système scolaire et restent dans sa langue.Lektionen und Übungen folgen dem Schulsystem und bleiben in seiner Sprache.
Every lesson of the chapter, ready for paper. Exercise sets are lettered A, B, C… and answered at the end.
To save a PDF, choose “Save as PDF” in the print dialog.
Statistics (AP-aligned)
Inference for regression
NicknameDate
Scan to open onlineenglish.sedifier.com/en-us/us-ma-2017/statistics/us.stats.regression-inference/
1. The regression model and its conditions
Warm-up
Three quick questions from earlier lessons.
A company claims that 80% of customers are satisfied. A researcher suspects fewer are. Which is the null hypothesis?
H₀: p = 0.80
H₀: p < 0.80
H₀: p̂ = 0.80
The mean wait at a clinic used to be 81 minutes. A manager wants to know whether it has changed, in either direction. Which alternative?
Hₐ: μ ≠ 81
Hₐ: μ > 81
Hₐ: x̄ ≠ 81
Why are hypotheses written about parameters such as p, not statistics such as p̂ (sample of 445)?
Because we know p̂ exactly; the question is about the unknown population value
Because statistics are harder to compute
There is no difference between p and p̂
Discover
Two random samples of 30 students give regression slopes 2.1 and 2.6 for hours studied vs score. Why do they differ?
Sampling variability: each sample estimates the same true slope
One of the samples must be wrong
Key idea
A model for the population.
Population regression line:μy=β0+β1x, estimated by y^=a+bx. Conditions for inference (LINER): Linear relationship, Independent observations, Normal residuals, Equal spread of residuals, Random sample or assignment.
Worked example
Checking the conditions with a residual plot.
Linear: residuals scattered around 0
No curve.
Equal spread across x
No fan.
Roughly normal: no strong skew or outliers
Histogram of residuals.
Your turn, guided
Your turn: how many conditions are in LINER?
5
L, I, N, E, R.
Practice ★
A regression on a random sample of 23 houses gives slope b. What does b estimate?
The slope β₁ of the population regression line
Nothing: b is the true slope
The correlation r
Which conditions are needed for inference about the slope from 75 pairs? Tick all that are.
The true relationship is linear
Observations are independent (random sample, 10% condition)
Residuals are roughly normal
Residuals have about equal spread for all x
The correlation is above 0.5
Watch out!
Practice ★★
The residual plot of a regression on 85 points fans out: residuals spread more as x grows. Which condition fails?
Equal variance of the residuals
Linearity
Normality of the population
Exit ticket
Three questions to check you have got it.
A regression on a random sample of 23 houses gives slope b. What does b estimate?
The slope β₁ of the population regression line
Nothing: b is the true slope
The correlation r
A regression on a random sample of 59 houses gives slope b. What does b estimate?
The slope β₁ of the population regression line
Nothing: b is the true slope
The correlation r
A regression on a random sample of 46 houses gives slope b. What does b estimate?
One more step is optional: a challenge. Try it, or finish the lesson now.
Challenge ★★★
The regression model gives ŷ = 17 + 0.4x. By how much does the predicted y change when x increases by 10?
2. The t statistic for the slope
Warm-up
Three quick questions from earlier lessons.
A regression on a random sample of 23 houses gives slope b. What does b estimate?
The slope β₁ of the population regression line
Nothing: b is the true slope
The correlation r
Which conditions are needed for inference about the slope from 75 pairs? Tick all that are.
The true relationship is linear
Observations are independent (random sample, 10% condition)
Residuals are roughly normal
Residuals have about equal spread for all x
The correlation is above 0.5
The residual plot of a regression on 85 points fans out: residuals spread more as x grows. Which condition fails?
Equal variance of the residuals
Linearity
Normality of the population
Discover
A sample slope is b = 0.6 with standard error 0.2. How many standard errors is it from 0?
3
0.12
Key idea
Standardize the slope.
t=SEbb−β1(0), usually with β1(0)=0, and df = n−2. Computer output lists b (Coef), SEb (SE Coef), t and the P-value in the row of x.
Worked example
Output for x: Coef 1.25, SE Coef 0.50, n=22.
t=2.5
1.25/0.50.
df =20
22 − 2.
P-value ≈0.021
Two-sided, from the t distribution.
Your turn, guided
Your turn: b = 0.9, SE_b = 0.3. Find t.
t=3
0.9/0.3.
Practice ★
Computer output gives slope b = 8/10 with standard error SE_b = 3/100. Find the t statistic for H₀: β₁ = 0.
Computer output gives slope b = 9/10 with standard error SE_b = 8/100. Find the t statistic for H₀: β₁ = 0.
A regression uses 20 data pairs. How many degrees of freedom does the t test for the slope have?
Watch out!
Practice ★★
In regression output, the row for x reads: Coef 0.97, SE Coef 0.08, and a t value. Which number is the slope's standard error?
The SE Coef in the x row
The Coef in the x row
The value s (the SD of the residuals)
Exit ticket
Three questions to check you have got it.
Computer output gives slope b = 8/10 with standard error SE_b = 3/100. Find the t statistic for H₀: β₁ = 0.
Computer output gives slope b = 9/10 with standard error SE_b = 8/100. Find the t statistic for H₀: β₁ = 0.
Computer output gives slope b = 7/10 with standard error SE_b = 3/100. Find the t statistic for H₀: β₁ = 0.
Summary card
t for the slope
t=b/SEb, df =n−2.
Watch out: divide by SEb; two parameters, so n−2.
One more step is optional: a challenge. Try it, or finish the lesson now.
Challenge ★★★
Output gives b = 3/10 with SE_b = 4/100. With a larger sample, SE_b is halved and b stays the same. Find the new t statistic for H₀: β₁ = 0.
3. Confidence interval for the slope
Warm-up
Three quick questions from earlier lessons.
A random sample of 25 has standard deviation s = 16. With t* = 2.064 (24 df, 95%), find the margin of error for the mean.
A one-sample t interval uses a sample of 10. How many degrees of freedom?
Why is a t critical value used for a confidence interval for a mean from a sample of 40?
Because σ is estimated by s, which adds variability
Because n is less than 1,000
Because the data are skewed
Discover
b = 0.5, SE_b = 0.1, t* = 2. Which interval estimates β₁?
(0.3, 0.7)
(0.4, 0.6)
Key idea
An interval for the population slope.
b±t∗⋅SEb, with t∗ from the t distribution with n−2 df. If the interval does not contain 0, there is convincing evidence of a linear relationship; interpret as the change in mean y per unit increase in x.
Worked example
n=30, b=0.42, SEb=0.10, t∗=2.048.
ME=0.2048
2.048 × 0.10.
(0.215,0.625)
0.42 ± 0.2048.
Evidence of a positive slope
0 excluded.
Your turn, guided
Your turn: ME = 0.15. The width of the interval.
0.3
2 × 0.15.
Practice ★
With 30 pairs, b = 2/10 and SE_b = 7/100. Using t* = 2.048 (28 df, 95%), find the upper end of the interval for β₁.
With 30 pairs, b = 4/10 and SE_b = 1/100. Using t* = 2.048 (28 df, 95%), find the upper end of the interval for β₁.
SE_b = 1/100 and t* = 2.048. Find the width of the 95% confidence interval for β₁.
Watch out!
Practice ★★
A 95% interval for the slope β₁ is (0.4, 0.8). What does it suggest?
Convincing evidence of a positive linear relationship: 0 is not in the interval
No evidence of a relationship
x causes y to increase
Exit ticket
Three questions to check you have got it.
With 30 pairs, b = 2/10 and SE_b = 7/100. Using t* = 2.048 (28 df, 95%), find the upper end of the interval for β₁.
With 30 pairs, b = 4/10 and SE_b = 1/100. Using t* = 2.048 (28 df, 95%), find the upper end of the interval for β₁.
With 30 pairs, b = 3/10 and SE_b = 6/100. Using t* = 2.048 (28 df, 95%), find the upper end of the interval for β₁.
Summary card
Interval for β₁
b±t∗SEb (df n−2); 0 outside → evidence of a relationship.
Watch out: association, not causation.
One more step is optional: a challenge. Try it, or finish the lesson now.
Challenge ★★★
A 95% confidence interval for the slope is (0.2, 0.5). What is the point estimate b?
4. Testing the slope and reading output
Warm-up
Three quick questions from earlier lessons.
Computer output gives slope b = 8/10 with standard error SE_b = 3/100. Find the t statistic for H₀: β₁ = 0.
A regression uses 20 data pairs. How many degrees of freedom does the t test for the slope have?
In regression output, the row for x reads: Coef 0.97, SE Coef 0.08, and a t value. Which number is the slope's standard error?
The SE Coef in the x row
The Coef in the x row
The value s (the SD of the residuals)
Discover
If x is no help in predicting y, what is the population slope?
0
1
Key idea
Test whether the slope is zero.
H0:β1=0 (no linear relationship), Ha:β1=0 (or one-sided). Use t=b/SEb, df =n−2. A small P-value is convincing evidence of a linear relationship in the population. Read b, SEb, t and P from the output.
Worked example
Output: b = 0.30, t = 6.0, P < 0.001.
SEb=0.30/6=0.05
t = b/SE_b.
Reject H₀
Very small P.
Convincing evidence of a linear relationship
In context.
Your turn, guided
Your turn: b = 0.8 and t = 4. Find SEb.
0.2
0.8/4.
Practice ★
To test whether x is useful for predicting y (from 51 pairs), what is the null hypothesis?
H₀: β₁ = 0, no linear relationship in the population
H₀: b = 0
H₀: r = 1
Output shows slope b = 6/10 and t = 9 for H₀: β₁ = 0. Find SE_b.
Output shows slope b = 3/10 and t = 9 for H₀: β₁ = 0. Find SE_b.
Watch out!
Practice ★★
An observational study tests H₀: β₁ = 0 for hours of sleep (x) and test score (y). The P-value is 0.006. What can you conclude?
Convincing evidence of a linear association between sleep and scores in the population sampled
More sleep causes higher scores
The correlation must be close to 1
Exit ticket
Three questions to check you have got it.
Output shows slope b = 6/10 and t = 9 for H₀: β₁ = 0. Find SE_b.
Output shows slope b = 3/10 and t = 9 for H₀: β₁ = 0. Find SE_b.
Output shows slope b = 3/10 and t = 6 for H₀: β₁ = 0. Find SE_b.
Summary card
Testing the slope
H₀: β₁ = 0; t=b/SEb, df =n−2.
Watch out: a significant slope is not proof of causation.
One more step is optional: a challenge. Try it, or finish the lesson now.
Challenge ★★★
The t statistic for H₀: β₁ = 0 leaves an area of 0.008 in one tail. Find the two-sided P-value.
5. Slope and r² from r
Discover
The regression line of height on arm span has r=0.8. What part of the variation in height does the line explain?
r2=0.64, about 64 %
0.8, 80 %
Key idea
Slope and r2 from r.
For the least-squares line of y on x:
b=rsxsy and r2 = fraction of the variation in y explained by x.
Worked example
r=0.6, sx=5, sy=10.
b=0.6×510=1.2
Slope.
r2=0.36: 36 %
Explained.
Your turn, guided
Your turn: r=0.5, sx=2, sy=8. Find b.
b=2
0.5×28.
Practice ★
r=51, sx=9, sy=27. Find the slope of the regression line of y on x.
r=51, sx=1, sy=9. Find the slope of the regression line of y on x.
r=54, sx=6, sy=48. Find the slope of the regression line of y on x.
Practice ★★
A regression has r=−53. What percent of the variation in y is explained by x?
A regression has r=−107. What percent of the variation in y is explained by x?
Exit ticket
Two questions to check.
r=51, sx=9, sy=27. Find the slope of the regression line of y on x.
r=51, sx=1, sy=9. Find the slope of the regression line of y on x.
Summary card
r, slope and r²
b=rsxsy.
r2 is the share explained.
Computer output shows 'R-sq': it is this r2, as a percent.
6. History of regression: Galton and Pearson
Discover
Very tall parents tend to have tall children, but usually a bit less tall. In the 1880s, Francis Galton gave this a name. Which?
Regression toward the mean
Correlation
Key idea
From heredity to inference.
Step 1Francis Galton1822–1911Eveleen Myers (née Tennant), 1890s. Public domainGalton (born 1822) names regression to the mean.
Step 2Karl Pearson1857–1936Unknown artist, 1910. Public domainKarl Pearson (1857–1936): regression analysis and the correlation coefficient.
Step 31900: the chi-square test; October 1901: first volume of Biometrika.
Francis Galton (born 1822) invented the idea of 'regression to the mean', first calling it 'reversion'. Karl Pearson (1857–1936) contributed to regression analysis and the correlation coefficient, and in 1900 the chi-square test. With Weldon and Galton he founded the journal Biometrika, first published in October 1901.
Worked example
Galton's idea: slope 0.6, parents 10 cm above the mean height.
0.6×10=6 cm above the mean
Predicted child.
regression
Closer to the mean.
Practice ★
What did Galton first call regression to the mean?
Reversion
Correlation
Least squares
Galton's line has slope 0.4. A father is 11 cm above the mean. How many cm above the mean is his son predicted to be?
Galton's line has slope 0.7. A father is 18 cm above the mean. How many cm above the mean is his son predicted to be?
A regression line is y^=12+0.6x. Predict y for x=8.
Summary card
Regression today
Economists test whether a slope is really different from 0.
Medicine: does a dose change blood pressure?
Sports analytics: performance regresses toward the mean.
Sources: MacTutor biographies of Galton and Pearson (see "Sources and credits").
Answers: Inference for regression
1. The regression model and its conditions
A
H₀: p = 0.80
Hₐ: μ ≠ 81
Because we know p̂ exactly; the question is about the unknown population value
B
The slope β₁ of the population regression line
C
The true relationship is linear · Observations are independent (random sample, 10% condition) · Residuals are roughly normal · Residuals have about equal spread for all x
D
Equal variance of the residuals
E
The slope β₁ of the population regression line
The slope β₁ of the population regression line
The slope β₁ of the population regression line
F
4
2. The t statistic for the slope
A
The slope β₁ of the population regression line
The true relationship is linear · Observations are independent (random sample, 10% condition) · Residuals are roughly normal · Residuals have about equal spread for all x
Equal variance of the residuals
B
380
11.25
C
18
D
The SE Coef in the x row
E
380
11.25
370
F
15
3. Confidence interval for the slope
A
6.6048
9
Because σ is estimated by s, which adds variability
B
0.34336
0.42048
C
0.04096
D
Convincing evidence of a positive linear relationship: 0 is not in the interval
E
0.34336
0.42048
0.42288
F
207
4. Testing the slope and reading output
A
380
18
The SE Coef in the x row
B
H₀: β₁ = 0, no linear relationship in the population
C
151
301
D
Convincing evidence of a linear association between sleep and scores in the population sampled