Lessons and exercises follow the school system and stay in its language.Les leçons et les exercices suivent le système scolaire et restent dans sa langue.Lektionen und Übungen folgen dem Schulsystem und bleiben in seiner Sprache.

Exploring two-variable data

Print the chapter

Every lesson of the chapter, ready for paper. Exercise sets are lettered A, B, C… and answered at the end.

To save a PDF, choose “Save as PDF” in the print dialog.

1. Two-way tables: joint, marginal and conditional proportions

Warm-up

Three quick questions from earlier lessons.

  1. A survey of 23 students records the following. Tick all the categorical variables.
    1. Eye color
    2. ZIP code
    3. Takes the bus to school (yes/no)
    4. Height in cm
    5. Number of siblings
  2. Area codes such as 770 are numbers. Is the area code of a phone number a quantitative variable?
    1. No: it is categorical, a label that only looks like a number
    2. Yes: its values are numbers
    3. Yes, a discrete quantitative variable
  3. A study measures the resting heart rate of 29 runners. What are the cases (observational units)?
    1. The 29 runners
    2. The heart rates
    3. The number 29

Discover

In a class, 12 of 20 girls and 9 of 15 boys play a sport. Who is more likely to play, a girl or a boy?

  1. Equally likely: 60% each
  2. A girl: 12 is more than 9

Key idea

A two-way table counts cases by two categorical variables.

Joint proportion: a cell ÷ grand total. Marginal proportion: a row or column total ÷ grand total. Conditional proportion: a cell ÷ its row (or column) total: the proportion within one group.

Worked example

Pets: 30 cat owners (18 adults, 12 children), 50 dog owners (35 adults, 15 children).

Joint: adults with cats =1880= \tfrac{18}{80}
Cell ÷ 80.
Marginal: adults =5380= \tfrac{53}{80}
Column total ÷ 80.
Conditional: adults among cat owners =1830=0.6= \tfrac{18}{30} = 0.6
Within cat owners.

Your turn, guided

Your turn: children among dog owners.

0.30.3
15 of 50.

Practice ★

  1. A survey asked 145 people whether they exercise weekly. Under 30: 22 yes, 43 no. Age 30 or over: 27 yes, 53 no. What proportion of the under-30s exercise weekly?
  2. A survey asked 121 people whether they exercise weekly. Under 30: 59 yes, 12 no. Age 30 or over: 23 yes, 27 no. What proportion of the under-30s exercise weekly?
  3. A survey asked 150 people whether they exercise weekly. Under 30: 41 yes, 40 no. Age 30 or over: 19 yes, 50 no. What proportion of the under-30s exercise weekly?

Watch out!

Practice ★★

  1. A survey asked 124 people whether they exercise weekly. Under 30: 21 yes, 50 no. Age 30 or over: 31 yes, 22 no. What proportion of all the people exercise weekly?
  1. A survey asked 140 people whether they exercise weekly. Under 30: 47 yes, 17 no. Age 30 or over: 37 yes, 39 no. What proportion of all the people are under 30 and exercise weekly?
  2. A survey asked 135 people whether they exercise weekly. Under 30: 44 yes, 25 no. Age 30 or over: 10 yes, 56 no. What proportion of all the people are under 30 and exercise weekly?

Exit ticket

Three questions to check you have got it.

  1. A survey asked 145 people whether they exercise weekly. Under 30: 22 yes, 43 no. Age 30 or over: 27 yes, 53 no. What proportion of the under-30s exercise weekly?
  2. A survey asked 121 people whether they exercise weekly. Under 30: 59 yes, 12 no. Age 30 or over: 23 yes, 27 no. What proportion of the under-30s exercise weekly?
  3. A survey asked 150 people whether they exercise weekly. Under 30: 41 yes, 40 no. Age 30 or over: 19 yes, 50 no. What proportion of the under-30s exercise weekly?

Summary card

Two-way tables

  • Joint: cell / total. Marginal: margin / total.
  • Conditional: cell / its group total.

Watch out: name the group you condition on.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. Of 150 students, 50 play an instrument, 40 play a club sport, and 12 do both. How many do neither?

2. Association between categorical variables

Warm-up

Three quick questions from earlier lessons.

  1. A survey asked 145 people whether they exercise weekly. Under 30: 22 yes, 43 no. Age 30 or over: 27 yes, 53 no. What proportion of the under-30s exercise weekly?
  2. A survey asked 124 people whether they exercise weekly. Under 30: 21 yes, 50 no. Age 30 or over: 31 yes, 22 no. What proportion of all the people exercise weekly?
  3. A survey asked 140 people whether they exercise weekly. Under 30: 47 yes, 17 no. Age 30 or over: 37 yes, 39 no. What proportion of all the people are under 30 and exercise weekly?

Discover

70% of students who sleep 8+ hours passed a test, and 40% of those who sleep less passed. Is there an association between sleep and passing?

  1. Yes, the pass rates differ
  2. We cannot tell without an experiment

Key idea

Compare conditional distributions.

Two categorical variables are associated if the conditional distribution of one differs across the groups of the other. If the conditional proportions are all equal, there is no association. Association in observational data does not show causation.

Worked example

Of 50 phone users, 30 sleep badly; of 50 non-users, 10 sleep badly.

60% vs 20% sleep badly
Within each group.
Association between phone use and sleep
A big difference.
It does not prove phones cause bad sleep
Observational data.

Your turn, guided

Your turn: the difference in proportions (users − non-users).

0.40.4
0.6 − 0.2.

Practice ★

  1. Of 40 drivers who texted, 30 had a near-miss; of 40 who did not text, 10 had one. Is there an association between texting and near-misses in these data?
    1. Yes: 75% of texters vs 25% of non-texters had a near-miss
    2. No: both groups have 40 drivers
    3. Yes, and it proves texting causes near-misses
  1. In a school, 50 of 125 girls and 100 of 250 boys walk to school. What does this suggest?
    1. No association: 40% of each group walks
    2. Boys are more likely to walk: 100 > 50
    3. Girls are more likely to walk

Watch out!

Practice ★★

  1. A survey asked 135 people whether they exercise weekly. Under 30: 44 yes, 40 no. Age 30 or over: 18 yes, 33 no. Find the proportion of under-30s who exercise minus the proportion of those 30 or over who exercise.
  2. A survey asked 163 people whether they exercise weekly. Under 30: 45 yes, 37 no. Age 30 or over: 48 yes, 33 no. Find the proportion of under-30s who exercise minus the proportion of those 30 or over who exercise.

Exit ticket

Three questions to check you have got it.

  1. A survey asked 135 people whether they exercise weekly. Under 30: 44 yes, 40 no. Age 30 or over: 18 yes, 33 no. Find the proportion of under-30s who exercise minus the proportion of those 30 or over who exercise.
  2. A survey asked 163 people whether they exercise weekly. Under 30: 45 yes, 37 no. Age 30 or over: 48 yes, 33 no. Find the proportion of under-30s who exercise minus the proportion of those 30 or over who exercise.
  3. A survey asked 112 people whether they exercise weekly. Under 30: 28 yes, 48 no. Age 30 or over: 13 yes, 23 no. Find the proportion of under-30s who exercise minus the proportion of those 30 or over who exercise.

Summary card

Association

  • Compare conditional proportions across groups.
  • Different: associated. Equal: not associated.

Watch out: association is not causation; counts are not rates.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. People were asked if they exercise weekly. Under 30: 37 yes, 28 no. Age 30 or over: 31 yes, 46 no. Of the people who said yes, what fraction are under 30?

3. Scatterplots and correlation

Warm-up

Three quick questions from earlier lessons.

  1. Test scores have mean 61 and standard deviation 10. Find the z-score of a score of 48.
  2. Heights are approximately normal with mean 76 and standard deviation 4. By the 68–95–99.7 rule, what percent are between 72 and 84?
  3. Mean 72, standard deviation 2. Which value has z-score −3-3?

Discover

Points lie close to a line going down from left to right. Which correlation fits?

  1. r=−0.95r = -0.95
  2. r=0.2r = 0.2

Key idea

Describe a scatterplot by direction, form, strength and unusual points. Correlation measures linear strength.

The correlation rr is between −1-1 and 11: its sign gives the direction, ∣r∣|r| the strength of a linear relationship. It has no units, does not change when units change, and is the same if xx and yy are swapped. It is sensitive to outliers and misleading for curved patterns.

Worked example

Hours of study vs test score: r=0.78r = 0.78.

Direction: positive (more study, higher scores)
Positive r.
Strength: fairly strong, linear
|r| = 0.78.
Scores out of 50 or 100: same rr
Unitless.

Your turn, guided

Your turn: r=0.6r = 0.6 for (x,y)(x, y). What is rr for (y,x)(y, x)?

0.60.6
Symmetric in x and y.

Practice ★

  1. A scatterplot of car age against resale value has r=−0.94r = -0.94. How would you describe the relationship?
    1. Strong, negative, linear: older cars tend to be worth less
    2. Weak, because r is negative
    3. Each extra year causes a fixed loss of value
  2. A scatterplot of car age against resale value has r=−0.92r = -0.92. How would you describe the relationship?
    1. Strong, negative, linear: older cars tend to be worth less
    2. Weak, because r is negative
    3. Each extra year causes a fixed loss of value
  1. Heights (cm) and weights (kg) of students have r=0.58r = 0.58. Heights are converted to inches. What is the new correlation?
    1. Still 0.580.58: changing units does not change rr
    2. Smaller, since inches are bigger units
    3. It cannot be known without the data

Watch out!

Practice ★★

  1. Which statements about the correlation rr of 50 pairs (x,y)(x, y) are true? Tick all that are.
    1. −1≤r≤1-1 \le r \le 1
    2. rr has no units
    3. Swapping xx and yy does not change rr
    4. rr measures the strength of any relationship, curved or straight
    5. r=0r = 0 means there is no relationship at all

Exit ticket

Three questions to check you have got it.

  1. A scatterplot of car age against resale value has r=−0.94r = -0.94. How would you describe the relationship?
    1. Strong, negative, linear: older cars tend to be worth less
    2. Weak, because r is negative
    3. Each extra year causes a fixed loss of value
  2. A scatterplot of car age against resale value has r=−0.92r = -0.92. How would you describe the relationship?
    1. Strong, negative, linear: older cars tend to be worth less
    2. Weak, because r is negative
    3. Each extra year causes a fixed loss of value
  3. A scatterplot of car age against resale value has r=−0.90r = -0.90. How would you describe the relationship?
    1. Strong, negative, linear: older cars tend to be worth less
    2. Weak, because r is negative
    3. Each extra year causes a fixed loss of value

Summary card

Correlation

  • −1≤r≤1-1 \le r \le 1; sign = direction, size = linear strength.
  • Unitless; symmetric in xx and yy.

Watch out: rr only sees straight lines.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. Every point of a data set lies exactly on the line y=50−9xy = 50 - 9x. What is the correlation rr?

4. The least-squares regression line

Warm-up

Three quick questions from earlier lessons.

  1. A scatterplot of car age against resale value has r=−0.94r = -0.94. How would you describe the relationship?
    1. Strong, negative, linear: older cars tend to be worth less
    2. Weak, because r is negative
    3. Each extra year causes a fixed loss of value
  2. Which statements about the correlation rr of 50 pairs (x,y)(x, y) are true? Tick all that are.
    1. −1≤r≤1-1 \le r \le 1
    2. rr has no units
    3. Swapping xx and yy does not change rr
    4. rr measures the strength of any relationship, curved or straight
    5. r=0r = 0 means there is no relationship at all
  3. Heights (cm) and weights (kg) of students have r=0.58r = 0.58. Heights are converted to inches. What is the new correlation?
    1. Still 0.580.58: changing units does not change rr
    2. Smaller, since inches are bigger units
    3. It cannot be known without the data

Discover

If r=1r = 1, every point is on a line. A student 1 SD above the mean in xx is then 1 SD above the mean in yy. If r=0.5r = 0.5, how many SDs above the mean in yy would you predict?

  1. 0.5 SD
  2. 1 SD

Key idea

The line that minimizes the sum of squared residuals.

Least-squares regression line y^=a+bx\hat{y} = a + bx with slope b=r sysxb = r\,\dfrac{s_y}{s_x} and intercept a=yˉ−bxˉa = \bar{y} - b\bar{x}; it always passes through (xˉ,yˉ)(\bar{x}, \bar{y}).

Worked example

r=0.6r = 0.6, xˉ=10\bar{x} = 10, sx=2s_x = 2, yˉ=50\bar{y} = 50, sy=8s_y = 8.

  1. Step 1The data.
  2. Step 2The point of means (x̄, ȳ) = (10, 50).
  3. Step 3The line ŷ = 26 + 2.4x passes through it.
  4. Step 4Residuals: no other line makes the sum of their squares smaller.

b=2.4b = 2.4
0.6 × 8/2.
a=26a = 26
50 − 2.4 × 10.
y^=26+2.4×12=54.8\hat{y} = 26 + 2.4 \times 12 = 54.8
At x = 12.

Your turn, guided

Your turn: r=0.5r = 0.5, sx=4s_x = 4, sy=6s_y = 6. The slope.

b=0.75b = 0.75
0.5 × 6/4.

Practice ★

  1. r=0.6r = 0.6, sx=15s_x = 15 and sy=16s_y = 16. Find the slope of the least-squares regression line of yy on xx.
  2. r=0.5r = 0.5, sx=11s_x = 11 and sy=17s_y = 17. Find the slope of the least-squares regression line of yy on xx.
  1. The regression line is y^=29+2x\hat{y} = 29 + 2x. Predict yy for x=3x = 3.

Watch out!

Practice ★★

  1. The least-squares line has slope 33, and xˉ=20\bar{x} = 20, yˉ=12\bar{y} = 12. Find its intercept.
  2. The least-squares line has slope 44, and xˉ=97\bar{x} = 97, yˉ=157\bar{y} = 157. Find its intercept.

Exit ticket

Three questions to check you have got it.

  1. r=0.6r = 0.6, sx=15s_x = 15 and sy=16s_y = 16. Find the slope of the least-squares regression line of yy on xx.
  2. r=0.5r = 0.5, sx=11s_x = 11 and sy=17s_y = 17. Find the slope of the least-squares regression line of yy on xx.
  3. r=0.9r = 0.9, sx=19s_x = 19 and sy=12s_y = 12. Find the slope of the least-squares regression line of yy on xx.

Summary card

Least-squares line

b=r sy/sxb = r\,s_y/s_x, a=yˉ−bxˉa = \bar{y} - b\bar{x}, through (xˉ,yˉ)(\bar{x}, \bar{y}).

Watch out: sys_y on top.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. r=0.9r = 0.9, sx=20s_x = 20 and sy=19s_y = 19. Every yy-value is doubled. What is the new slope of the least-squares line?

5. Residuals and residual plots

Warm-up

Three quick questions from earlier lessons.

  1. r=0.6r = 0.6, sx=15s_x = 15 and sy=16s_y = 16. Find the slope of the least-squares regression line of yy on xx.
  2. The least-squares line has slope 33, and xˉ=20\bar{x} = 20, yˉ=12\bar{y} = 12. Find its intercept.
  3. The regression line is y^=29+2x\hat{y} = 29 + 2x. Predict yy for x=3x = 3.

Discover

The model predicts 70 for a student who actually scored 76. Is the student above or below the line, and by how much?

  1. 6 above
  2. 6 below

Key idea

Residuals show what the line misses.

Residual = observed −- predicted =y−y^= y - \hat{y}. Positive: point above the line. A residual plot (residuals against xx) should look like random scatter around 0; a curve or a fan shape means a linear model is not appropriate.

Worked example

y^=12+3x\hat{y} = 12 + 3x; the data point is (5,30)(5, 30).

y^=27\hat{y} = 27
Predicted.
Residual =30−27=3= 30 - 27 = 3
Observed − predicted.
The point is 3 above the line
Positive.

Your turn, guided

Your turn: same line, point (4,20)(4, 20).

Residual −4-4
20 − 24.

Practice ★

  1. y^=30+3x\hat{y} = 30 + 3x. A data point is (7,46)(7, 46). Find its residual.
  2. y^=36+4x\hat{y} = 36 + 4x. A data point is (3,41)(3, 41). Find its residual.
  3. y^=32+6x\hat{y} = 32 + 6x. A data point is (7,70)(7, 70). Find its residual.

Watch out!

Practice ★★

  1. The residual plot of a linear model fitted to 56 points shows a clear U-shaped curve. What does that tell you?
    1. A straight line is not a good model: the relationship is curved
    2. The model is good, because residuals are both positive and negative
    3. There is one outlier
  1. A point has residual +8+8. Where is it relative to the regression line?
    1. 8 units above the line: the model underpredicts it
    2. 8 units below the line
    3. On the line

Exit ticket

Three questions to check you have got it.

  1. y^=30+3x\hat{y} = 30 + 3x. A data point is (7,46)(7, 46). Find its residual.
  2. y^=36+4x\hat{y} = 36 + 4x. A data point is (3,41)(3, 41). Find its residual.
  3. y^=32+6x\hat{y} = 32 + 6x. A data point is (7,70)(7, 70). Find its residual.

Summary card

Residuals

  • e=y−y^e = y - \hat{y}; positive is above the line.
  • Residual plot: no pattern means a linear model is appropriate.

Watch out: observed minus predicted, in that order.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. A least-squares line is fitted to three points. Two residuals are 3 and −7-7. What is the third residual?

6. Interpreting a regression model: slope, r², extrapolation

Warm-up

Three quick questions from earlier lessons.

  1. r=0.6r = 0.6, sx=15s_x = 15 and sy=16s_y = 16. Find the slope of the least-squares regression line of yy on xx.
  2. The least-squares line has slope 33, and xˉ=20\bar{x} = 20, yˉ=12\bar{y} = 12. Find its intercept.
  3. The regression line is y^=29+2x\hat{y} = 29 + 2x. Predict yy for x=3x = 3.

Discover

price^=2+0.5 weight\widehat{\text{price}} = 2 + 0.5\,\text{weight} for bags of rice (dollars, kg). What does 0.5 mean?

  1. Each extra kg adds 0.5 dollars to the predicted price
  2. Every bag costs exactly 0.5 dollars per kg

Key idea

Interpret in context, and know the limits.

Slope: the predicted change in yy for each one-unit increase in xx. Intercept: the predicted yy when x=0x = 0 (meaningful only if x=0x = 0 makes sense). r2r^2: the fraction of the variation in yy explained by the linear model. Predicting outside the range of the data is extrapolation and is unreliable.

Worked example

r=0.9r = 0.9 for study hours and scores.

r2=0.81r^2 = 0.81
Square r.
81% of the variation in scores is explained by the linear model on hours
Interpretation.
19% is not explained
The rest is other factors and chance.

Your turn, guided

Your turn: r=−0.7r = -0.7. Find r2r^2.

0.490.49
(−0.7)².

Practice ★

  1. A regression has r=0.2r = 0.2. What fraction of the variation in yy is explained by the linear model (r2r^2)?
  2. A regression has r=0.8r = 0.8. What fraction of the variation in yy is explained by the linear model (r2r^2)?
  1. For trees, height^=3+8 age\widehat{\text{height}} = 3 + 8\,\text{age} (feet, years). What does 88 mean?
    1. For each extra year of age, the predicted height increases by 8 feet
    2. Every tree grows exactly 8 feet each year
    3. A tree of age 0 is 8 feet tall

Watch out!

Practice ★★

  1. A model for children aged 2 to 6 predicts height from age. Is it reasonable to use it for a 40-year-old?
    1. No: 40 is far outside the ages used to fit the model (extrapolation)
    2. Yes, if r is close to 1
    3. Yes: a line can be extended forever

Exit ticket

Three questions to check you have got it.

  1. A regression has r=0.2r = 0.2. What fraction of the variation in yy is explained by the linear model (r2r^2)?
  2. A regression has r=0.8r = 0.8. What fraction of the variation in yy is explained by the linear model (r2r^2)?
  3. A regression has r=0.4r = 0.4. What fraction of the variation in yy is explained by the linear model (r2r^2)?

Summary card

Interpreting regression

  • Slope: predicted change per unit of xx.
  • r2r^2: share of variation explained.
  • No extrapolation.

Watch out: r2r^2, not rr, is the explained fraction.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. A regression line has a negative slope and r2=64100r^2 = \dfrac{ 64 }{100}. What is rr?

7. Outliers, leverage and influential points

Warm-up

Three quick questions from earlier lessons.

  1. r=0.6r = 0.6, xˉ=13\bar{x} = 13, sx=5s_x = 5, yˉ=54\bar{y} = 54, sy=10s_y = 10.
    1. The slope bb:   ?\;?
    2. The intercept aa:   ?\;?
    3. A point (13,57)(13, 57) is in the data. Its residual:   ?\;?
    4. The fraction of the variation in yy explained by the line (r2r^2):
  2. y^=30+3x\hat{y} = 30 + 3x. A data point is (7,46)(7, 46). Find its residual.
  3. The residual plot of a linear model fitted to 56 points shows a clear U-shaped curve. What does that tell you?
    1. A straight line is not a good model: the relationship is curved
    2. The model is good, because residuals are both positive and negative
    3. There is one outlier

Discover

A scatterplot of 20 points has a line through them. One more point is added far to the right, well below the trend. What happens to the line?

  1. It tilts towards the new point
  2. Nothing: one point cannot change a line fitted to 20

Key idea

Some unusual points matter more than others.

An outlier in regression has a large residual. A high-leverage point has an xx-value far from xˉ\bar{x}. An influential point changes the slope (or intercept) a lot when removed; high-leverage points are often influential.

Worked example

Three unusual points.

Outlier in yy: lowers rr, barely moves the slope
x near x̄, large residual.
High leverage, not influential: it strengthens rr
x far away, on the trend.
High leverage and influential: tilts the line
x far away, off the trend.

Your turn, guided

Your turn: removing a point changes the slope from 2.1 to 0.4. By how much did the slope change?

1.71.7
2.1 − 0.4: influential.

Practice ★

  1. In a scatterplot, the xx-values lie between 10 and 51, except one point at x=153x = 153. What is that point called?
    1. A high-leverage point: its x is far from the others
    2. A point with a large residual
    3. The median point
  1. Removing one of 52 points changes the regression slope a lot. What is that point?
    1. An influential point
    2. Any outlier: all outliers change the slope a lot
    3. A typical point near the center of the data

Watch out!

Practice ★★

  1. Among 46 points, one has xx close to xˉ\bar{x} but a very large residual. Removing it mostly changes what?
    1. The correlation (it rises); the slope barely changes
    2. The slope a lot
    3. Nothing at all
  2. Among 77 points, one has xx close to xˉ\bar{x} but a very large residual. Removing it mostly changes what?
    1. The correlation (it rises); the slope barely changes
    2. The slope a lot
    3. Nothing at all

Exit ticket

Three questions to check you have got it.

  1. In a scatterplot, the xx-values lie between 10 and 51, except one point at x=153x = 153. What is that point called?
    1. A high-leverage point: its x is far from the others
    2. A point with a large residual
    3. The median point
  2. In a scatterplot, the xx-values lie between 10 and 36, except one point at x=108x = 108. What is that point called?
    1. A high-leverage point: its x is far from the others
    2. A point with a large residual
    3. The median point
  3. In a scatterplot, the xx-values lie between 10 and 30, except one point at x=90x = 90. What is that point called?
    1. A high-leverage point: its x is far from the others
    2. A point with a large residual
    3. The median point

Summary card

Unusual points

  • Outlier: large residual. Leverage: extreme xx.
  • Influential: removing it changes the line.

Watch out: leverage, not residual size, drives influence.

One more step is optional: a challenge. Try it, or finish the lesson now.

Challenge ★★★

  1. Removing one influential point changes the correlation from r=0.4r = 0.4 to r=0.7r = 0.7. By how much does r2r^2 increase?

8. The least-squares line

Discover

A line fits some points, but none lies exactly on it. Which line is 'best'?

  1. The one that makes the sum of the squared vertical misses as small as possible
  2. Any line through two of the points

Key idea

Least squares.

The least-squares line y=mx+by = mx + b makes ∑(yi−mxi−b)2\sum (y_i - mx_i - b)^2 as small as possible. It always passes through the mean point (xˉ,yˉ)(\bar{x}, \bar{y}).

Worked example

Data: (1,2)(1, 2), (2,3)(2, 3), (3,7)(3, 7). The least-squares slope is 2.5. Find bb.

xˉ=2\bar{x} = 2, yˉ=4\bar{y} = 4
Mean point.
4=2.5×2+b4 = 2.5 \times 2 + b, so b=−1b = -1
The line passes through it.

Your turn, guided

Your turn: the line has slope 3 and xˉ=4\bar{x} = 4, yˉ=20\bar{y} = 20. Find bb.

b=8b = 8
20=3×4+b20 = 3 \times 4 + b.

Practice ★

  1. A least-squares line has slope 7 and the mean point is (7,57)(7, 57). Find its intercept bb.
  2. A least-squares line has slope 8 and the mean point is (8,25)(8, 25). Find its intercept bb.
  3. A least-squares line has slope 8 and the mean point is (5,43)(5, 43). Find its intercept bb.

Practice ★★

  1. Three points have residuals 11, −1-1 and 00 for the line y=2x+1y = 2x + 1. Find the sum of the squared residuals.
  2. Three points have residuals 11, −1-1 and 00 for the line y=2x+9y = 2x + 9. Find the sum of the squared residuals.

Exit ticket

Two questions to check.

  1. A least-squares line has slope 7 and the mean point is (7,57)(7, 57). Find its intercept bb.
  2. A least-squares line has slope 8 and the mean point is (8,25)(8, 25). Find its intercept bb.

Summary card

Least squares

  • Minimize the sum of squared residuals.
  • The line passes through (xˉ,yˉ)(\bar{x}, \bar{y}).

Every spreadsheet's 'trend line' is a least-squares line.

9. History of regression: Legendre, Gauss and Galton

Discover

Around 1800, astronomers had more observations of a comet than they needed, and the observations did not quite agree. How could they combine them?

  1. Choose the orbit that makes the squared errors as small as possible
  2. Keep only the best observation

Key idea

From comets to heights of children.

1800182018401860188019001806Legendre1806Legendre
  1. Step 11800182018401860188019001806Legendre1806LegendreAdrien-Marie Legendre 1752–1833Unknown artist. Public domain1806: Legendre's book on comet orbits gives the least-squares method in an appendix.
  2. Step 21800182018401860188019001806Legendre1809Gauss1806Legendre1809GaussCarl Friedrich Gauss 1777–1855Gottlieb Biermann / After Christian Albrecht Jensen, 1840. Public domain1809: Gauss publishes his own version of least squares.
  3. Step 31800182018401860188019001806Legendre1809Gaussc. 1886Galton1806Legendre1809Gaussc. 1886GaltonFrancis Galton 1822–1911Eveleen Myers (née Tennant), 1890s. Public domainGalton (born 1822) compares heights of parents and children and names regression.

Legendre gave the least-squares method in an appendix to his 1806 book on comet orbits; Gauss published his version in 1809. Later Francis Galton (born 1822) studied how tall children are compared with their parents: very tall parents tend to have children closer to the average. He called this regression, and drew the regression line.

Worked example

Galton's idea with numbers: a regression slope of 0.5 means that 10 cm above average for the parents predicts…

0.5×100.5 \times 10
Multiply by the slope.
5 cm above average for the child
Closer to the average.

Practice ★

  1. In which book did Legendre first print the method of least squares, in 1806?
    1. A book on the orbits of comets
    2. A book on heights of children
    3. A book on games of chance
  1. Galton's regression: the slope is 0.7. Parents are 17 cm above average. How many cm above average is the predicted child?
  2. Galton's regression: the slope is 0.5. Parents are 12 cm above average. How many cm above average is the predicted child?
  1. Galton's line passes through the mean point (170 cm, 170 cm) with slope 0.7. Predict the height (cm) of the child of parents 19 cm above the mean, that is 189 cm.

Summary card

Regression today

  • Economists predict prices and sales with regression lines.
  • Doctors relate dose and effect, or age and blood pressure.
  • Machine learning starts with least squares.

Sources: MacTutor biographies of Legendre, Gauss and Galton (see "Sources and credits").