The Line of Best Fit
A regression line summarizes a scatter with one equation you can predict from, but only within the range the data actually covers. · 14 min
The correlation number rated how tightly a cloud follows a line, but it never drew the line. This folio does. A regression line is one straight equation laid through a scatter so you can do something correlation cannot: put in a value of the explanatory variable and read out a prediction for the response. It comes with one firm rule about where that prediction can be trusted.
Guess before you learn
A line fitted to toddlers aged 1 to 4 says height is about 75 + 8 × (age in years) centimeters. Use it to predict the height of a 25-year-old.
The line is honest only near the ages it learned from. Reaching out to 25 is extrapolation — the one move this folio warns against. Everything else here is about predicting safely, inside the data.
A regression line is written ŷ = a + bx. The hat on ŷ marks it as a predicted value, not a measured one. The slope b is the heart of it: it says how much the predicted response changes for each one-unit increase in x. The intercept a is the predicted response when x is zero — sometimes meaningful, often just where the line crosses the axis.
9–12
3–5
A line of best fit is one straight line drawn to pass as near the dots as possible. Slide your finger along the line to any point and read off a fair guess — as long as you stay over the cloud of dots, not past its edges.
6–8
The regression line is written ŷ = a + bx, where ŷ is the predicted response. The slope b tells how much ŷ rises for each extra unit of x; the intercept a is ŷ when x is 0. To predict, put in an x and compute ŷ. But only for x-values inside the range of your data — past that, the line is only guessing.
9–12
The least-squares regression line is the one line that makes the sum of squared residuals — the vertical gaps between each point and the line — as small as possible. It always passes through the point of averages (x̄, ȳ), and its slope is b = r·(s_y / s_x): the correlation scaled by the ratio of the spreads. Interpret b as the predicted change in y per unit of x, and a as the predicted y at x = 0. Predicting outside the observed range of x is extrapolation, and nothing in the data justifies it.
K–2
Dots are scattered on paper. Lay a straight stick so it sits as close to all the dots as it can. Now the stick guesses a dot you did not draw — but only near the other dots.
Undergrad
Ordinary least squares chooses (a, b) to minimize the sum of (yᵢ − a − bxᵢ)². Setting the partial derivatives to zero gives the normal equations, solved by b = Σ(xᵢ − x̄)(yᵢ − ȳ) / Σ(xᵢ − x̄)² and a = ȳ − b·x̄, so the fit passes through (x̄, ȳ). The coefficient of determination r² is the share of variance in y explained by the fit. Residual plots, not r² alone, diagnose whether a linear model is appropriate; extrapolation fails precisely because the fitted form is unconstrained outside the support of x.
Postgrad
In matrix form y = Xβ + ε with E[ε] = 0 and Var(ε) = σ²I, the OLS estimator β = (XᵀX)⁻¹Xᵀy is, by Gauss–Markov, the best linear unbiased estimator of β. The fitted line estimates the conditional mean E[Y | X = x], valid as an interpolant over the covariate support; extrapolation is prediction at high-leverage points outside that support, where neither the unbiasedness of the functional form nor the variance approximation is warranted. Regression summarizes association; a causal reading demands assumptions the least-squares algebra never supplies.
regression line
The straight line ŷ = a + bx fit through a scatter to predict the response from the explanatory variable.
extrapolation
Using a regression line to predict outside the range of x-values the data covers — where its accuracy is unsupported.
Why is this true?
Why does the regression line always pass through the point of averages (x̄, ȳ)?
Because the least-squares fit balances the points. Put x = x̄ into ŷ = a + bx with intercept a = ȳ − b·x̄, and the b·x̄ terms cancel, returning exactly ȳ. The average input predicts the average output.
Predict a quiz score from ŷ = 30 + 5x — the steps fade as you master them
ŷ = 30 + 5(4)
ŷ = 30 + 20
ŷ = 50
1 to 5 covers 4 — the prediction is safe
The slope is not pulled from nowhere. It equals the correlation times the ratio of the spreads: b = r·(s_y / s_x). A stronger correlation, or a response that varies more per unit of x, makes a steeper line. Once you have the equation, prediction is arithmetic — but the gap between each real point and the line, called the residual, is what the line could not explain.
You can now fit a line, read its slope as a rate, predict from it, and — most importantly — refuse to predict where the data cannot vouch for you. One caution remains, large enough for its own folio: a line that predicts well does not prove that x causes y. That is where you head next.
Practice — new ink and old, interleaved
1.Using ŷ = 12 + 2x, predict y for x = 10.
2.In an otherwise tight upward cloud, one point sits far below all the others. What is it called?
3.The regression slope is b = r·(s_y / s_x). If r = 0, what is the slope?
4.A survey answer runs from strongly disagree to strongly agree — an ordinal variable. Which measure of center is defensible?
5.The data average to x̄ = 6 and ȳ = 40. Because a regression line passes through the point of averages, what is ŷ when x = 6?
6.Before fitting a line, why plot the scatter first?
7.The intercept of ŷ = 30 + 5x is 30. With x measured as hours studied, what does it mean?
8.A scatterplot of hours studied (across) and test score (up) has points rising from lower left to upper right. What is its direction?
9.Without looking back: what does each point on a scatterplot represent?
One subject measured on two variables — its position across is one variable, its position up is the other.
How close were you? Grade yourself honestly — it sets your review date.