Linear regression & correlation: practice problems with step-by-step solutions

Free Linear regression & correlation practice problems with diagrams, hints and full worked solutions. New values every time, at three levels: slope and intercept, correlation coefficient, prediction.

Free practice problemEasy
xy242352445555660760xy2465060

The table gives 6 paired observations of x = hours studied and y = exam score. Use the least-squares line ŷ = a + b·x. Find the intercept a.

n = 6
Next problem →
Practise Linear regression & correlation for freePractise Linear regression & correlation now5 free problems every day. No card needed.

What you practise

  1. Easyslope and intercept
  2. Mediumcorrelation coefficient
  3. Hardprediction

Key formulas

Sums of squares
Sxx = Σ(x − x̄)² = Σx² − (Σx)²/n, Sxy = Σxy − ΣxΣy/n
Slope
b = Sxy/Sxx
Intercept
a = ȳ − b·x̄
Correlation
r = Sxy/√(SxxSyy)
Residual
e = y − ŷ

Learn Linear regression & correlation from scratch

Before this: Descriptive statistics

1What regression is

You measure two quantities for each case, for example x = hours studied and y = exam score. Each pair is one point in a scatter plot.

xy123456246810

If the points lie roughly along a straight line, you can describe them with a line and use it to predict y from x. This is linear regression.

ŷ = a + b · x
  • ŷ (“y hat”) is the predicted value of y.
  • b is the slope: how much ŷ changes when x increases by 1.
  • a is the intercept: the value of ŷ at x = 0.
Self-check A regression line is ŷ = 40 + 5x, with x in hours studied and ŷ the predicted score. What does the 5 mean?
  1. A student who does not study scores 5
  2. Each extra hour adds about 5 points
  3. The line passes through (5, 40)
  4. The score is 5 times the hours

Answer: Each extra hour adds about 5 points. The slope b is the change in ŷ per unit of x. The predicted score at x = 0 is the intercept a = 40, not 5.

2The least-squares line

No line passes through every point. The vertical distance from a point to the line is its residual, e = y − ŷ.

The least-squares line is the line with the smallest possible sum of squared residuals. Its slope and intercept follow from sums of deviations from the means:

Sxx = Σ(x − x̄)², Sxy = Σ(x − x̄)(y − ȳ)
b = Sxy / Sxx, a = ȳ − b · x̄

The line always passes through the point of means (x̄, ȳ). That is why a = ȳ − b · x̄.

If only the sums are given, use the shortcuts Sxx = Σx² − (Σx)²/n and Sxy = Σxy − ΣxΣy/n.

Self-check For a data set x̄ = 4, ȳ = 10 and b = 2. What is the intercept a?
  1. 10
  2. 18
  3. 2
  4. −2

Answer: 2. a = ȳ − b · x̄ = 10 − 2 · 4 = 2. Adding instead of subtracting gives 18, and 10 is ȳ itself, which forgets the slope.

3The correlation coefficient

Pearson’s correlation coefficient r measures how close the points lie to a straight line. It is always between −1 and 1.

r = Sxy / √(Sxx · Syy), Syy = Σ(y − ȳ)²
  • r close to 1: a strong rising trend.
  • r close to −1: a strong falling trend, such as car price against car age.
  • r close to 0: no straight-line relationship.

r has the same sign as the slope b. Its square r² is the fraction of the variation in y that the line explains.

A strong correlation does not prove that x causes y. Both may depend on a third quantity.

Self-check For car age x and price y, r = −0.95. What does that tell you?
  1. A strong falling straight-line relationship
  2. A weak relationship
  3. Older cars cost more
  4. The calculation is wrong, because r is negative

Answer: A strong falling straight-line relationship. r near −1 means the points lie close to a falling line: price drops as age rises. A negative r is perfectly valid; only values outside −1 to 1 are impossible.

4The method step by step

  1. Compute the means x̄ and ȳ.
  2. Make a table with the columns x − x̄, y − ȳ, their product and (x − x̄)². Add (y − ȳ)² if r is asked.
  3. Add the columns to get Sxy, Sxx and Syy.
  4. Slope b = Sxy/Sxx, then intercept a = ȳ − b · x̄.
  5. Correlation r = Sxy/√(SxxSyy).
  6. Prediction: put x into ŷ = a + bx. Residual: e = y − ŷ.

5A worked example

Follow every step. Try to predict the next one before you open it.

Worked example

Five pairs are measured: x = 1, 2, 3, 4, 5 and y = 3, 5, 6, 8, 8. Find the least-squares line ŷ = a + b·x and predict ŷ for x = 6.

1

The means

x̄ = 15 / 5 = 3, ȳ = 30 / 5 = 6
2

Deviations

x − x̄: −2, −1, 0, 1, 2.

y − ȳ: −3, −1, 0, 2, 2.

3

The sums

Multiply the pairs: 6, 1, 0, 2, 4. Square the x-deviations: 4, 1, 0, 1, 4.

Sxy = 6 + 1 + 0 + 2 + 4 = 13, Sxx = 4 + 1 + 0 + 1 + 4 = 10
4

Slope and intercept

b = 13 / 10 = 1.3
a = 6 − 1.3 · 3 = 6 − 3.9 = 2.1
xy123456246810ŷ = 2.1 + 1.3x

The dashed lines are the residuals. They are small, so the line fits well.

5

Predict

ŷ = 2.1 + 1.3 · 6 = 2.1 + 7.8 = 9.9

Check: the line passes through the point of means, 2.1 + 1.3 · 3 = 6 = ȳ.

Answer

ŷ = 2.1 + 1.3x, and the prediction for x = 6 is ŷ = 9.9.

6Common mistakes

  • Computing the residual as ŷ − y instead of y − ŷ.
  • Dividing by Syy instead of Sxx for the slope.
  • Mixing up a and b: the slope multiplies x.
  • Forgetting that r has the same sign as the slope.
  • Predicting far outside the range of the data. The straight line may not hold there.
Self-check The line is ŷ = 2.1 + 1.3x. A new point (6, 8) is measured. What is its residual?
  1. 1.9
  2. 17.9
  3. 9.9
  4. −1.9

Answer: −1.9. ŷ = 2.1 + 1.3 · 6 = 9.9, so e = y − ŷ = 8 − 9.9 = −1.9. The point lies below the line. 1.9 computes ŷ − y, the wrong way round.

7How exam questions look

  • Easy: slope and intercept. Five pairs in a table; find b or a of the least-squares line.
  • Medium: the correlation coefficient r, from the data or from given sums.
  • Hard: prediction. Use the line to predict ŷ for a new x, or find the residual of a new observation.

A quick check: the sign of b and of r must match the direction of the scatter plot.

Learn it step by step in the appLearn it step by step in the app

Worked examples

Example 1Easy
xy2441262082210301235xy510102030

The table gives 6 paired observations of x = load in kN and y = extension in mm. Use the least-squares line ŷ = a + b·x. Find the intercept a.

n = 6

Find a

Show hints
  1. First find the slope b = Sxy/Sxx.
  2. x̄ = 7, ȳ = 20.5.
  3. The line passes through (x̄, ȳ), so a = ȳ − b·x̄.
Show the answer and the step-by-step solution

Least-squares regression: with x̄ and ȳ the means, Sxx = Σ(x − x̄)², Syy = Σ(y − ȳ)² and Sxy = Σ(x − x̄)(y − ȳ), the line ŷ = a + b·x has slope b = Sxy/Sxx and intercept a = ȳ − b·x̄, so it always passes through (x̄, ȳ). Pearson's correlation coefficient r = Sxy/√(Sxx·Syy) lies between −1 and 1; r² is the fraction of the variation in y explained by the line.

Means

x̄ = 42/6 = 7
ȳ = 123/6 = 20.5

Sums of squares and products

Sxx = Σ(x − x̄)² = 70
Sxy = Σ(x − x̄)(y − ȳ) = 211

Slope

b = 211/70 = 3.014

Intercept

a = 20.5 − 3.014·7 = −0.6

Answer

a = −0.6
Example 2Medium
xy1787199921100231242512727146xy2025100120140

The table gives 6 paired observations of x = temperature in °C and y = ice creams sold. Their sums are given. Find Pearson's correlation coefficient r from these sums.

n = 6Σx = 132Σy = 683Σx² = 2974Σy² = 80191Σxy = 15429

Find r

Show hints
  1. Sxy = Σxy − Σx·Σy/n.
  2. Sxx = Σx² − (Σx)²/n, Syy = Σy² − (Σy)²/n.
  3. r = Sxy/√(Sxx·Syy).
Show the answer and the step-by-step solution

Least-squares regression: with x̄ and ȳ the means, Sxx = Σ(x − x̄)², Syy = Σ(y − ȳ)² and Sxy = Σ(x − x̄)(y − ȳ), the line ŷ = a + b·x has slope b = Sxy/Sxx and intercept a = ȳ − b·x̄, so it always passes through (x̄, ȳ). Pearson's correlation coefficient r = Sxy/√(Sxx·Syy) lies between −1 and 1; r² is the fraction of the variation in y explained by the line.

Centred sums

Sxy = 15429 − 132·683/6 = 403
Sxx = 2974 − 132²/6 = 70
Syy = 80191 − 683²/6 = 2443

Correlation coefficient

r = 403/√(70·2443) = 0.9746

Answer

r = 0.9746
Example 3Hard
xy123221318412513xy241520x = 1.5

The table gives 5 paired observations of x = car age in years and y = price in k€. Use the least-squares line ŷ = a + b·x. Predict ŷ for x = 1.5.

n = 5x = 1.5

Find ŷ

Show hints
  1. First find the least-squares line: b = Sxy/Sxx and a = ȳ − b·x̄.
  2. x̄ = 3, ȳ = 17.4.
  3. ŷ = a + b·1.5.
Show the answer and the step-by-step solution

Least-squares regression: with x̄ and ȳ the means, Sxx = Σ(x − x̄)², Syy = Σ(y − ȳ)² and Sxy = Σ(x − x̄)(y − ȳ), the line ŷ = a + b·x has slope b = Sxy/Sxx and intercept a = ȳ − b·x̄, so it always passes through (x̄, ȳ). Pearson's correlation coefficient r = Sxy/√(Sxx·Syy) lies between −1 and 1; r² is the fraction of the variation in y explained by the line.

Means

x̄ = 15/5 = 3
ȳ = 87/5 = 17.4

Sums of squares and products

Sxx = Σ(x − x̄)² = 10
Sxy = Σ(x − x̄)(y − ȳ) = −29

Regression line

b = −29/10 = −2.9
a = 17.4 − (−2.9)·3 = 26.1
ŷ = 26.1 − 2.9·x

Prediction

ŷ = 26.1 + (−2.9)·1.5 = 21.75

Answer

ŷ = 21.75

Questions and answers

What is Linear regression & correlation?

You measure two quantities for each case, for example x = hours studied and y = exam score. Each pair is one point in a scatter plot.

If the points lie roughly along a straight line, you can describe them with a line and use it to predict y from x. This is linear regression.

How do I solve Linear regression & correlation problems step by step?

Least-squares regression: with x̄ and ȳ the means, Sxx = Σ(x − x̄)², Syy = Σ(y − ȳ)² and Sxy = Σ(x − x̄)(y − ȳ), the line ŷ = a + b·x has slope b = Sxy/Sxx and intercept a = ȳ − b·x̄, so it always passes through (x̄, ȳ). Pearson's correlation coefficient r = Sxy/√(Sxx·Syy) lies between −1 and 1; r² is the fraction of the variation in y explained by the line.

What are common mistakes in Linear regression & correlation?
  • The intercept is a = ȳ − b·x̄: the line passes through (x̄, ȳ).
  • ȳ is the height of the line at x = x̄, not at x = 0. Use a = ȳ − b·x̄.
  • Use the same divisor (n or n − 1) in the covariance and in both standard deviations. With Sxy, Sxx and Syy the divisors cancel.
What do the Linear regression & correlation exercises cover?
  • Easy: slope and intercept
  • Medium: correlation coefficient
  • Hard: prediction

Every problem has new values, so you learn the method instead of an answer. Stuck? Open a small hint first, and the full explanation only when you need it.

How practising works

  • Every problem has new values, so you learn the method instead of an answer.
  • Stuck? Open a small hint first, and the full explanation only when you need it.
  • Your progress, mistakes and saved problems are kept, so you can review before the exam.

Topics before and after this one

Learn first

Descriptive statistics

Calculators

Linear regression calculatorCorrelation coefficient calculator (Pearson and Spearman)

Courses with this topic

Research Methods and Statistics for PsychologyIntroductory StatisticsStatistics 2: Statistical InferenceAP Statistics practice problemsCFA Level I quantitative methods and fixed income practice

More topics in Mathematics & statistics

LimitsDerivativesOptimizationIntegralsSequences & seriesMultivariable calculusDifferential equationsComplex numbersNumerical methodsMatrices & determinantsVectors: dot & cross productPropositional logic & proofsSets, relations & functionsGraph theoryNumber theory & modular arithmeticProbabilityDescriptive statisticsNormal distributionConfidence intervalsHypothesis tests & confidence intervalsChi-square tests

Printable worksheet

Linear regression & correlation worksheet with answers
Practise Linear regression & correlation for freePractise Linear regression & correlation now5 free problems every day. No card needed.