The Analyst's Path

Multiple regression

Multiple regression explains one outcome with several predictors at once, so each coefficient is a partial effect: the change in y for a one-unit change in that predictor, holding the others constant. Adjusted R² and the overall F-test judge the model as a whole, and the partial F-test judges a subset of predictors. Analysts use it to separate one variable's effect from the others and to check that added predictors earn their place.

4 min readPart of Hypothesis Testing, Correlation & RegressionOpen in course mode

Advertisement

Add predictors (y = β₀ + β₁x₁ + β₂x₂ + … + βₖxₖ + u) and two things change. First, each coefficient becomes a partial effect: b₁ is the change in y for a one-unit change in x₁, holding every other included regressor constant. Read that qualifier every time; drop it and you have silently claimed something the regression never proved. Second, R² can only rise (or stay flat) as you add predictors, even useless ones, which is why adjusted R² exists, it imposes a penalty for each added predictor:

Adjusted R² = 1 − (1 − R²)(n − 1)/(n − k − 1)

where k is the number of predictors (not counting the intercept). Adjusted R² can fall when a genuinely useless variable is added, which is exactly the point: it is the honest scorecard, R² is the flattering one.

How does the overall F-test work?

The regression's overall F-test asks a single joint question, are all the slopes simultaneously zero?: using

F = [R² / k] / [(1 − R²) / (n − k − 1)]

Advertisement

against an F-distribution with (k, n − k − 1) degrees of freedom. A significant overall F with several individually insignificant t-stats is not a contradiction; it is frequently the signature of multicollinearity, covered in regression diagnostics. A joint test on a subset of coefficients, the partial F-test, generalizes the same idea to any group of predictors, and it is worth building rather than naming. Fit the model twice: once unrestricted, with every predictor in, and once restricted, with the q predictors you are testing removed. Each fit returns a sum of squared errors, SSE_U and SSE_R, and SSE_R can only be the larger of the two, because dropping predictors never improves fit. The question is whether it is larger by more than chance alone would produce.

F = [(SSE_R − SSE_U) / q] / [SSE_U / (n − k − 1)]

Here q is the number of restrictions (coefficients set to zero), k is the number of predictors in the unrestricted model, and the degrees of freedom are (q, n − k − 1). The numerator is the fit you gave up, per restriction; the denominator is the unrestricted model's own error variance, which is what "more than chance" is measured against. An equivalent form uses R² and saves carrying sums of squares around:

F = [(R²_U − R²_R) / q] / [(1 − R²_U) / (n − k − 1)]

Two special cases anchor the formula. Set q = k, so the restricted model is the intercept alone, and the partial F collapses into the overall F above: the overall F is a partial F testing every slope at once. Set q = 1 and something more useful appears. The partial F on a single restriction equals the square of that coefficient's own t-statistic, exactly, and the two return the same p-value. Run that identity as your first check whenever you compute a partial F by hand, because an F that misses t² on one restriction means an arithmetic slip rather than a discovery. Worked Example 8 works both cases on the credit-spread model.

One extension is worth knowing exists. The restriction does not have to be "set these coefficients to zero". Any linear restriction fits the same frame, including β₁ = 1 (test a beta against the market instead of against zero, as Worked Example 5 does) or β₂ = β₃ (two predictors forced to share a coefficient). Impose the restriction, refit, and compare the two SSEs the same way. CL-QM.1 grades a partial F as a two-point task and it grades the q = 1 case, so the t² identity is the whole of what that task needs.

Advertisement

Part of the course

This page is part of Hypothesis Testing, Correlation & Regression (QM1.02), in Phase 12: Finance Plus, AI and the quant-code track. The module's flashcards and its quiz are in the app.