Cointegration

Two independent random walks, regressed on each other, produce one of the most seductive traps in applied statistics: a spurious regression (high R², a "highly significant" slope with a tiny p-value) despite there being no real relationship whatsoever between the two series. This is not a subtle edge case; Granger and Newbold demonstrated it decisively (Granger & Newbold, 1974), and its classic tell is a Durbin-Watson statistic near zero (extreme positive residual autocorrelation, because the "residual" of a nonsense regression on two unit-root series is itself just another near-random-walk, not the well-behaved white noise OLS assumes). Worked Example 5 regresses two entirely unrelated synthetic indices on each other and produces exactly this signature, a textbook-perfect trap, met once in a setting where you already know the truth.

What is cointegration, and how do you test for it?

The escape from this trap, when it exists, is cointegration: two (or more) series that are each individually non-stationary (I(1)) are said to be cointegrated if some linear combination of them is stationary (I(0)), although each wanders on its own, a specific weighted spread between them is anchored and mean-reverting. This is the formal statement of a genuine long-run equilibrium relationship: two PSU banks' share prices, driven by a shared banking-sector trend, might drift together even as each individually looks like a random walk, because competitive and regulatory forces keep the spread between them from wandering off forever. The standard detection procedure is the Engle-Granger two-step test (Engle & Granger, 1987): step 1, regress one series on the other by OLS to estimate the long-run relationship (the hedge ratio); step 2, test the residual for a unit root, a rejection means the pair is cointegrated and the residual is your tradeable, mean-reverting spread. Worked Example 4 runs this on two synthetic PSU-bank-style stocks, and along the way corrects a mistake worth its own warning: the residual from step 1 is not raw data, it is the leftover of an estimated regression, and that estimation shifts the test's null distribution. Using the plain Dickey-Fuller table on an Engle-Granger residual (an easy, common error) systematically overstates significance; the correct table is stricter (Engle & Granger's own tabulation, refined by MacKinnon, implemented as statsmodels.tsa.stattools.coint), and Worked Example 4 shows exactly how much the correction matters.

How do you run a cointegration test in Python?

Modern Data Analysis. Everything in this section has a coded, production-grade home. statsmodels.tsa.stattools.adfuller runs the ADF test exactly as described there (and is what generated every ADF statistic in this module); statsmodels.tsa.stattools.coint runs the correctly-adjusted Engle-Granger cointegration test in one call, so you never have to remember to swap critical-value tables by hand; pmdarima.auto_arima (or statsmodels's own order-search helpers) automates the AR/MA order search this section teaches you to do by eye. None of that automation is a substitute for reading the correlogram yourself first, a search procedure that never sees the shape of your ACF/PACF can happily hand you an order that fits noise. The full, hidden-tested, at-scale build, seasonal ARIMA, rolling re-estimation, a real forecasting evaluation harness: lives in DA1.05 (Statistics in Code) and DA2.03 (Deep Learning, NLP & Time-Series Forecasting). This module stops at the concept and the by-hand fit; that module grades the code.