Autoregressive model

The simplest way to give a series a memory is to let today depend directly on yesterday. An AR(1) process is

x_t = c + φ·x_{t-1} + ε_t,        ε_t ~ i.i.d.(0, σ²)

This is nothing more than the OLS regression QM1.02 already taught you, with one twist: the "independent variable" is the series' own lagged self. Stationarity requires |φ| < 1, if φ were 1 or more, shocks would never decay (that is exactly the random-walk case, revisited formally). When |φ| < 1, the process has a well-defined unconditional mean μ = c/(1-φ) and unconditional variance σ²/(1-φ²), and its theoretical ACF decays geometrically: ρ(k) = φᵏ. A useful, memorable derived quantity is the half-life of a shock, how many periods it takes a deviation from the mean to decay halfway back: given by ln(0.5)/ln(φ). A φ of 0.8 has a half-life of about 3.1 periods; a φ of 0.95 (very persistent) has a half-life of about 13.5 periods; a φ of 0.5 decays halfway in exactly one period. The AR(1)'s PACF cuts off sharply after lag 1, the direct one-period link is all there is; anything the second lag appears to explain is fully mediated through the first. That ACF-decays / PACF-cuts-off signature is the single most useful fact in this section, and Worked Example 1 fits exactly this process, coefficient and half-life included, to a synthetic Indian stock's monthly returns.

What is an AR(p) model?

An AR(p) generalizes the idea to p lags (x_t = c + φ₁x_{t-1} + φ₂x_{t-2} + … + φₚx_{t-p} + ε_t); its PACF cuts off after lag p and its ACF decays (possibly with oscillation, if some φ are negative) rather than cutting off. You will not be asked to hand-derive AR(p) beyond AR(1) or AR(2) in this module, the code build for higher-order and seasonal variants belongs to DA2.03.

Seasonality inside a fitted AR model

The decomposition strips seasonality out before anything is modelled. An AR model does the opposite: it is handed the raw series and asked to explain today from its own past. On quarterly or monthly data those two habits collide, because the seasonal cycle is itself a form of memory, sitting at lag 4 for quarters and lag 12 for months, and an AR(1) has nowhere to put it. The result is a fit that looks respectable and leaves its whole seasonal structure in the residuals.

One diagnostic catches it. After fitting, compute the autocorrelation of the residuals at every lag out to at least two full seasonal cycles, and compare each against the approximate two-standard-error band ±2/√T, where T is the number of residuals. The rule to hold is simple and unforgiving: a correctly specified AR model leaves no significant residual autocorrelation at any lag. Residuals are supposed to be the part with no pattern left in them, so a spike anywhere is the model telling you what it failed to absorb, and a spike at exactly the seasonal lag names the omission precisely.

Worked example: quarterly same-store sales growth

Work it once. Below are forty quarters of a synthetic retailer's same-store sales growth, in percent, laid out four quarters to a row.

Fit an AR(1) by OLS on the 39 usable observations and you get x_t = 3.8601 + 0.4722·x_{t-1}, with a standard error of 0.1449 on the slope, so t = 3.258 and R² = 0.2229. Nothing in that printout asks to be questioned. A persistence coefficient near 0.47 on quarterly growth is entirely plausible, the slope is comfortably significant, and an analyst in a hurry writes down a half-life of ln(0.5)/ln(0.4722) = 0.92 quarters and moves on.

Now look at the residuals. With T = 39 the band is 2/√39 = ±0.3203, and the residual autocorrelations run:

lag  1      2      3      4       5      6      7      8
   +0.002 -0.201 +0.107 +0.688* +0.033 -0.255 +0.060 +0.503*

Lag 4 sits at +0.688, more than twice the band, and lag 8 at +0.503 is the same structure showing up again one cycle later. That pattern, a spike at the seasonal lag and a decaying echo at its multiples, is the signature. The AR(1) has absorbed none of the annual cycle, so every fourth residual is still large and positive together.

Add the seasonal lag and refit as x_t = c + φ₁·x_{t-1} + φ₄·x_{t-4} + ε_t. On the 36 usable observations that gives c = 1.0369, φ₁ = 0.1267 (se 0.1174, t = 1.079) and φ₄ = 0.7555 (se 0.1225, t = 6.166), with R² rising from 0.2229 to 0.6368. Re-test the residuals against the new band of 2/√36 = ±0.3333: the largest absolute value across lags 1 to 8 is 0.322 at lag 7, and every one of them now sits inside the band. The specification passes the test it previously failed.

Two readings of that result are worth more than the fit itself. First, φ₁ collapses from 0.4722 to 0.1267 and stops being significant, which says the first lag in the original model was never really measuring one-quarter persistence at all; it was picking up a fraction of the annual cycle through the only channel it had. Any half-life computed from the AR(1) was describing something that does not exist. Second, φ₁ + φ₄ = 0.882 stays below 1, so the fitted process is still stationary, and that sum is worth checking every time you add a lag, because small samples over-state persistence and a fitted sum above 1 quietly makes your forecasts explode.

Two boundaries close this off. The ±2/√T band is an approximation that treats each lag on its own, so scanning eight lags and stopping at the first star is a multiple-testing problem; QM1.04 owns the corrections. And the systematic build, seasonal ARIMA with automated order selection and the SARIMA notation, stays in DA2.03. What lives here is the by-hand diagnosis: fit, look at the residual autocorrelations, and let the lag that fails tell you which term is missing.