Volatility Models, Penalised Regression and Machine Learning for Analysts

Learning objectives

By the end you can:

1. Fit a linear trend and a log-linear trend to the same series by hand, back-transform the log-linear forecast correctly, state what each model assumes about growth, and read the Durbin-Watson statistic on both sets of residuals against the published bounds.
2. Explain why a failed Durbin-Watson on a trend model is a specification verdict rather than a nuisance, and state what a correctly specified autoregressive model implies about residual autocorrelation at every lag, with the standard error 1/√T that makes the test operational.
3. Apply the chain rule of forecasting to an estimated AR(1), iterate two and three periods ahead by hand, reconcile the chained answer against the closed form, and locate the mean-reverting level b₀/(1 − b₁).
4. Compare models on out-of-sample rather than in-sample criteria: run a split-sample exercise in which the better in-sample model loses, quantify the loss in root mean squared error, and read coefficient instability across estimation windows as the warning it is.
5. Run an ARCH(1) regression of squared residuals on their own lag, test the coefficient, say what a positive coefficient does to the parent regression's standard errors, and produce a one-step-ahead conditional variance and the value at risk it implies.
6. State GARCH(1,1) at concept depth: persistence as α + β, the long-run variance ω/(1 − α − β), the half-life of a variance shock, and why volatility clustering is a measured fact rather than a figure of speech.
7. Explain and compute penalised regression: why a penalty on coefficient size trades bias for variance, how ridge shrinks while LASSO shrinks and zeroes, what elastic net blends, and why λ must come from cross-validation.
8. Describe support vector machines, k-nearest neighbours, CART, bagging, boosting, random forests and feed-forward neural networks accurately enough to say what each assumes, what its complexity dial is, and how it fails.
9. Build an agglomerative hierarchical clustering from a distance matrix by hand, read a dendrogram at two cut heights, and demonstrate that the linkage rule and the cut height each change the answer.
10. Choose the evaluation metric that fits the decision: compute accuracy, precision, recall, F1 and the area under the ROC curve from a confusion matrix, and show numerically why an accuracy-maximising threshold can be the expensive one.

The duality, stated once. Objectives 1 to 9 are what the gate rewards, and they are gated because they are what lets you audit an output. Objective 10 is where the money is. A tool will fit a GARCH, tune a LASSO and draw a dendrogram in four seconds; none of those outputs carries any visible sign of whether the window was long enough, whether standardisation happened inside the cross-validation folds, or whether the cut height was chosen before or after somebody saw the answer they wanted.

Prerequisites & connections

Builds on. QM1.02 supplied the regression machinery every technique here reuses: ordinary least squares, the standard error, the t-statistic, and the four diagnostics. Its Durbin-Watson treatment is the direct parent of the trend-model work below, and its heteroskedasticity section is the reason an ARCH coefficient matters at all. QM1.03 supplied autoregressive and moving-average processes, stationarity, the unit root and the machine-learning survey; the trend and AR work here completes what it started, and its supervised/unsupervised framing is assumed rather than repeated. QM1.04 supplied the dispersion and shape statistics, the sampling biases, and the multiple-testing discipline that the mini-project's honest model count feeds. From the analyst core, M2.03 and M4.01 supply the margin and returns vocabulary the worked examples borrow, and M9.01 supplies the position-sizing frame in which a conditional value at risk is actually used.

Feeds forward. DA2.01 builds supervised learning in code, with the tree and ensemble machinery described here turned into a graded scikit-learn pipeline. DA2.02 builds unsupervised learning, feature engineering and leak-free validation, including k-means and principal components. DA2.03 builds deep learning, natural language and the production forecasting pipeline, and it is where a network's forward pass and backpropagation get computed rather than described. QD1.02 constructs empirical factors and evaluates them, and it consumes the penalised-regression discipline set out here. QD1.03's backtesting harness enforces the chronological-split rule as an automated guard. QD2.02 turns a conditional variance forecast into a production risk engine with a value-at-risk backtest.