Deep Learning, NLP & Time-Series Forecasting

Branch: Data Science & ML (Quant-Code galaxy) · Node DA2.03 · ~22 focused hours · CROWN node · Mastery gate ≥ 90%

There is a specific kind of disappointment waiting at the end of this module, and the whole point of the crown is that you should walk toward it deliberately rather than trip over it in front of an investment committee. It goes like this. You build a neural network. It trains. The loss curve comes down beautifully. On the training window it explains 31% of the variation in next-day returns, a number that, if it were real, would make you very rich. Then you run it forward through time, one honest day at a time, against the dumbest possible benchmark, tomorrow will look like today, and it loses. Not catastrophically. By about 1%. Which is worse, because 1% is the size of a result you can talk yourself into.

That experiment is in this module, run for you on both an Indian and a US price series, with every number reproducible from the shipped snapshots. The in-sample R² of 0.306 (US) and 0.322 (India) is real, in the sense that the arithmetic is correct. The out-of-sample loss to naive is also real. Learning to hold both of those facts at once, and to report the second one as loudly as the first, is the skill this crown certifies. Everything else here, the linear algebra and the chain rule and the sparse matrices, is machinery in service of that judgement.

So this node has three parts and one spine. Part I builds a neural network from nothing: forward pass, loss, and backpropagation derived by hand and implemented in NumPy, verified against a numerical gradient. Not a framework tutorial, a from-scratch build, because the analyst who has written backprop is permanently immune to treating a network as an oracle. Part II turns text into numbers and runs real classifiers over filing language, from tokenisation through TF-IDF to a scikit-learn pipeline, including the vocabulary leak that silently inflates almost every text-classification result an analyst will ever be shown. Part III forecasts a time series and then judges it honestly: walk-forward validation with look-ahead made structurally impossible, scale-free metrics, and a mandatory comparison against the naive benchmark that most published forecasts quietly omit.

The spine running through all three is a single sentence: a model that cannot beat a naive baseline has produced no information, and a report that does not show the baseline has produced no evidence.

Code pack: this node ships with DA2.03 (app/public/code-packs/DA2.03.json): ten graded exercises, DA2.03-e01 through DA2.03-e10. The gradients in e02 are checked against central differences rather than a stored answer; the softmax in e04 is fed logits of magnitude 1,000 so a naive exp returns nan and fails; the text pipeline in e07 reports its own vocabulary size so a train-plus-test fit is caught; and e10 runs a moving average against a naive benchmark on real snapshots and requires your code to report that it lost. The exercises are the assessment. This document is the argument that makes them passable.

A note on what is not here, and why. PyTorch and TensorFlow do not run in this app's browser Python runtime, and this is a feature rather than an apology. Graded deep-learning work here is from-scratch NumPy, which is the only version that teaches anything anyway; frameworks and transformer architectures are covered conceptually, with an external notebook pointer for when you want to run them at scale. You will finish this module able to read framework code fluently, because you will have written by hand the thing it abstracts.

Learning objectives

By the end you can:

1. State what a neural network is in mechanical terms (an alternating stack of affine maps and elementwise non-linearities, fitted by gradient descent) and say precisely what the hidden layer buys that a linear model cannot have, plus what it costs.
2. Implement the forward pass of a multi-layer perceptron in NumPy with correct shapes and bias broadcasting, keeping the intermediate values a backward pass will need.
3. Derive backpropagation by hand for that network under both mean squared error and softmax cross-entropy, and implement it so that every gradient's shape matches its parameter's shape.
4. Verify a gradient numerically with central differences, state the relative-error thresholds that mean right, wrong and look harder, and explain why the check is the only reason to trust a hand-derived backward pass.
5. Implement softmax and cross-entropy in a numerically stable way (the row-max shift and log-sum-exp) and explain exactly which computation overflows without them and why the fused gradient collapses to (P − onehot)/n.
6. Train a network end to end: He initialisation, a full-batch loop, a seeded and reproducible run, a loss curve, and a comparison against the mean baseline and an OLS line that says whether the non-linearity earned its keep.
7. Name what deep learning is and is not good for in finance (where the data volume, signal-to-noise ratio and non-stationarity of market data put a hard ceiling on it) and describe what a framework (PyTorch/TensorFlow) adds to what you built, in terms of autodiff, hardware and ecosystem rather than magic.
8. Turn a corpus of filings or news into a matrix: tokenisation, vocabulary, bag-of-words counts, and TF-IDF reproduced to nine decimal places against scikit-learn's own implementation.
9. Build and evaluate a text classifier with a chronological split, a majority-class baseline reported alongside every accuracy, and the vectorizer fitted on training documents only, and explain why fitting on train-plus-test is look-ahead rather than mere sloppiness.
10. Describe modern NLP conceptually (contextual embeddings, transformers, fine-tuning versus prompting, and finance-specific lexicons) and say what each buys over TF-IDF and what each costs in compute, latency and auditability.
11. Build a walk-forward evaluation harness in which look-ahead is structurally impossible, not merely discouraged, and explain why a random train/test split on a time series is a wrong test rather than a weak one.
12. Choose and compute honest forecast metrics (MAE, RMSE, MASE) say when each is the right loss, and state why MASE's denominator makes it comparable across a ₹800 share and a 4,500-point index.
13. Compare any forecast against the mandatory naive benchmark, decide whether a difference is evidence or noise using a paired test, and report a loss to naive as a result rather than as a failure.
14. (Productivity objective: R10 duality.) Use a coding copilot on deep-learning, NLP and forecasting code the way a working practitioner does, after you can derive it, and review its output for the failure modes specific to this material: silently wrong gradients that still train, hallucinated framework APIs, leaky pipelines, and evaluation code that scores the model against itself.

The duality, stated once (R10). The gated skill is objectives 1–13, and the gate is built to be hostile to shortcuts: the pack's hidden tests check gradients against numerical differentiation rather than stored values, feed inputs that only a numerically stable implementation survives, and, in DA2.03-e10, require your code to report a loss to the naive benchmark, which no amount of prompt engineering will guess. Objective 14 is the payoff you keep. The app never calls an AI at runtime: every AI-Augment callout below asks you to generate something in your own tool, paste it back, and let the local hidden tests judge it. The machine drafts; the tests judge; you are the one who has to know why it failed.

Prerequisites & connections

Builds on. DA1.01 for NumPy, broadcasting, axes, @, and the habit of thinking in shapes rather than loops; if (n, d) @ (d, h) -> (n, h) is not automatic, one hour there saves five here. DA1.02 and DA1.03 for getting data into a tidy, correctly-indexed frame in the first place. DA1.05 for the statistical vocabulary the evaluation section speaks. DA2.01 for train/validation/test discipline, cross-validation and metric choice, and DA2.02 for pipelines, feature engineering and, critically, leakage, which this node extends from the tabular case into text and time. And CS1.04 for the cost model: a network is a stack of matrix multiplies, and you should be able to say what one epoch costs before you run it.

Cross-links, not duplications (R1, C-3). The C-3 boundary is load-bearing in this node: the Finance-Plus quantitative-methods branch owns the concepts, and this module owns the code. QM1.01 derives probability, distributions and estimation. QM1.02 derives regression, hypothesis testing and what a p-value does and does not mean, the paired-test reasoning in the evaluation section applies QM1.02's machinery and does not re-derive it. QM1.03 owns time-series theory outright: AR, MA, ARMA and the ARIMA(p,d,q) notation, stationarity and unit roots, spurious regression and cointegration, Monte Carlo, and the conceptual introduction to machine learning for finance including the bias–variance decomposition. This node does not re-derive any of that. When you read "differencing makes the series stationary" here, the why lives in QM1.03; what lives here is the code that does it, the harness that evaluates it, and the verdict that reports it. Where a finance object appears, a limit order book, a factor panel, a risk metric, it belongs to QD2.01, QD1.02 and QD2.02 respectively and is named so you can follow it home.

Feeds forward. This is the last node of the data-science branch and it hands three things to the quant-developer branch. The walk-forward harness of DA2.03-e09 is the direct ancestor of QD1.03's no_lookahead harness, same instinct, one level more rigorous, applied to a full backtester. The honest-verdict function of DA2.03-e10 is the code-level form of the honesty criterion that QD2.05's capstone grades on its own line, where a strategy that only works in sample still passes if its limitations are reported and one that hides them fails. And the text pipeline feeds QD1.02's signal construction, where a sentiment score computed from filings becomes one more factor that has to survive IC, decay and turnover analysis before anyone trades it. QD2.03's deflated Sharpe is the same idea as this node's paired test on forecast errors: a result must be large enough to survive the number of times you looked.

Core teaching content

Part I: Deep learning, built from nothing

What a neural network actually is

Strip the branding and a feed-forward neural network is an alternating stack of two operations, repeated. The first is an affine map: multiply by a weight matrix and add a bias vector, Z = XW + b. That is linear regression, done many times in parallel, each column of W is one regression's coefficient vector, so a layer with eight hidden units is eight linear models computed simultaneously. The second is an elementwise non-linearity applied to the result, of which the modern default is ReLU, max(z, 0). Then you do it again.

That is the entire architecture. The only reason it is interesting is what happens if you leave the non-linearity out. Stack two affine maps with nothing between them and you get (XW₁ + b₁)W₂ + b₂ = X(W₁W₂) + (b₁W₂ + b₂), which is a single affine map with weights W₁W₂. Ten layers, a hundred layers, a million parameters: still one linear regression, wearing an expensive coat. The non-linearity is not a detail, it is the only thing that makes depth mean anything. Every hidden unit computes a linear function of the inputs and then switches off wherever that function is negative; a layer of them tiles the input space into regions, and the output head fits a different linear response on each region. A ReLU network is a piecewise-linear function whose pieces are learned rather than specified.

That is also, precisely, what it buys you. In DA2.03-e05 you fit y = |x|, a V. The best straight line through a V has a mean squared error of 0.336 (the target has variance 0.336, so the line does exactly as well as predicting the mean, which is to say: nothing). An eight-unit ReLU network gets 0.00009, a factor of about 3,700 better, by learning two pieces and the kink between them. That factor is the whole value proposition of a hidden layer, and it is why the exercise's hidden test demands you beat OLS by 5× before it passes: if your network cannot beat a straight line on a V, you have paid for non-linearity and not received it.

And it hints at the cost. A model that can bend to fit a V can bend to fit noise, and financial data is mostly noise. Hold that thought until Part III, where we measure exactly how much bending a 449-parameter network does to 394 rows of return data.

The forward pass

Concretely, for a one-hidden-layer network with a linear output head, a regression network:

Z1 = X @ W1 + b1      # (n, d) @ (d, h) + (h,)  ->  (n, h)
A1 = np.maximum(Z1, 0)                          #  ->  (n, h)
Z2 = A1 @ W2 + b2     # (n, h) @ (h, 1) + (1,)  ->  (n, 1)

Three lines and three things to be careful about, each of which is a hidden test in DA2.03-e01.

Shapes are the type system. Rows are observations, columns are features, and W1 maps d features to h hidden units. If you find yourself writing .T to make a multiply work, stop and re-derive rather than transposing until the error goes away, a transposed multiply that happens to have compatible dimensions computes something, silently, and the something is wrong.

Biases broadcast. b1 has shape (h,), one bias per hidden unit, added identically to every row. NumPy does this for free. A bias accidentally shaped (n,) also broadcasts (against the wrong axis) and gives you one bias per observation, which is a different and much worse model that trains happily and generalises to nothing.

The output head has no activation. Putting a ReLU on Z2 makes the network structurally incapable of emitting a negative number. For a regression head predicting a return, that is not a bug you find in a metric; it is a network that has been forbidden from ever forecasting a down day.

Keep X, Z1, A1 and Z2 in a cache dict. The backward pass needs every one of them, and recomputing them is exactly the waste that real frameworks avoid by recording a tape of operations as the forward pass runs.

Backpropagation, derived

Backpropagation is not an algorithm you memorise. It is the chain rule applied to a computation graph, evaluated right to left so that each intermediate derivative is computed once and reused. Derive it once, by hand, and it stops being frightening forever.

Take mean squared error over n rows, L = (1/n) Σ (Z2ᵢ − yᵢ)². Work backwards.

Step 1, the loss with respect to the output. ∂L/∂Z2 = 2(Z2 − y)/n. Shape (n, 1): one number per observation, the scaled residual. Everything downstream is this quantity routed backwards through the network.

Step 2: through the output layer. Z2 = A1 W2 + b2. For the weights, ∂L/∂W2 = A1ᵀ (∂L/∂Z2), shape (h, 1): matching W2, as it must. The intuition is worth having: a weight's gradient is the input that flowed along it multiplied by the error that came back down it, summed over the batch. For the bias, ∂L/∂b2 = Σ_rows ∂L/∂Z2, a sum over the batch, not a mean. The 1/n was applied once already in step 1; applying it twice does not error, it just makes the network train mysteriously slowly, and it is one of the two most common hand-derivation bugs.

Step 3: back to the hidden activations. ∂L/∂A1 = (∂L/∂Z2) W2ᵀ, shape (n, h). Note the mirror: forward multiplies by W2, backward multiplies by W2ᵀ. That symmetry holds for every affine layer and is the single most useful shape check you have.

Step 4: through the ReLU. ReLU's derivative is 1 where its input was positive and 0 where it was not, so ∂L/∂Z1 = ∂L/∂A1 ⊙ (Z1 > 0), an elementwise gate. Two things. First, the gate is on Z1, the pre-activation, not A1, they agree numerically here because A1 > 0 exactly where Z1 > 0, but only one of them is the derivative you derived, and on any other activation the distinction bites. Second, this gate is what "dead unit" means: a hidden unit whose pre-activation is negative for every row in the batch receives exactly zero gradient and therefore never updates. It is not learning slowly. It is not learning at all. DA2.03-e02's second hidden test constructs a dead unit deliberately and requires its weight gradient to be exactly zero.

Step 5: through the first layer. Identical in form to step 2: ∂L/∂W1 = Xᵀ(∂L/∂Z1) and ∂L/∂b1 = Σ_rows ∂L/∂Z1.

Seven lines of NumPy, and the invariant that catches most mistakes before any test does: every gradient has exactly the shape of the parameter it belongs to. That is what makes param -= lr * grad legal, and a shape mismatch is a derivation error, not a broadcasting inconvenience to be papered over.

The gradient check, which is the only reason to believe any of this

You will never be sure a hand-derived gradient is correct by looking at it. A wrong gradient does not raise; it points in a slightly wrong direction and the network still trains, just worse, and you attribute the shortfall to the data. The fix takes five lines and is the difference between an implementation you hope is right and one you have verified.

Perturb one parameter by a small ε, recompute the loss, perturb it the other way, recompute again, and estimate the derivative as (L(θ+ε) − L(θ−ε)) / 2ε. This is the central difference and its truncation error is O(ε²); the forward difference (L(θ+ε) − L(θ))/ε has error O(ε). On DA2.03-e03's own quadratic test problem at ε = 1e-5, that is the gap between a measured relative error of 5.8e-12 (central) and 2.0e-6 (forward), between proved and probably fine, six orders of magnitude apart. Compare analytic against numeric with a scale-invariant measure, max |a − n| / max(|a| + |n|, tiny), dividing by the sum rather than by |n| alone so the measure stays finite when a true gradient is zero, which with ReLU happens constantly.

The working thresholds, which are worth memorising:

Two disciplines make the check trustworthy. Restore every coordinate you perturb, bit for bit, a checker that quietly corrupts the parameters it is checking is worse than no checker, and DA2.03-e03's hidden tests verify the array comes back unchanged. And check on small random inputs, not on your real data: you are testing arithmetic, not fit, and a 4×4 problem finds the same bugs in a thousandth of the time.

Softmax, cross-entropy, and the two lines that stop them exploding

Swap the regression head for classification (tag each filing as miss, in line or beat) and two new pieces appear.

Softmax turns k raw scores into a probability distribution: pⱼ = exp(zⱼ) / Σ exp(zₗ). Cross-entropy scores that distribution against the truth: L = −log p_y, the negative log of the probability assigned to the correct class. Perfect confidence in the right answer costs 0; confidence in the wrong answer costs unboundedly much, which is the property that makes it the right loss for a classifier.

Both have a naive implementation that works on toy numbers and destroys real ones. np.exp(1000) is inf, and inf/inf is nan. The fix is one line and rests on an exact identity: softmax is invariant to a constant shift within a row, because exp(zⱼ − m)/Σexp(zₗ − m) cancels exp(−m) top and bottom. So subtract each row's maximum before exponentiating. Every exponent is then at most 0, every exp is in (0, 1], nothing overflows, and the answer is identical, not approximated.

For the loss, do not compute −log(softmax(...)) at all. Go through log-sum-exp directly: with m the row max, Lᵢ = log Σⱼ exp(zᵢⱼ − m) − (z_{i,yᵢ} − m). This never forms a probability that could underflow to 0 and then be logged to −inf. DA2.03-e04 proves the point with a single row of logits [0, 800] and a true label of 0: the correct answer is a loss of 800 nats, and the naive path returns inf.

The gradient is the reward for the algebra. For softmax followed by cross-entropy, the whole composite derivative collapses to

∂L/∂Z = (P − onehot(y)) / n        where P = softmax(Z)

Predicted probability minus truth, averaged over the batch. No exponentials, no divisions, no chain of intermediate terms. That collapse is why every real framework implements these two functions as one fused unit rather than as two composable ones, and why an implementation that computes them separately and then differentiates through both is both slower and less stable than the one you are about to write.

Initialisation, the training loop, and the seed

Three practical facts finish Part I.

Initialisation is not arbitrary. All-zero weights make every hidden unit compute the identical function and receive the identical gradient, forever, a symmetry gradient descent cannot break. Large weights push ReLU units into permanent saturation on one side. The standard for ReLU networks is He initialisation: draw from a normal distribution scaled by sqrt(2 / fan_in), which keeps the variance of the activations roughly constant as signal propagates through layers, so the network neither dies out nor blows up in depth. Biases start at zero because there is no symmetry there to break. DA2.03-e05 specifies the draw order exactly: W1, b1, W2, b2, from a single np.random.default_rng(seed), because a result is only reproducible if its starting point is.

The loop is four lines. Forward, record the loss, backward, then θ ← θ − lr·∇θ for every parameter. Full-batch gradient descent uses all rows each step. Mini-batch stochastic gradient descent uses a random subset, which is noisier per step, far cheaper per step, and (for reasons beyond this node's scope) often generalises better; on datasets of the size an analyst handles, full batch is simpler and adequate. The learning rate is the one hyperparameter you will tune by hand: too small and the loss crawls, too large and it oscillates or diverges to nan. Plot the loss curve. A curve that rises is a learning rate problem; one that plateaus instantly is a dead-unit or scaling problem; one that falls smoothly and then flattens is a network that has learned what it can.

Standardise your features. A network fed one column in rupees (hundreds) and one in log-volume (tens) will have gradients that differ by orders of magnitude between columns, and the learning rate that suits one will be wrong for the other. Subtract the mean, divide by the standard deviation, computed on the training window only, then applied to validation and test. Computing them over everything is leakage, and Part III will show you what it costs.

Seed everything, and check that you did. np.random.default_rng(seed) gives a stream that is stable across NumPy versions by policy. DA2.03-e05's hidden test runs the same configuration twice and demands bit-identical loss curves, then runs a different seed and demands a different one. A result you cannot reproduce is not a result; it is an anecdote about a computer.

What frameworks add, and what they do not

You have now built by hand what PyTorch and TensorFlow automate. It is worth being precise about what they add, because the honest list is shorter and less magical than the marketing:

• Automatic differentiation. Frameworks record the operations of the forward pass on a tape and apply the chain rule mechanically. You still have to know what the gradient is to debug one, which is why you derived it first. This is the big one: it means you can change the architecture without re-deriving anything.
• Hardware. GPU and TPU kernels, and the memory management to keep them fed. For a network with millions of parameters this is the difference between hours and weeks. For the models an analyst actually fits on a few hundred rows of monthly data, it is the difference between 0.4 seconds and 0.3 seconds.
• Layers, optimisers and schedules. Adam, dropout, batch normalisation, learning-rate schedules, convolutions, attention blocks, a library of components that are each a small amount of arithmetic and a large amount of accumulated empirical knowledge about what tends to work.
• An ecosystem. Pretrained weights, model zoos, serialisation formats, deployment tooling.

What they do not add: understanding of your data, protection from leakage, a stationarity assumption that holds, or a reason to believe your out-of-sample result. Every failure in Part III of this module happens identically in PyTorch, faster.

Running frameworks outside this app. Neither PyTorch nor TensorFlow is available in the browser Python runtime this app uses, so all graded deep-learning work here is from-scratch NumPy, which is the version that teaches, in any case. When you want to run a framework at scale, use a free hosted notebook (Google Colab and Kaggle Notebooks both offer free GPU sessions) and start with the official 60-minute PyTorch tutorial. Port your DA2.03-e05 network to torch.nn as your first exercise there: the same architecture, the same seed, the same data, and a check that the loss curves match. Half an hour that makes the whole framework legible, precisely because you already know what it is doing.

Where deep learning belongs in finance, and where it does not

Deep learning earns its reputation on problems with three properties: abundant data, high signal-to-noise ratio, and a stable generating process. Images have all three, a cat is a cat in 2015 and in 2026, and there are a billion labelled examples. Language has all three. Board games have them absolutely.

Financial return series have none of them, and it is worth being blunt about the magnitudes.

• Data volume. Twenty years of daily data on one instrument is about 5,000 rows. A modest image model trains on a million images. Monthly macro data over the same span is 240 rows. You are not data-rich; you are in the regime where a 449-parameter model on 394 rows will memorise the training set, which is exactly the measurement Part III makes.
• Signal-to-noise. In the shipped US snapshot the lag-1 autocorrelation of daily log returns is 0.018, giving an R² of about 0.0003: three parts in ten thousand. In the India snapshot it is 0.046, R² about 0.002. Even if those were real and stable rather than sampling noise, a model that captured all of the linear predictability would explain a fifth of one percent of the variance. Compare the 90%-plus achievable on a well-posed vision task.
• Non-stationarity. The relationship changes. Regimes shift, the participants who created a pattern trade it away, and the market is an adversarial system in which a discovered edge is a decaying asset. QM1.03 derives what stationarity means and how to test for it; the practical consequence here is that your training distribution is not your deployment distribution, and no amount of capacity fixes that.

Where deep learning does earn its place in finance: unstructured data at volume, text (millions of filings, news items, transcripts), images (satellite imagery of parking lots and storage tanks), audio (earnings-call tone), sequences of high-frequency market events where the data really is abundant. Notice the pattern: it wins where the inputs are rich and the model's job is representation, and it struggles where the input is a short numeric series and the model's job is to find a faint signal in overwhelming noise. Those are different problems that happen to use the same tool.

The professional summary, worth carrying: on tabular financial data of the size an analyst handles, a well-regularised linear model or a gradient-boosted tree is a stronger default than a neural network, and the neural network's job is to earn its way in against them. (Gradient-boosted trees (XGBoost, LightGBM, CatBoost) are the production workhorse for tabular problems industry-wide. They are named here because you will meet them constantly; they are not installable in this app's browser runtime, so the graded work uses scikit-learn's ensembles, which are the same idea with different engineering.)

{
  "skill": "Deriving and verifying a backward pass — and reviewing a copilot's version of it",
  "use": "AFTER you have derived the gradients on paper and passed DA2.03-e02 and DA2.03-e03 by hand, use an assistant to (a) red-team your derivation — ask it to name the input on which your ReLU gate is wrong — and (b) draft the boilerplate around the interesting part: the gradient-check driver, the loss-curve plot, the shape-assertion suite. Generate it in your own tool, paste it back, and let this node's hidden tests judge it.",
  "tools": ["Chat assistants — Claude, ChatGPT, Gemini", "IDE copilots — GitHub Copilot, Cursor", "Free hosted notebooks with GPUs for the framework port — Google Colab, Kaggle Notebooks"],
  "prompt": "Here is my hand-derived backward pass for a one-hidden-layer ReLU network under MSE, and my implementation. (1) Do NOT rewrite it. Instead, for each of the six gradients, state the shape it must have and whether mine matches. (2) Name every input on which my ReLU gate would give a different answer from the true subgradient. (3) Write ONLY a central-difference gradient-check driver that perturbs each parameter, restores it exactly, and reports the max relative error per parameter — no other code. (4) List the three most common sign or transpose errors in this exact derivation and the numerical symptom each one produces.",
  "verify": "Read every line before you run it. Then: run the gradient check and require relative error below 1e-7 on all four parameter blocks — that number, not the model's assurance, is what tells you the gradient is right. Confirm no NumPy or framework API in the generated code was invented (a copilot will cheerfully call `np.grad`, `torch.nn.MSE` or `model.backprop()`, none of which exist) by checking each against the real documentation. Train once with your gradient and once with the generated one on the same seed: if the loss curves differ at all, at least one of them is wrong.",
  "diy": "Gated. You must have written the five-step derivation out longhand and passed DA2.03-e02 and DA2.03-e03 before asking for anything here. A red-team of a derivation you cannot defend is a second opinion you have no way to evaluate — and a wrong gradient does not raise an error, it just trains slightly worse forever.",
  "market": "US"
}

Part II: Natural language, from counting words to reading filings

The five stages, and where most of the work actually happens

Text projects fail earlier than model projects do, and they fail in a stage nobody writes up. The sequence a working desk runs is always the same five stages, and it is worth naming them so you can say out loud which one you are in: problem formulation, then data curation, then text preparation and wrangling, then exploration, then model training. The first stage decides what the label even is, which for filings usually means deciding whether you are predicting a disclosed event, an analyst reaction or a price move, three quite different problems that all get called "sentiment". The second gathers the documents and, more importantly, records where each one came from and what date it was knowable, which is the point-of-time discipline QD1.01 owns. The last one is the part this module has already built from scratch.

The middle two are where the hours go, and they are the two that Part II has so far skipped over. Preparation turns a document into tokens you are willing to count. Exploration looks at those tokens before any model does, and it exists to stop you from weighting a vocabulary you have never looked at.

Normalisation, on one filing sentence

Take a sentence of the kind a filing actually contains and put it through the standard normalisation chain. Every step below is a decision, and every one of them throws information away on purpose.

"The Company recorded an impairment of $12.4 million in the Retail segment, and management is evaluating additional impairments relating to two underperforming stores."

Lowercase. "Company" and "company" become one term. Cheap, almost always right, and it costs you the ability to distinguish a defined term in a contract from its ordinary use, which occasionally matters.

Strip punctuation. The dollar sign, the commas and the full stop go. So does the decimal point inside 12.4, which is why the number arrives at the next step as two fragments rather than one.

Strip numbers. "12" and "4" disappear. This is the step people regret. In a filing, the magnitude of an impairment is frequently the entire signal, so a pipeline that discards digits has thrown away the number and kept the word. The defensible versions either keep numeric tokens and bucket them, or extract them separately before normalisation and feed them as their own features.

Collapse whitespace. The gaps left by the two steps above become single spaces, and the sentence is now one clean run of words.

Tokenise and drop stop words. Applying the same (?u)\b\w\w+\b pattern the vectoriser uses gives 22 tokens. Removing scikit-learn's English stop list, 318 terms, drops 9 of them and leaves 13: company, recorded, impairment, million, retail, segment, management, evaluating, additional, impairments, relating, underperforming, stores. Look at what left. Along with "the", "of", "in" and "is" went the word "two", because it sits in that stop list. A pipeline built to count store closures has just deleted the count, silently, and the only way to know is to have printed the dropped tokens once.

Stemming against lemmatisation, on the same sentence

The remaining thirteen tokens still contain a duplicate the matrix cannot see: impairment and impairments will occupy two separate columns and split one concept's weight across both. Two families of fix exist, and they are not interchangeable.

Stemming chops suffixes by rule, fast and dictionary-free, and does not care whether the result is a word. Porter-style stemming on this sentence gives impairment → impair, impairments → impair (the two columns merge, which was the goal), evaluating → evalu, relating → relat, underperforming → underperform, stores → store. Lemmatisation maps each token to its dictionary form using a vocabulary and, ideally, the part of speech: impairments → impairment, evaluating → evaluate, relating → relate, stores → store, with impairment unchanged. Both collapse the duplicate. Only one leaves you with terms a human can read.

The trade is straightforward. Stemming is faster, needs no language resources, and produces stems like evalu that are useless in any output a person reads, including the feature-importance table you will eventually put in front of a portfolio manager. Lemmatisation is slower, needs a dictionary and a part-of-speech tag to be accurate, and gets meeting wrong in one of its two senses without one. Use stemming when the vocabulary is an internal artefact and speed matters; use lemmatisation when a human will read the feature names. Apply exactly one of them, and apply it identically at fit time and at prediction time, or the two vocabularies will not line up and every unseen inflection will be dropped in silence.

Exploration: look at the counts before you weight them

Before any weighting scheme is chosen, count. On a six-document filing corpus with English stop words removed, the vocabulary is 25 terms, and the raw corpus frequencies rank as follows.

Two facts jump out of that table and both of them are arguments for the weighting scheme in the next section. First, the most frequent term, company, appears in four of six documents and separates nothing; it survived stop-word removal because it is not a general-purpose English stop word, only a domain-specific one. Every corpus has its own, and the frequency table is how you find them. Second, the terms that would actually distinguish a distressed filing from a healthy one, covenant and impairment, sit near the bottom of the count column, exactly where a raw-count model will ignore them.

Run the smoothed inverse document frequency on the same corpus and the ranking inverts where it should: company gets 1.336472, retail and segment get 1.559616, covenant, impairment and revenue get 1.847298, and underperforming, appearing once, gets 2.252763. Everything the next section builds follows from that inversion, and it is visible in a frequency table you can produce in three lines before fitting anything. Skipping the exploration pass is how corpora ship with a domain stop word doing half the ranking.

Text becomes a matrix

A model cannot read. Before anything else, a document must become a row of numbers, and the oldest recipe is still the most useful default: bag-of-words. Count which terms appear; throw away the order.

Three steps, each with a decision in it. Tokenise, lowercase the text and cut it into terms. This node uses scikit-learn's own default pattern, (?u)\b\w\w+\b, which keeps runs of two or more word characters and drops single letters and bare punctuation; matching it exactly is what lets you check your work against the library. Build a vocabulary, the sorted set of distinct tokens across the corpus, one column per term. Sorted, because a column order that depends on Python dictionary insertion order is a reproducibility bug waiting for a rainy day. Count: fill an (n_docs, n_terms) matrix. A token that is not in the vocabulary is silently skipped, which is exactly what a fitted vectorizer does to an unseen word at prediction time.

The result is enormous and almost entirely zero. A 10,000-document corpus with a 50,000-term vocabulary is a 500-million-cell matrix in which perhaps 0.2% of cells are non-zero. This is why scikit-learn's vectorizers return a scipy.sparse matrix rather than a dense array: calling .toarray() on a real corpus is the fastest way to exhaust memory in applied NLP. Sparse structures store only the non-zeros and their coordinates, and every scikit-learn estimator that matters accepts them directly.

TF-IDF: weighting words by how much they distinguish

Raw counts have a defect. In a corpus of filings, "the", "company" and "quarter" appear everywhere and separate nothing; "impairment" and "covenant" appear rarely and separate a great deal. TF-IDF reweights each count by how distinguishing its term is.

Using scikit-learn's default smoothed form, which DA2.03-e06 reproduces exactly:

idf_j = ln((1 + n_docs) / (1 + df_j)) + 1
X     = counts * idf                      # broadcast down the columns
X     = X / ||X_row||_2                   # L2-normalise each row

Every piece has a job. TF (the raw count) says a term repeated within a document matters more there. IDF damps terms that appear in many documents; the +1s smooth it so that a term appearing in every document gets weight exactly 1 rather than 0, and no document frequency can ever cause a division by zero. The trailing +1 keeps every weight positive so that no term is annihilated. And the L2 row normalisation makes a forty-page filing and a one-line headline comparable, without it, document length dominates every similarity you compute, and your model learns to detect long documents.

DA2.03-e06 builds all of this from scratch and then checks it against TfidfVectorizer on the same four-sentence corpus. On that corpus the vocabulary is 28 terms and the two matrices agree to within 5.6e-17: floating-point noise. That exact agreement is the point of the exercise: after it, TF-IDF is not a library call you trust, it is arithmetic you have reproduced.

Classification, the baseline, and the leak

With a matrix, everything from DA2.01 applies unchanged: logistic regression, linear SVMs and naive Bayes are all strong on sparse text, and a linear model on TF-IDF features is a genuinely competitive baseline that many production systems never beat.

Two disciplines make the result mean something.

Always report the majority-class baseline. A classifier that is 92% accurate on a corpus that is 91% one class has learned to say one word. On the balanced corpora in DA2.03-e07 the baseline is 50% and a correct pipeline reaches 83.3% (5 of 6) on both the India and the US held-out sets, a real gain, reported against the thing it beat. Accuracy without a baseline is not a weak result; it is not a result.

Fit on train, transform on test. fit_transform(train + test) is the most common leak in applied NLP, and it is nearly invisible: nothing errors, the score just quietly rises, because both the vocabulary and the IDF weights have been computed with the test documents in the room. On a chronological corpus (filings, news, transcripts) it is worse than ordinary leakage; it is look-ahead, because the test documents are from the future. On the India corpus in DA2.03-e07, fitting on train alone gives a 68-term vocabulary and fitting on train plus test gives 78, those extra ten terms are the model being told which words the future contains. That is why the exercise makes you return n_features and checks it: the leak leaves a fingerprint, and the fix is to look for it.

The correct split for text with timestamps is chronological, exactly as in Part III. Randomly shuffling filings before splitting trains on 2026 to predict 2024, and every metric you compute afterwards is fiction.

The ceiling of bag-of-words, stated honestly

Bag-of-words throws away word order, and word order carries meaning. DA2.03-e07's held-out set contains the sentence "Net loss widened and net debt rose for a third straight quarter", which a correctly-built classifier predicts as positive, because rose was a positive word in training and the representation has no idea what rose. Its US counterpart misses "Free cash flow rose and net debt fell" for the mirror-image reason.

That is not a bug in your code. It is the ceiling of the representation, and naming it is what separates an analyst from a scoreboard. Three other ceilings worth knowing by name:

• Negation. "not profitable" and "profitable" share every token but one, and a unigram model treats "not" as just another word. Character or word n-grams (ngram_range=(1, 2)) recover some of this by making "not profitable" its own feature, at the cost of a vocabulary several times larger.
• Domain vocabulary. General-purpose sentiment lexicons mislabel financial language systematically: liability, depreciation, tax, cost and capital are all negative in ordinary English and neutral accounting terms in a filing. Loughran & McDonald (2011) built a finance-specific word list for exactly this reason, and it remains the standard free starting point for filing sentiment.
• Boilerplate. A large fraction of any filing is unchanged from the previous filing. Much of the measurable signal in filing text is in the change (new risk-factor language, a removed forward-looking sentence) not the level. Practitioners routinely diff consecutive filings before scoring them, and doing so changes the answer more than any modelling choice.

Modern NLP, conceptually

Everything above represents a word as a column index: revenue is feature 4,182 and sales is feature 4,507, and nothing in the representation knows they are related. The last decade of NLP is the story of fixing that.

Static embeddings (word2vec, GloVe) map each word to a dense vector of a few hundred numbers, learned so that words used in similar contexts land near each other. Revenue and sales become neighbours. The limitation is that each word gets exactly one vector, so bank has a single position that is a compromise between a river and a financial institution.

Contextual embeddings and transformers fix that. A transformer computes a different vector for each occurrence of a word based on the words around it, using an attention mechanism that, for every position, learns how much to weight every other position. "The company's bank covenants" and "the river bank" get different vectors for bank. This is the architecture behind BERT, GPT and every current large language model, and AI0.01 in the AI-Augmented branch teaches its mechanism properly, tokens, embeddings, attention, and why fluent output is not evidence of correctness. Cross-reference it rather than re-deriving it here.

Fine-tuning versus prompting. Two ways to use a pretrained model. Fine-tuning continues training on your labelled data, producing a specialised model, accurate, reproducible, auditable, and requiring labels and GPU time. Prompting asks a general model to classify in natural language, no training, no labels, immediate, and non-deterministic, unauditable and dependent on a vendor's model version. For a research signal that must be reproducible three years from now, fine-tuning (or, honestly, TF-IDF plus logistic regression) is often the right professional answer, and the sophistication is in knowing that.

The practical hierarchy for an analyst, in order of what to try first:

Do not skip the first row. A TF-IDF logistic regression that takes eight seconds to fit is the benchmark every fancier approach has to beat, and reporting how much it beat it by is the entire content of the result.

Part III: Forecasting, and the honesty that makes it worth reading

Why a time series breaks ordinary machine learning

Everything in DA2.01 and DA2.02 assumed rows were exchangeable, that shuffling them and splitting at random produced a fair test. A time series violates that assumption in three separate ways, and each one has a specific consequence.

Order carries the information. Shuffle a return series and you destroy exactly the property you were trying to model. A random train/test split trains on Thursday to predict Wednesday. This is not a weak test; it is a wrong one, and it does not produce a noisy estimate of performance, it produces an optimistic one, systematically.

The distribution moves. Volatility clusters, correlations spike in crises, relationships decay as they are discovered and traded. Your training distribution is not your deployment distribution. QM1.03 derives stationarity and the unit-root tests that detect this; the code consequence is that a model validated on 2015–2020 has told you about 2015–2020.

Levels are trivially autocorrelated, and that makes fake results look spectacular. This is the trap that catches the most people, so here it is with numbers. Regress tomorrow's price on today's price in the shipped US snapshot and you get an R² of 0.9945; in the India snapshot, 0.9754. A chart of predicted against actual looks like a diagonal line. Both models are worthless: the slope is essentially 1, so each is literally the naive forecast ("tomorrow's price is today's price") dressed in the language of regression. Do the same on returns and R² falls to 0.000318 and 0.002152 respectively. Same data, same model, honest question.

Model returns or changes, not levels, unless you have a reason and a cointegration argument (QM1.03 derives that case). And whatever you model, compare it against the naive benchmark, which is the only reliable way to notice that your model has quietly become the benchmark.

The baselines you have to beat

Three trivial forecasters, and there is no excuse for reporting a model without them:

• Naive. Tomorrow equals today. On a true random walk this is the optimal forecast (no method can do better in expectation) which is precisely why it is so hard to beat on prices.
• Seasonal naive. This period equals the same period last season. The benchmark for anything with a calendar: quarterly earnings, monthly seasonality, day-of-week effects.
• Drift. The naive forecast plus the average per-period change over the training window, the straight line through the first and last training points, extended.

DA2.03-e08 implements all three. They cost nothing and they are the difference between "our model achieves an MAE of 1.34" and "our model achieves an MAE of 1.34 against a naive benchmark's 0.91, and therefore does not work."

Metrics: MAE, RMSE, and why MASE

MAE (mean absolute error) is in the units of the series and treats all errors proportionally. RMSE squares before averaging, so a single large miss dominates it; if one catastrophic error is much worse than several small ones (a risk limit, a margin call, a stop-out) then RMSE is the honest loss. If errors cost proportionally, MAE is. Decide which before you fit, not after you have seen which one flatters you.

Both are scale-bound, and that makes them useless for comparison. An MAE of 10.79 means nothing until you know whether the series is a ₹800 share or a 4,500-point index. The usual fix, MAPE, is worse than it looks: undefined at zero, explosive near zero, and asymmetric, it punishes over-forecasts more than under-forecasts, so optimising it biases your forecasts downward.

MASE (mean absolute scaled error (Hyndman & Koehler, 2006)) divides MAE by the in-sample MAE of the naive method, mean|yₜ − yₜ₋ₘ| over the training window. One number, one universally readable meaning, any currency, any scale:

• MASE < 1: beating the naive benchmark.
• MASE = 1: matching it.
• MASE > 1: losing to it.

Report MASE alongside MAE and RMSE, and report the benchmark's MASE next to your model's. Three numbers, no ambiguity, and no way to hide.

Walk-forward validation

The correct evaluation moves forward through time, refitting at each origin on nothing but the past:

for each origin t = initial, initial + h, initial + 2h, ...:
    fit / forecast using ONLY y[:t]
    score the forecast against y[t : t+h]

DA2.03-e09 builds this harness and makes look-ahead structurally impossible rather than merely discouraged: the forecaster is handed y[:t].copy(), so it is not physically capable of seeing a future observation, and the copy means a careless forecaster cannot write into your data mid-evaluation. That instinct (make the wrong thing impossible, not merely forbidden) is the same one behind QD1.03's no_lookahead harness, one module along.

Two design choices worth stating. Expanding versus rolling window: expanding uses all history at each origin (more data, but old regimes get a vote); rolling uses a fixed-length trailing window (adapts to regime change, discards information). Neither is universally right; state which you used, and if the result depends on the choice, report both. Refit frequency: refitting at every origin is honest and expensive; refitting every k origins is cheaper and is itself a modelling decision that must be disclosed, because a model refit daily and a model refit annually are different models.

Then two things that are not optional. A gap, if your features need one. If a feature at time t is computed from data only published at t + 2 (most fundamentals, most macro releases) you must skip those periods or you have look-ahead through the back door; QD1.01 owns point-in-time data and the vendor mechanics of getting it right. And honest counting: with 100 out-of-sample points you have 100 observations, not 500, and any claim you make has to survive a standard error computed on 100.

The verdict, and the discipline of reporting a loss

Here is the crown's centrepiece, run for you on both markets. All figures come from the app's shipped seeded synthetic EOD snapshots (us-eod, nse-eod, see app/public/datasets/MANIFEST.json, entirely fictional tickers generated by tools/build_datasets.py), so they are reproducible exactly and are not claims about any real security.

A one-hidden-layer, eight-unit ReLU network is fitted on five lagged daily log returns, standardised, refit at every origin, and walked forward one step at a time over the final 100 trading days of each series. Against the naive benchmark:

And the same architecture, given 64 hidden units and 3,000 epochs (449 parameters fitted to 394 rows) reaches an in-sample R² of 0.306 (US) and 0.322 (India). In sample it looks like a fund. Out of sample it loses to tomorrow equals today.

That gap is not a bug to be fixed with a better architecture. It is the honest measurement of what a small network can extract from a low-signal, non-stationary series, and it is the single most valuable number in this module.

Now the second discipline, which is subtler: a difference has to be big enough to be evidence. A simple 5-day moving average loses to naive by about 47% (US) and 50% (India) of MAE, a difference so large that no test is needed. But a drift forecast on the India series beats naive: MAE 10.7145 against 10.7885, an improvement of 0.69%. Is that a signal?

No. Paired on the same 100 days, the mean difference in absolute error is +0.074 ₹ with a standard error of 0.064, giving t = 1.15, and drift wins on 56 of 100 days, which under a fair coin has a two-sided probability of 0.27. On the US series the mirror case: drift loses by 0.28%, t = −0.34, winning 50 of 100 days. The MLP results above are the same story: paired t = −0.63 (p = 0.53) on US and t = −0.05 (p = 0.96) on India, with directional accuracy of 49/100 and 56/100 (binomial p = 0.92 and 0.27).

The 56% directional accuracy on the India series is exactly the number that gets a strategy funded in a bad shop. It is one ticker, one window, one hundred observations, and a p-value of 0.27. Compare on paired errors, over the same periods, with a standard error attached: QM1.02 derives the paired test, and the specialist form for forecast comparison is the Diebold–Mariano test (1995), which is the same idea with a heteroskedasticity- and autocorrelation-robust variance. QD2.03's deflated Sharpe applies the identical logic to strategy returns: the result must be large enough to survive the number of times you looked.

So the professional habit this crown certifies, in one paragraph. Build the model. Compute the baseline. Compute both metrics on the same out-of-sample window. Attach a standard error. And then write down what you found, including (especially) when what you found is that it did not work. DA2.03-e10 makes this mechanical: the function returns the comparison as data, a boolean and a sentence produced by the same call that produced the metrics, so "we didn't get around to benchmarking it" becomes an impossible sentence to say. Its hidden tests run a moving average on both markets and fail your submission if it reports a win.

That is the whole node. A model that cannot beat naive has produced no information. A report that does not show the baseline has produced no evidence. And an analyst who publishes the negative result is worth ten who publish only the positive ones, because only the first one's positives mean anything.

Common mistakes & how experts think differently

Trusting a gradient because the loss went down. A wrong gradient still points roughly downhill often enough for training to appear to work; the loss falls, just less than it should, and the shortfall gets blamed on the data. Experts gradient-check every hand-derived backward pass on a tiny random problem before running it on anything real, and treat a relative error above 1e-7 as a bug rather than a rounding artefact.

Computing exp before shifting. The naive softmax works on every toy example and produces nan the first time a logit reaches a few hundred. Experts write the row-max shift as reflex, and never form a probability just to take its logarithm, they go through log-sum-exp, because log(exp(x)) round-trips through a number that can overflow and x does not.

Averaging twice. The 1/n belongs in dZ2 once. Putting it in the bias gradients too is silent: no error, no warning, just a network that trains at a fraction of the expected rate on the bias parameters. This is the bug the shape-and-scale discipline of DA2.03-e02 exists to catch.

Gating the ReLU on A1 instead of Z1. Numerically identical for ReLU, conceptually wrong, and immediately wrong for any other activation. Experts write the derivative of the operation, not of the array that happens to be lying around.

Reaching for a network first. On a few hundred rows of tabular financial data, a regularised linear model or a gradient-boosted tree is the stronger default. Experts start with the simplest model that could work, measure it, and make every increase in complexity earn its place against that number. Complexity is a cost you pay in interpretability, reproducibility and overfitting risk, and it is charged whether or not it buys you anything.

Reporting accuracy without a baseline. 92% on a 91%-majority corpus is a model that has learned to say one word. Experts report the majority-class rate next to every accuracy, and the naive MASE next to every forecast, in the same table, without being asked.

Fitting the vectorizer on everything. fit_transform(train + test) leaks the vocabulary and the IDF weights, and on a chronological corpus that leak is look-ahead. Nothing errors; the score just rises. Experts fit on train, transform on test, and use a Pipeline so the discipline is enforced by structure rather than by memory.

Splitting a time series at random. It trains on the future to predict the past. The optimism is systematic, not noisy. Experts split chronologically, always, and treat a random split on temporal data as a result-invalidating error rather than a methodological preference.

Modelling price levels. An R² of 0.99 predicting tomorrow's price from today's is the naive forecast in disguise. Experts model returns or changes, and read a suspiciously high R² on a level series as a red flag rather than a triumph.

Tuning on the test set. Trying forty architectures and reporting the best out-of-sample result means the test set has been used for selection and is no longer a test set. Experts hold out a final window that is looked at once, and count how many configurations they tried, because that count is what determines how large a result has to be to mean anything.

Reporting only the run that worked. The deepest failure in the list, and the only one that is a character problem rather than a technique problem. Experts write down the negative result with the same care as the positive one, and they are the reason anyone can believe their positive ones.

How experts think differently, in one line each. They ask what is the baseline before they ask what is the model. They know the difference between a result and a measurement of their own optimism. They can state, before running anything, what result would make them abandon the idea. They treat "it works in sample" as a statement about their code, not about the world. And they regard a clearly-reported failure as a completed piece of work, because it is.

Worked examples

All market figures below come from the app's shipped seeded synthetic datasets (us-eod, nse-eod, us-index, nifty-index, fictional tickers, generated deterministically by tools/build_datasets.py; see app/public/datasets/MANIFEST.json for provenance). They are reproducible exactly and are not claims about any real security. Illustrative arithmetic is labelled as such.

Worked example 1: A forward pass computed by hand (mechanism, illustrative)

Take X = [[1, 2], [−1, 0.5]], W1 = [[1, −1], [0.5, 2]], b1 = [0, 1], W2 = [[2], [−1]], b2 = [0.5].

Row 1. [1, 2] @ W1 = [1(1) + 2(0.5), 1(−1) + 2(2)] = [2, 3]; add b1 → Z1 = [2, 4]. ReLU leaves both → A1 = [2, 4]. Then [2, 4] @ W2 = 2(2) + 4(−1) = 0; add b2 → Z2 = 0.5.

Row 2. [−1, 0.5] @ W1 = [−1(1) + 0.5(0.5), −1(−1) + 0.5(2)] = [−0.75, 2]; add b1 → Z1 = [−0.75, 3]. ReLU clips the first → A1 = [0, 3]. Then 0(2) + 3(−1) = −3; add b2 → Z2 = −2.5.

Three lessons in six lines of arithmetic. The bias was added once per unit and broadcast across both rows. The ReLU changed the answer for row 2, without it, Z2 would have been −0.75(2) + 3(−1) + 0.5 = −4.0, so the non-linearity is doing real work here. And the output is negative, which a ReLU on the output head would have clipped to zero. This is DA2.03-e01's harness.

Worked example 2: Backprop by hand, then verified numerically (mechanism, illustrative)

Take the smallest network with a dead unit: one input x = 1 on two rows, W1 = [[1, 1]], b1 = [−5, 0], W2 = [[1], [1]], b2 = [0], target y = 0 for both rows.

Forward: Z1 = [−4, 1] on each row, so A1 = [0, 1] and Z2 = 1. Loss = mean((1 − 0)²) = 1.

Backward, with n = 2: dZ2 = 2(1 − 0)/2 = 1 per row. dW2 = A1ᵀ dZ2 = [[0], [2]], the dead unit contributed nothing forward, so it earns nothing back. dA1 = dZ2 W2ᵀ = [1, 1] per row. The gate: Z1 = [−4, 1], so (Z1 > 0) = [0, 1] and dZ1 = [0, 1] per row. Finally dW1 = Xᵀ dZ1 = [[0, 2]] and db1 = [0, 2].

Unit 0's weight gradient is exactly zero. Not small: zero. It will never update, and unless something else changes its pre-activation sign it is dead for the rest of training. That is what "dying ReLU" means, and it is why practitioners watch the fraction of units active on a batch as a diagnostic.

Now verify it. Perturb W1[0,1] by ε = 1e-6 either way and recompute the loss: the central difference returns 2.0000000 to seven places against the analytic 2. Across all four parameter blocks on the exercise's own random 8×3 problem (seed 201, ε = 1e-6), the maximum relative error is 7.2e-9: comfortably inside the right band. This is DA2.03-e02's first hidden test, and the reason it uses numerical differentiation rather than a stored answer is that a stored answer can be reverse-engineered and a derivative cannot.

Worked example 3: Softmax and cross-entropy, by hand and at scale (mechanism, illustrative)

Logits z = [2, 1, 0.1] for a three-way filing tag. Shift by the row max: [0, −1, −1.9]. Exponentiate: [1, 0.367879, 0.149569], summing to 1.517448. Divide: p = [0.659001, 0.242433, 0.098566], which sums to 1.

If the true class is 0, the loss is −ln 0.659001 = 0.417. If the truth is class 2, −ln 0.098566 = 2.317. Same distribution, same confidence, five and a half times the loss, that asymmetry is what makes cross-entropy train a classifier rather than merely score it.

Now break it. Take z = [1000, 999, 998]. Naively, exp(1000) is inf in float64 (the maximum is about 1.8e308, and exp(709) already overflows), so every probability becomes inf/inf = nan and the batch is destroyed. With the shift, the exponents are [0, −1, −2], and the answer is [0.665241, 0.244728, 0.090031], the identical distribution you would get from [0, −1, −2], exactly as shift-invariance guarantees.

And the other failure. Take z = [0, 800] with true class 0. The softmax probability of class 0 underflows to exactly 0.0, so −log p is −inf. Through log-sum-exp: m = 800, shifted = [−800, 0], log Σ exp = log(1 + e⁻⁸⁰⁰) = 0, and the loss is 0 − (−800) = 800 nats. Large, correct, and finite. DA2.03-e04 tests both cases.

Worked example 4: The V that proves the hidden layer is worth something (mechanism, illustrative)

Fit y = |x| on 200 points evenly spaced over [−2, 2].

An OLS line through a symmetric V has slope 0 by symmetry, so it predicts the mean everywhere and its MSE equals the variance of the target: 0.336. An eight-unit ReLU network trained for 800 epochs at learning rate 0.05 from seed 1 reaches 0.00009, about 3,700× lower.

The mechanism is visible if you look at what the units learn: some activate for x > 0 and some for x < 0, and the output head combines them into +x on one side and −x on the other. The network has learned the kink, which is the one thing a linear model structurally cannot represent. This is exactly the capability that, applied to noise, becomes overfitting. Worked Example 8 measures that.

Worked example 5: TF-IDF reproduced exactly against scikit-learn (mechanism, illustrative)

Four filing-style sentences (the corpus in DA2.03-e06's first hidden test) tokenise under (?u)\b\w\w+\b to a 28-term sorted vocabulary beginning ['12', 'again', 'and', 'at', 'book', ...].

Take the term guidance, which appears in 3 of the 4 documents. Its smoothed IDF is ln((1 + 4)/(1 + 3)) + 1 = ln(1.25) + 1 = 1.2231. Compare revenue, also in 3 documents: identical weight. Now loss, in 1 document: ln(5/2) + 1 = 1.9163, a term appearing in a quarter of the corpus carries about 57% more weight than one appearing in three quarters. And a hypothetical term in all four: ln(5/5) + 1 = 1.0 exactly, which is the smoothing doing its job: damped to neutral, never annihilated.

After multiplying counts by IDF, each row is divided by its own L2 norm, so every document vector has length exactly 1 and a 40-page filing is comparable with a headline.

Checked against TfidfVectorizer() on the same corpus, the from-scratch matrix agrees to 5.6e-17, the vocabulary matches term for term and every weight matches to floating-point noise. That exact agreement is the deliverable: TF-IDF is now arithmetic you have reproduced rather than a library call you trust.

Worked example 6: A filing-language classifier on both markets, with its baseline and its ceiling (India ₹ + US $, synthetic corpora)

DA2.03-e07's hidden tests carry two synthetic filing-language corpora (an India set (₹, NSE-flavoured, fictional tickers NILAM and TARANG) and a US set ($)) each 18 training sentences split evenly between positive and negative tone, and 6 held-out sentences.

A TF-IDF + logistic-regression pipeline, fitted on training documents only, scores 5 of 6 = 83.3% on each held-out set against a majority-class baseline of 50%. Reporting the 83.3% without the 50% would be meaningless; reporting them together says the model found something.

The miss is the interesting part. On the India set the model calls "Net loss widened and net debt rose for a third straight quarter" positive. On the US set it calls "Free cash flow rose and net debt fell for a third straight quarter" negative. Both errors have the same cause: rose and fell carry the training set's tone, and a bag of words cannot know which noun the verb attached to. Bigrams would help ("net debt rose" as a single feature); a finance-specific lexicon would help; a contextual model would help most and cost most.

And the leak, measured. The India training vocabulary is 68 terms. Fitting the vectorizer on training plus held-out documents gives 78, ten extra terms, each one a word from the future that the IDF weights have now seen. The US corpus: 64 against 76. That is why fit_predict returns n_features, and why the hidden test compares it against the train-only count and fails on any other number.

Worked example 7: The spurious R² of predicting a price from a price (India ₹ + US $, synthetic)

Regress tomorrow's close on today's close, whole series, both markets:

The left column is the number that appears in bad research decks. It is also, read correctly, a confession: a slope of 0.9975 with a near-zero intercept is the naive forecast, ŷₜ₊₁ ≈ yₜ, and the R² is measuring how autocorrelated a price level is, which is a fact about arithmetic rather than about the market.

The right column is the honest version of the same question, and the honest answer is three parts in ten thousand for the US series and two in a thousand for the India one, and even those are in-sample estimates of a lag-1 autocorrelation (0.018 and 0.046 respectively) that a standard error would not distinguish from zero.

The diagnostic to internalise: an R² above 0.9 on a financial forecasting problem is not a triumph, it is a symptom. Check whether you are predicting a level from its own lag before you check anything else.

Worked example 8: The crown: a network that fits beautifully and forecasts nothing (India ₹ + US $, synthetic)

The full pipeline, on both markets, from the shipped snapshots.

Features. Five lagged daily log returns, standardised on the training window only. Model. The DA2.03-e05 network. Evaluation. DA2.03-e09's walk-forward harness, expanding window, refit at every origin, one-step-ahead, over the final 100 trading days. Verdict. DA2.03-e10.

In sample, with 64 hidden units and 3,000 epochs (449 parameters fitted to 394 rows) the network reaches R² 0.306 (US) and 0.322 (India). Presented alone, that is a chart that gets a strategy funded.

Out of sample:

Is the loss itself significant? Paired on the same 100 days: US t = −0.63 (p = 0.53), India t = −0.05 (p = 0.96). So the correct statement is not "the network is worse than naive" but the more useful "the network is indistinguishable from naive, at a great deal more cost and complexity".

Directional accuracy, the metric people reach for when MAE disappoints: 49 of 100 (US, binomial p = 0.92) and 56 of 100 (India, p = 0.27). The 56% is exactly the number that gets over-interpreted. One ticker, one window, one hundred days, p = 0.27.

And for scale, the simpler alternatives on the same windows: a 5-day moving average loses to naive by 47% (US) and 50% (India) of MAE. A drift forecast beats naive on the India series by 0.69% (t = 1.15, winning 56/100 days) and loses on the US series by 0.28% (t = −0.34, 50/100 days), the same method, opposite signs, both indistinguishable from noise.

What to write in the memo. Not "the model failed." This: "A five-lag MLP, walked forward over 100 out-of-sample days on each series, produced MASE of 0.814 (US) and 0.776 (India) against naive benchmarks of 0.804 and 0.775, statistically indistinguishable from the naive forecast (paired t = −0.63 and −0.05). In-sample R² of 0.31 and 0.32 on a 449-parameter model fitted to 394 rows is consistent with memorisation rather than signal. We do not find evidence of one-step-ahead predictability in these series at this frequency, and we would not deploy this model."

That paragraph is a complete, professional, publishable result. It is the deliverable of this crown.

Practice set

Work each problem fully before reading its solution. Numeric answers are to about ±2%. All market figures are from the shipped seeded synthetic snapshots; illustrative arithmetic is labelled.

P1 (guided). Shapes end to end. A batch of 64 rows with 12 features feeds a network with 32 hidden units and a scalar output. Give the shape of W1, b1, Z1, A1, W2, b2, Z2, and of every gradient. How many trainable parameters are there?

Solution. W1 is (12, 32), b1 is (32,), Z1 and A1 are (64, 32); W2 is (32, 1), b2 is (1,), Z2 is (64, 1). Every gradient has exactly the shape of its parameter: dW1 (12,32), db1 (32,), dW2 (32,1), db2 (1,). Parameters = 12×32 + 32 + 32×1 + 1 = 384 + 32 + 32 + 1 = 449. Note that the batch size appears in no parameter shape, that is what makes the same network usable on a batch of any size, and it is why a bias accidentally shaped (n,) is such a destructive bug.

P2 (guided). Why depth without non-linearity is a lie. Show algebraically that two stacked affine layers with no activation collapse to one, and state the practical consequence.

Solution. (XW₁ + b₁)W₂ + b₂ = X(W₁W₂) + (b₁W₂ + b₂). Setting W* = W₁W₂ and b* = b₁W₂ + b₂ gives a single affine map XW* + b*. The consequence: a network of any depth with linear activations has exactly the representational power of one linear regression, so it can never fit y = |x| better than the mean. Depth buys nothing; the activation is what buys everything. A corollary worth carrying: if your deep network performs identically to a linear regression, check that your activation is actually being applied.

P3. The bias gradient. Under L = (1/n)Σ(Z2 − y)², show that ∂L/∂b2 = Σ_rows ∂L/∂Z2 and explain why writing a mean there is a bug rather than a scaling choice.

Solution. b2 enters every row's output identically, so by the chain rule its derivative is the sum of the per-row derivatives: ∂L/∂b2 = Σᵢ ∂L/∂Z2ᵢ · ∂Z2ᵢ/∂b2 = Σᵢ ∂L/∂Z2ᵢ, since ∂Z2ᵢ/∂b2 = 1. The 1/n was already applied inside ∂L/∂Z2 = 2(Z2 − y)/n. Applying it again gives a bias gradient that is n times too small, so with n = 64 the biases learn at 1/64 the correct rate. Nothing errors; the network simply trains its biases 64× too slowly and you conclude the data is weak.

P4. The dead unit. For x = 1, W1 = [[1, 1]], b1 = [−5, 0], W2 = [[1],[1]], b2 = [0], two rows, target 0: compute dW1 and explain what happens to unit 0 for the rest of training.

Solution. Z1 = [−4, 1], A1 = [0, 1], Z2 = 1, loss 1. With n = 2, dZ2 = 1 per row; dA1 = [1, 1]; the gate (Z1 > 0) = [0, 1] gives dZ1 = [0, 1]; dW1 = Xᵀ dZ1 = [[0, 2]]. Unit 0's gradient is exactly 0, so its weight and bias never update. Since its pre-activation depends only on those parameters and on x, it stays negative for this input forever: the unit is dead. Mitigations in practice: a smaller learning rate (large steps are what kill units), He initialisation, or a leaky ReLU with a small negative slope so the gradient never fully closes.

P5. Central versus forward differences. At ε = 1e-5, roughly what relative error does each achieve on a smooth function, and what is the practical consequence for certifying a gradient?

Solution. Truncation error is O(ε) forward and O(ε²) central. Measured on DA2.03-e03's own quadratic test problem at ε = 1e-5: forward 2.0e-6, central 5.8e-12: six orders of magnitude. The consequence: a forward difference can detect a sign flip or a missing factor of 2, but it cannot distinguish a correct gradient from one that is wrong in the eighth decimal, so it cannot certify. Since the certification threshold is 1e-7, only the central difference can clear it. Note also that ε must not be too small: below about 1e-8 the subtraction f(x+ε) − f(x−ε) loses catastrophic precision in float64 and the estimate degrades again.

P6. Softmax shift-invariance. Prove that subtracting a constant from every entry of a row leaves softmax unchanged, and compute softmax([1000, 999, 998]).

Solution. exp(zⱼ − m)/Σₗ exp(zₗ − m) = [exp(zⱼ)exp(−m)]/[exp(−m)Σₗ exp(zₗ)] = exp(zⱼ)/Σₗ exp(zₗ). The exp(−m) cancels exactly, the result is identical, not approximated. So softmax([1000, 999, 998]) = softmax([0, −1, −2]) = [1, 0.367879, 0.135335]/1.503214 = [0.665241, 0.244728, 0.090031]. Naively, exp(1000) overflows float64 (whose maximum is ≈1.8e308, with exp(709) already too large) and every entry becomes nan.

P7. Cross-entropy at the extreme. A binary classifier outputs logits [0, 800] and the true class is 0. What is the loss, why does −log(softmax(...)) fail, and what does log-sum-exp return?

Solution. The softmax probability of class 0 is exp(−800)/(exp(−800) + 1), which underflows to exactly 0.0 in float64, so −log(0) is −inf (or raises a warning and poisons the batch). Through log-sum-exp with m = 800: shifted logits [−800, 0], log Σ exp = log(1 + e⁻⁸⁰⁰) = 0, and L = 0 − (−800) = 800 nats. The loss is genuinely enormous (the model is maximally, confidently wrong) but it is finite and differentiable, so training continues rather than collapsing.

P8. The fused gradient. State ∂L/∂Z for softmax followed by cross-entropy, verify it sums to zero across each row, and say why frameworks fuse the two operations.

Solution. ∂L/∂Z = (P − onehot(y))/n. Each row of P sums to 1 and each row of onehot sums to 1, so every row of the difference sums to 0 and the whole gradient sums to 0, a free correctness check. Fusing them matters for two reasons: numerical stability (the composite never needs to form a probability just to take its log) and efficiency (the analytic collapse avoids differentiating through an exponential and a division). It is also why torch.nn.CrossEntropyLoss takes raw logits, not probabilities, and why passing it softmax output is a classic and silent bug.

P9. Parameters against rows. The Worked Example 8 configuration has 5 inputs, 64 hidden units and 1 output, fitted to 394 rows. Count the parameters, compute the ratio, and say what in-sample R² = 0.31 means in that light.

Solution. 5×64 + 64 + 64×1 + 1 = 320 + 64 + 64 + 1 = 449 parameters for 394 rows, a ratio of about 1.14 parameters per observation. A model with more free parameters than observations can, in principle, interpolate the training data exactly; an in-sample R² of 0.31 in that regime is evidence of memorisation capacity, not of signal. The only informative number is the out-of-sample one, which here is a MASE of 0.814 against naive's 0.804, i.e. no skill.

P10. Sparse or dense. A corpus of 10,000 filings has a 50,000-term vocabulary, with an average of 300 distinct terms per document. Compare dense and sparse storage, and state the failure mode.

Solution. Dense: 10,000 × 50,000 = 5×10⁸ cells at 8 bytes = 4 GB. Sparse: about 10,000 × 300 = 3×10⁶ non-zeros, at roughly 12 bytes each (value plus index) ≈ 36 MB, about 110× less. Density is 0.6%. The failure mode is calling .toarray() on a fitted vectorizer's output, which materialises the 4 GB version and takes the process down; every scikit-learn estimator that matters accepts the sparse matrix directly, so the conversion is never necessary.

P11. IDF arithmetic. In a 4-document corpus, a term appears in 3 documents, another in 1, another in all 4. Compute each smoothed IDF and interpret.

Solution. idf = ln((1 + 4)/(1 + df)) + 1. For df = 3: ln(1.25) + 1 = 1.2231. For df = 1: ln(2.5) + 1 = 1.9163. For df = 4: ln(1.0) + 1 = 1.0000. The rare term carries about 57% more weight than the common one, and the universal term is damped to exactly neutral rather than to zero, which is what the two +1s buy: a term in every document is uninformative but not forbidden, and no df can make the ratio undefined.

P12. Measure the leak. The India corpus in DA2.03-e07 has a train-only vocabulary of 68 terms and a train-plus-test vocabulary of 78. Explain precisely what has leaked, why it inflates the score, and how the exercise catches it.

Solution. Two things leak. The vocabulary: ten terms that appear only in held-out documents get columns, so the model's feature space is defined using the future. The IDF weights: document frequencies are counted over train plus test, so every weight in the matrix (including on the 68 shared terms) is contaminated. The score inflates because the representation has been tuned to the exact documents it is about to be evaluated on. On chronological text this is look-ahead, not merely optimism. The exercise catches it by requiring fit_predict to return n_features and comparing it against the train-only count of 68; a train-plus-test fit reports 78 and fails. The structural fix is a Pipeline, so the discipline is enforced by construction rather than by memory.

P13. Where bag-of-words stops. Name three sentence pairs whose meanings differ but whose unigram bags are similar or identical, and give the cheapest fix for each.

Solution. (a) Negation: "not profitable" versus "profitable", differ by one token that the model treats as ordinary; fix with bigrams (ngram_range=(1,2)) so "not profitable" becomes its own feature. (b) Attachment: "net debt rose" versus "revenue rose", identical sentiment word, opposite meaning; fix with bigrams or dependency parsing, or accept the ceiling and report it. (c) Domain polarity: "increase in liability" versus "increase in revenue", liability is negative in general-purpose lexicons and neutral in accounting; fix with a finance-specific lexicon such as Loughran–McDonald. Each fix costs vocabulary size, and none of them recovers full compositional meaning, that requires a contextual model.

P14. Diagnose the R². A colleague reports R² = 0.97 forecasting a stock price one day ahead. State your first three questions and the most likely explanation.

Solution. (1) Are you predicting the level or the return? (2) What does your model give when you feed it "tomorrow equals today", i.e. what is the naive benchmark's R² on the same window? (3) Is the evaluation out of sample and chronological? The overwhelmingly likely explanation is a level-on-level regression, whose fitted slope will be within a whisker of 1: the model is the naive forecast, and R² is measuring the autocorrelation of a price series. In the shipped snapshots the same regression gives 0.9945 (US) and 0.9754 (India), while the equivalent on returns gives 0.000318 and 0.002152.

P15. MASE by hand. Training series [10, 12, 11, 13, 12, 14]; actuals [13, 15]; forecast [14, 14]. Compute MAE, the MASE scale, and MASE, and interpret.

Solution. MAE = mean(|13−14|, |15−14|) = mean(1, 1) = 1.0. The scale is the in-sample naive MAE: consecutive absolute differences are 2, 1, 2, 1, 2, mean = 1.6. MASE = 1.0/1.6 = 0.625. Below 1, so the forecast is better than a naive method would have been on the training data, by about 37.5%. The virtue of the number is that it carries no units: 0.625 means the same thing on a ₹800 share and a 4,500-point index, which is exactly what MAE cannot do.

P16. Why not MAPE. Give three reasons MAPE is a poor default for financial series and name the situation where it is still reasonable.

Solution. (a) Undefined at zero and explosive near it, returns, spreads and P&L series cross zero routinely, and one observation near zero can dominate the whole average. (b) Asymmetric, for a positive actual an under-forecast is capped at 100% error while an over-forecast is unbounded, so minimising MAPE biases forecasts systematically low. (c) Not comparable across series with different signs or scales of the base. It remains reasonable for strictly positive, well-away-from-zero quantities where relative error is the natural business unit: unit demand, revenue, volumes. For anything that can be negative or near zero, MASE.

P17. Walk-forward arithmetic. A series has 500 observations. You use an expanding window with initial = 400 and horizon = 5, non-overlapping. How many origins, how many forecasts, what history does the last origin see, and how many scored observations do you have for a significance claim?

Solution. Origins at t = 400, 405, …, 495 → 20 origins. Each produces 5 forecasts → 100 forecasts. The last origin, t = 495, sees y[:495], the first 495 observations, and never observation 495 or later. For significance you have 100 scored observations, not 500, and their errors are not independent (5-day blocks share a forecast origin), so an honest standard error should account for the block structure, which is exactly what the Diebold–Mariano test's autocorrelation-robust variance is for.

P18. Evidence or noise. Drift beats naive on the India series with MAE 10.7145 against 10.7885 over 100 days, mean paired difference +0.074 with standard error 0.064, winning 56 of 100 days. Is this evidence?

Solution. No. The improvement is 0.69% of MAE. The paired t-statistic is 0.074/0.064 = 1.15, nowhere near conventional significance; the sign test on 56 of 100 gives a two-sided probability around 0.27. And the same method on the US series loses by 0.28% (t = −0.34, 50/100), one method, two markets, opposite signs, both consistent with pure noise. The correct write-up is "drift and naive are indistinguishable on these series," and the correct next step is more data or a different hypothesis, not a paragraph about drift's promise.

P19. Choose the loss. For each: (a) a daily VaR-adjacent forecast where one huge miss is far worse than several small ones; (b) a volume forecast where all errors cost proportionally; (c) comparing forecast quality across a ₹800 stock and a 4,500-point index. Which metric, and why?

Solution. (a) RMSE: squaring makes a single large error dominate, matching the asymmetric cost of a tail miss. (b) MAE, linear in the error, matching a linear cost. (c) MASE, the only one of the three that is scale-free, because it divides by each series' own in-sample naive error. Report all three where they are cheap; the discipline is choosing the primary one before you fit, so the choice cannot be made by whichever number turned out best.

P20. Write the honest paragraph. Your model achieves out-of-sample MASE 0.814; the naive benchmark achieves 0.804. In sample the model reached R² 0.31 on 394 rows with 449 parameters. Write the four-sentence result paragraph.

Solution. "Walked forward over 100 out-of-sample days, the model achieved MASE 0.814 against the naive benchmark's 0.804, a 1.2% deterioration, statistically indistinguishable from zero on a paired test (t = −0.63, p = 0.53). The in-sample R² of 0.31 was obtained with 449 parameters on 394 observations, a regime in which memorisation is the expected outcome and in-sample fit carries no information about skill. We therefore find no evidence of one-step-ahead predictability in this series at this frequency. We do not recommend deployment, and we would revisit only with a different feature set, a lower frequency, or a cross-sectional rather than time-series formulation." Four sentences: result with benchmark and significance, why the in-sample number is not evidence, the conclusion, and the falsifiable next step. That is a complete piece of work.

P21. The token the normalisation ate. Run the standard chain (lowercase, strip punctuation, strip numbers, collapse whitespace, tokenise on (?u)\b\w\w+\b, remove scikit-learn's English stop list) over "The Company recorded an impairment of $12.4 million in the Retail segment, and management is evaluating additional impairments relating to two underperforming stores." State the token count before and after stop-word removal, name the tokens dropped, and identify the two pieces of information the chain destroyed that a filing analyst would want back.

Solution. Tokenisation gives 22 tokens; stop-word removal drops 9 and leaves 13: company, recorded, impairment, million, retail, segment, management, evaluating, additional, impairments, relating, underperforming, stores. The nine dropped are the, an, of, in, the, and, is, to, two. Two losses matter. First, the amount: "12.4" was split by the punctuation strip into "12" and "4" and then deleted by the number strip, so a sentence whose entire point is a $12.4 million impairment now carries only the word "million". Second, the count: "two" is a member of scikit-learn's 318-term English stop list, so the number of underperforming stores was removed by a general-purpose list that has no idea it was counting anything. Both are recoverable only by extracting numeric spans before normalisation and carrying them as separate features. The habit worth building is to print the dropped-token list once per corpus and read it, because a domain word hiding in a general stop list is invisible in every downstream metric.

P22. Stem it or lemmatise it. For each output below, decide whether it is going into an internal feature matrix or into a table a portfolio manager will read, then choose stemming or lemmatisation and say what the chosen method does to impairments, evaluating and stores. (a) A 40,000-document nightly classifier whose only output is a probability. (b) A quarterly note listing the twenty terms that moved a filing's distress score most.

Solution. (a) Stemming. Nobody reads the vocabulary, the corpus is large enough that speed and the absence of a dictionary dependency both matter, and merging impairment and impairments into one column is the only property being bought. Porter-style output: impairments → impair, evaluating → evalu, stores → store. (b) Lemmatisation. The feature names are the deliverable, and a table whose top row reads evalu will not survive contact with a portfolio manager. Output: impairments → impairment, evaluating → evaluate, stores → store. Both methods collapse the duplicate column, which was the shared goal; they differ only in whether the surviving term is a word. Two rules apply either way. Choose exactly one method, never both, because a token stemmed at fit time and lemmatised at prediction time lands in no column at all. And apply it inside the same fitted object that builds the vocabulary, so the transformation cannot drift between the two stages.

Applied mini-project

Forecast something, and then tell the truth about it.

This project composes the entire node. You will build a forecasting system end to end (features, model, walk-forward evaluation, benchmark comparison, significance) and produce a memo. The memo is graded on honesty as its own line, and a model that loses to naive can score full marks while one that hides the comparison cannot pass.

Choose one target from the shipped datasets (all seeded synthetic, say so in the memo):

• Track A (price/return). One-day-ahead returns for one ticker in us-eod and the same design for one in nse-eod (R6: both markets).
• Track B (macro). One-month-ahead change in a fred-macro series and its rbi-macro counterpart.
• Track C (text-augmented). Either track above, plus a sentiment feature built with your DA2.03-e06/e07 pipeline over a filing-language corpus you construct and label yourself. Hardest, most interesting, and the one where the leakage discipline earns its keep twice over.

Do this, in order. Write down step 2's answer before you run anything.

1. State the hypothesis in one sentence, with the horizon, the frequency and the universe. "Five lagged daily returns predict the sign of the next day's return for large-cap Indian equities" is a hypothesis. "Machine learning on stock data" is not.
2. State your kill criterion, in advance. What result would make you abandon this? Write the number down before you see any output. This is the single most valuable habit in the project, and a memo without it is a memo you cannot trust, including from yourself.
3. Build the features, standardised on the training window only. Include a written list of every input and the exact timestamp at which it would have been knowable. Anything published with a lag needs a gap in the harness (QD1.01 owns the point-in-time mechanics).
4. Fit three models: your DA2.03-e05 MLP, a linear model (ridge or OLS), and one scikit-learn ensemble. Seed everything. Record every hyperparameter you tried, including the ones that did not work, the count matters for step 7.
5. Evaluate with DA2.03-e09. Chronological, expanding or rolling (say which), refit at a stated frequency, on a final window you look at once.
6. Compute the verdict with DA2.03-e10 for every model, against naive and, where the series has a calendar, seasonal naive. Report MAE, RMSE and MASE for the model and the benchmark, in one table.
7. Attach a standard error. Paired differences of absolute errors, mean, standard error, t-statistic. State how many configurations you tried; a result selected from twenty attempts needs to be much larger than one from a single pre-registered attempt.
8. Ablate. Rerun with (a) a random rather than chronological split, and (b) standardisation computed over the whole series rather than the training window. Report how much each inflates your result. These two numbers are the most persuasive evidence in the whole memo that your main result is clean.
9. Write the memo. Two pages maximum.

Deliverable. One reproducible script plus a two-page memo containing: the hypothesis and the pre-registered kill criterion; the feature table with knowability timestamps; the results table (three models × MAE/RMSE/MASE, with naive and seasonal-naive rows); the significance paragraph; the two ablation numbers; and a limitations section that names at least three specific things that could still be wrong with your own result.

Rubric (100 points; pass at ≥ 80 with no zero on any line, this is a crown node).

Self-check before you submit. Can you say out loud, in under sixty seconds: what you forecast, what you compared it against, what the difference was, whether that difference was larger than its own standard error, and what you would need to see before you would trade it? If the honest answer to the middle question is "it lost," and you can say that sentence without flinching, you have passed the part of this node that matters.

{
  "skill": "Red-teaming your own forecasting result before anyone else does",
  "use": "AFTER your walk-forward numbers exist and your memo draft is written, paste the METHODOLOGY (not the conclusion you want) into an assistant and ask it to attack. Use it as an adversarial reviewer for leakage, benchmark omissions and multiple-comparison problems, and to draft the ablation scripts. Generate in your own tool, paste the code back here, and let this node's harness and hidden tests judge the result.",
  "tools": ["Chat assistants — Claude, ChatGPT, Gemini", "IDE copilots — GitHub Copilot, Cursor", "This node's code pack (DA2.03) as the grader — the app never calls a model at runtime"],
  "prompt": "Here is my forecasting methodology: <FEATURES, SPLIT, MODEL, REFIT SCHEDULE, METRICS, BENCHMARK>. Do NOT tell me whether the result is good. Instead: (1) List every path by which information from the evaluation window could have reached the model — feature construction, scaling, vectorizer fitting, hyperparameter selection, target definition, survivorship of the universe. (2) Name the benchmark I should have compared against and did not. (3) Given that I tried <N> configurations, state how much larger my result needs to be to survive that multiplicity, and name the correction. (4) Write ONLY the two ablation scripts — random-split and full-series-standardisation — that would quantify how much each mistake would have inflated my number.",
  "verify": "Every leak the model names, go and check in your own code rather than accepting the list; it will invent some and miss others. Re-derive any statistical correction it proposes against QM1.02 rather than taking the formula on trust — copilots reproduce multiple-testing corrections wrongly at a high rate. Confirm every API in the generated ablation scripts exists (a plausible-looking `sklearn.model_selection.TimeSeriesSplit(gap=...)` argument may or may not be real for your version — check the installed documentation, not the model). And note the asymmetry that makes this safe: an assistant that hallucinates an extra leak costs you an hour of checking, while one you never asked costs you a published result.",
  "diy": "Gated: run the walk-forward evaluation and compute the naive benchmark BY HAND first, and write your own limitations paragraph before asking for a critique. An adversarial review of a result you have not yet interrogated yourself just replaces your judgement with a plausible-sounding one — and this is the exact node where that trade is most expensive.",
  "market": "IN"
}

Reading & resources

Everything below is an anchor to learn from and verify against, never a source to quote a number from without re-deriving it (R3). Tags: ✓ free / $ paid, and ▲ beginner–intermediate / ▼ advanced.

Deep learning, in the order that actually works:

• 3Blue1Brown, Neural Networks (YouTube series), the best visual intuition for what a layer and a gradient are. Watch before anything else. [✓] [▲]
• Andrej Karpathy, Neural Networks: Zero to Hero and the micrograd repository, builds autodiff and backprop from nothing, in the same spirit as this node and at greater depth. The single best free resource on this list. [✓] [▲–▼]
• Stanford CS231n notes, especially "Optimization: Backpropagation" and the gradient-check section, the source of the relative-error thresholds used above. [✓] [▼]
• Goodfellow, Bengio & Courville, Deep Learning: the standard reference; read chapters 6 and 8 as a reference, not front to back. Free online. [✓ online / $ print] [▼]
• PyTorch's official 60-Minute Blitz and the Colab/Kaggle free-GPU notebooks, where to port your DA2.03-e05 network once you have written it by hand. [✓] [▲]

NLP:

• Jurafsky & Martin, Speech and Language Processing (3rd ed. draft, free online), chapters on n-grams, naive Bayes, logistic regression and vector semantics. The reference. [✓] [▲–▼]
• scikit-learn's "Working With Text Data" tutorial and the TfidfVectorizer documentation, read the docstring's formula and check it against what you built in DA2.03-e06. [✓] [▲]
• Loughran & McDonald (2011), "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks", Journal of Finance, why general-purpose sentiment lexicons mislabel financial language, plus the free finance-specific word lists. The paper to read before any filing-sentiment project. [✓ lists / paper via library] [▲]
• Hugging Face's NLP Course: transformers, tokenisers and fine-tuning, hands-on and free. [✓] [▲–▼]
• Cross-reference AI0.01 (How Modern AI Actually Works) for tokens, embeddings, attention and why fluent output is not evidence of correctness, this node deliberately does not re-derive it.

Forecasting and honest evaluation, the most important block on this page:

• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed., free online), the single best resource here. Read the chapters on benchmark methods, evaluation and cross-validation for time series. If you read one thing after this node, read this. [✓] [▲]
• Hyndman & Koehler (2006), "Another look at measures of forecast accuracy", International Journal of Forecasting, the MASE paper, and the definitive argument against MAPE. [$/library] [▼]
• Diebold & Mariano (1995), "Comparing Predictive Accuracy", JBES, the formal test for "is this forecast better than that one," and Diebold's own later note on when it should not be used. [$/library] [▼]
• The M4 and M5 forecasting competitions and Makridakis et al.'s result papers, thousands of series, dozens of methods, and the repeated finding that simple methods and combinations are extremely hard to beat. Free, and a permanent inoculation against method-worship. [✓] [▲–▼]
• López de Prado, Advances in Financial Machine Learning, purged and embargoed cross-validation, the deflated Sharpe ratio, and the multiple-testing problem in strategy research. Cross-reference QD1.03 and QD2.04. [$] [▼]
• statsmodels documentation for tsa (ARIMA, SARIMAX, ETS), the classical models QM1.03 derives, in code you can run in this app. [✓] [▲]

The finance bridge:

• Cross-reference QM1.03 for the time-series theory (AR/MA/ARIMA, stationarity, unit roots, cointegration, Monte Carlo) that this node implements rather than re-derives, and QM1.02 for the paired-test machinery behind every significance claim above.
• Your own DA2.03 code pack: ten exercises whose hidden tests check gradients against numerical differentiation, feed logits that only a stable implementation survives, and require your evaluation code to report a loss to the naive benchmark. The only resource here that answers back.

Do this, not just read: open Hyndman & Athanasopoulos chapter 5, run their benchmark methods on one of this app's shipped series, and reproduce the MASE table from DA2.03-e08 against their fable output. Ninety minutes, and forecast evaluation stops being something you look up.

Teach it back & journal

Feynman prompt. Explain to a capable colleague who uses scikit-learn confidently but has never written a gradient why their neural network's beautiful training curve tells them almost nothing, using the sentence "a model that cannot beat a naive baseline has produced no information." Then, without equations, explain what a hidden layer actually does that a linear regression cannot (use the V), and what that same flexibility costs when the data is mostly noise. Finish with the two numbers you would demand to see next to any forecast result they ever show you, the benchmark's metric on the identical window, and the standard error of the difference, and why one without the other is still not evidence. If they interrupt to ask "but what was the naive number?" about someone else's result, you have installed the reflex, which is the whole point of this node.

Journal prompt. Take one model you have actually built (in this program or at work) and audit it against this node. Write down, in order: what you compared it against; whether that comparison was computed on the identical window; whether your split was chronological; where your scaling statistics came from; and how many configurations you tried before reporting the one you reported. Then answer four questions in writing. First: if you had to attach a standard error to the difference between your result and its baseline, could you, and if not, what would you need? Second: what was your kill criterion, and if you did not have one, what would you have accepted as evidence that the idea did not work, honestly, before you saw any output? Third: think of a result you have seen presented, by a vendor, a colleague, a paper, or yourself, where the baseline was absent. What would you now ask, and what do you suspect the answer would have been? And finally, the one that matters most for a career rather than a module: over the next year, which will do more for how your work is trusted, the models you got to work, or the clearly-written record of the ones you correctly abandoned? Name the specific habit from this node you will make automatic, and the next piece of analysis you will apply it to.

This module's flashcards and mastery quiz are wired into the app: see the node's Quiz and Reviews.