Reproducible Research, Deployment & the Quant Stack

Learning objectives

By the end you can:

1. State the module's thesis from memory, a number is not a result until it reproduces, and not a finding until you know how many tries it took, and explain why reproducibility and multiple-testing discipline are one habit, not two.
2. Resolve a layered production configuration (defaults, file, environment, invocation) into one auditable config, in a fixed, stated precedence order, without mutating any input layer.
3. Distinguish a .env secret (CS2.01) and a logging redaction filter (CS2.06's redact) from a secret-reference indirection system, and explain why a pipeline should fail loudly, not silently, on a missing secret.
4. Build and use a run manifest (the single artifact that ties a run's seed, resolved config, code version, and data snapshot together) and use it to detect drift between two runs that should have been identical.
5. Explain why "I set the seed" is necessary but not sufficient for determinism once a pipeline has more than one source of randomness, and connect this to QD1.03's determinism harness as the test your own infrastructure must be built to pass.
6. Design a schedule (cron-style), and explain idempotency, why a scheduler that fires at least once, not exactly once, requires your pipeline to tolerate being invoked twice for "the same" run without corrupting its own record of what happened.
7. Implement exponential backoff with full jitter for a retrying scheduled job, from your own seeded generator, and explain why a retry policy is part of a pipeline's reproducible behavior, not an implementation detail beneath notice.
8. State the multiple-testing problem in one sentence, compute the expected number of false "discoveries" from N independent skill-less trials at a stated significance level, and explain why judging each trial against the ordinary threshold guarantees false positives as your trial count grows.
9. Implement the Holm–Bonferroni step-down correction by hand and in code, explain why it dominates plain Bonferroni without sacrificing rigor, and apply it to a family of backtest trial p-values.
10. Explain why an experiment ledger must be append-only and outside any single researcher's power to quietly edit, and connect this directly to why QD2.05's capstone scores honesty about limitations as its own graded line, separate from a strategy's raw performance.
11. State, precisely and without re-deriving any of it, the division of labor across QM1.02/QM1.03 (the statistics), QD1.03 (a single backtest's own parameter-search deflation and the four harnesses), QD2.03 (deflated Sharpe as a reported performance statistic), and this module (the logged, ledgered, scheduled infrastructure that makes every one of those numbers honest).
12. (Productivity objective: R10 duality.) Use an AI copilot to draft the unglamorous scaffolding of deployment infrastructure, a scheduler config, a logging skeleton, a ledger schema, while keeping the decision-bearing logic, and every subtle correctness question a reviewer must check, for yourself.

The duality, stated once (R10). Objectives 1–11 are the understanding gate; you clear it by building the infrastructure and reasoning about it correctly, with or without an AI tool in the room. Objective 12 is the productivity payoff you keep afterward. No tool buys you a pass on the gate, this region's own rule, R10, holds here exactly as it does everywhere else in the app: the app itself never calls a model at runtime, so every AI-flavored exercise is "generate it in your own tool, paste it back, our hidden tests grade it locally."

Prerequisites & connections