Statistics in Code

Learning objectives

By the end you can:

1. Simulate a sampling distribution in code (seeded bootstrap resampling with NumPy) and use it to confirm, numerically rather than algebraically, the standard-error law QM1.01 derives: SD(x̄) = σ/√n.
2. Run a one-sample t-test (scipy.stats.ttest_1samp), choose correctly between an independent two-sample test (ttest_ind, Welch's equal_var=False as the safer default) and a paired test (ttest_rel), and name the specific way a real dataset can silently violate the independence a ttest_ind call never checks for you.
3. Compute a two-sample F-test for equal variances entirely from scipy.stats.f, knowing that scipy carries no single built-in function for it, and run a chi-square test of independence via scipy.stats.chi2_contingency.
4. Test a correlation coefficient for significance with scipy.stats.pearsonr in one call, and separate a tiny-but-significant correlation (large n) from an economically meaningful one.
5. Fit and read a simple and a multiple OLS regression (statsmodels.api.OLS(y, sm.add_constant(X)).fit()), mapping every field in .summary() (coefficient, SE, t, p, R², adjusted R², overall F) back to the exact QM1.02 formula that produces it.
6. Run and correctly interpret all four regression diagnostics in code: het_breuschpagan, variance_inflation_factor, linear_reset (all statsmodels.stats), and Durbin–Watson, and name the one condition (a single, correctly time-ordered series) that makes the last one meaningful at all.
7. Fit a logistic regression (statsmodels.api.Logit), read a coefficient as a log-odds effect, convert it to an odds ratio, report McFadden's pseudo-R², and build a confusion matrix at a chosen probability threshold.
8. Build a small, linear pipeline, load real data with pandas, pick the statistically correct test or model for its actual structure, run it, and state the conclusion in one sentence that separates statistical significance from economic size, the automated form of QM1.02's "read a full regression printout in one pass."
9. (Productivity objective: R10 duality.) Use an AI copilot under the Copilot Discipline to draft statistical code, and verify it against the specific failure modes this module names: a hallucinated function name, a wrong default (equal_var=True when the data demands Welch's), a missing intercept, or a diagnostic silently never run.

The duality, stated once (R10). Objectives 1–8 are the understanding objective: the gate (quiz ≥ 85%, code-pack ≥ 85% with hidden tests green) rewards choosing the statistically correct call for a given dataset and reading what comes back correctly, closed-book where the reading is concerned. Objective 9 is the productivity payoff: a copilot can write a scipy.stats.ttest_ind(a, b) call in one second, it cannot tell you, on its own authority, whether your a and b are actually independent, and it will not notice a missing intercept unless you know to look. Understanding is what makes the acceleration safe.

Prerequisites & connections

Builds on. DA1.01 (NumPy & Vectorized Computation) is where arrays, vectorized arithmetic, and a seeded random generator (numpy.random.default_rng) first become fluent tools, every simulation in §3 leans on exactly that. DA1.02/DA1.03 (pandas I/II) are where pd.read_csv, filtering, groupby, and time-indexed slicing become second nature, every dataset load in this module assumes that fluency and does not re-teach it. DA1.04 (EDA & Visualization) is where you learned to look at a distribution before testing it, a histogram or a residual plot before a p-value, the same "look before you test" habit Worked Example 6 in QM1.02 modeled by hand. Above all, this module builds on QM1.01 (Probability, Distributions, Sampling & Estimation) and QM1.02 (Hypothesis Testing, Correlation & Regression), the formulas, the assumptions, the failure modes, all taught there, by hand, first.

BINDING BOUNDARY (C-3). QM1.01 and QM1.02 teach statistics and regression as concepts, the intuition, the exact math, the failure modes, computed by hand and by table lookup. This module teaches the identical material in software: scipy.stats, statsmodels.OLS/Logit, real datasets, hidden-test grading. The boundary runs in both directions and neither side crosses it: this module never re-derives a formula QM1.01 or QM1.02 already gave you, every test statistic's formula, every assumption, every diagnostic's definition is cited back, not rebuilt from scratch, and in exchange this module owns everything QM1.02 explicitly declined to teach: which function to call, what its defaults actually do, how a real dataset's shape can silently break an assumption the function itself never checks, and how to read what comes back without being fooled by clean formatting. If you find yourself re-deriving Sxy/Sxx or the Breusch–Pagan LM statistic's algebra here, stop, that content already exists, in QM1.02, and citing it is the correct move, not repeating it.