ML I — Supervised Learning

Learning objectives

By the end you can:

1. State the one-sentence discipline that governs everything in this node (fit on data with known answers, judge only on data the model never touched during fitting) and explain, in mechanism, why grading a model on its own training rows is not merely optimistic but actively meaningless.
2. Fit and score a linear regression using scikit-learn's fit/predict contract, and compute its two standard regression metrics (RMSE and R²) correctly, on the test set only, by hand from sklearn.metrics (not by re-deriving least squares, which is QM1.02's job).
3. Build a leakage-free three-way train/validation/test split in code (sklearn.model_selection.train_test_split, applied correctly twice) that is deterministic under a fixed seed and exactly partitions every row exactly once.
4. Run k-fold cross-validation (KFold + cross_val_score) to score a candidate model's complexity using only the training set, and correctly read neg_mean_squared_error's sign convention.
5. Fit a decision tree for both regression and classification, control its capacity with max_depth, and connect a max_depth sweep's train/test score table directly to QM1.03's bias-variance shape, a concept you already hold, now a number you compute.
6. Fit a random forest, explain the bagging mechanism (bootstrap resampling + averaging) that lets an ensemble of individually overfit trees generalize better than any single one, and demonstrate this numerically on held-out data.
7. Fit a logistic-regression or tree-based classifier and compute the full metric set that matters for it (accuracy, precision, recall, F1, ROC-AUC) and explain precisely why accuracy alone is actively misleading on an imbalanced target.
8. Choose the right primary metric for a stated task (regression versus classification, and (within classification) balanced versus imbalanced classes), and defend the choice from first principles rather than habit.
9. Diagnose underfitting versus overfitting from a train/test (or train/CV) score table alone, identifying the best-generalizing complexity level and the point at which the train-test gap signals a model has started memorizing noise.
10. (Productivity objective: R10 duality.) Use an AI copilot to accelerate the unglamorous parts of a supervised-learning workflow, drafting a first-pass sklearn pipeline, or explaining a cryptic warning, while applying the three-law Copilot Discipline: understand the by-hand version first, then accelerate, then always review every line and verify no API was hallucinated before trusting a single number it produced.

The duality, stated once (R10). This module carries two objectives at once, as every module in this program does. The understanding objective (items 1–9) is what the mastery gate rewards; you earn the pass by fitting real models, computing real metrics, and reading them correctly, not by having watched someone else do it. The productivity objective (item 10) is the payoff you keep afterward: knowing when a copilot can safely draft the unglamorous scaffolding of a modeling script, and knowing exactly what "review it first" has to mean before a metric it computed enters a report. A tool can never buy you a pass on the gate.

Prerequisites & connections

Builds on. DA1.01 (NumPy) and DA1.02 (pandas I), every function in this node takes plain lists or arrays of features and targets, and in real work those arrays come out of a pandas DataFrame you already know how to build and slice. DA1.04 (EDA & Visualization) is where you would have first plotted a feature against a target and felt whether a relationship looked linear, clustered, or noisy, the instinct this node turns into a number. DA1.05 (Statistics in Code) implements the descriptive and inferential statistics that sit conceptually one level below everything here.

The three-way boundary this node sits inside (read this once, carefully). Three modules touch "regression," and each owns a genuinely different question about it, so that none of them re-teaches the other:

Cross-link every derivation back to its owner; this node re-derives no theory from either QM module, it builds the code that operationalizes QM1.03's concepts and complements, never replaces, QM1.02's inferential lens (C-3 binding boundary).