Glossarytaught in DA2.02, QD1.03
Overfitting
Also called curve fitting, data mining bias.
Building a model that describes the noise in a sample rather than the signal, so it performs well historically and poorly afterwards.
Its tell is complexity that pays off only on the training data: many parameters, thresholds tuned to specific values, rules with exceptions.
Defend against it with a held-out period and a preference for simpler specifications. Also keep a full count of how many variations were tried before the good one appeared.
The more strategies tested, the more likely the best one is luck.