The Analyst's Path

Glossary

Calibration

M9.04

Also called forecast calibration, probability calibration.

Whether stated probabilities match observed frequencies. A well-calibrated forecaster is right about 70% of the time when they say 70%.

It is measurable, it is trainable, and it is rare. Most people are overconfident at high probabilities and underconfident at low ones.

The training method is simple and tedious: make many predictions with explicit probabilities, score them, and look at the pattern.

Calibration is the difference between an opinion and a forecast.