Glossary
Calibration
M9.04Also called forecast calibration, probability calibration.
Whether stated probabilities match observed frequencies. A well-calibrated forecaster is right about 70% of the time when they say 70%.
It is measurable, it is trainable, and it is rare. Most people are overconfident at high probabilities and underconfident at low ones.
The training method is simple and tedious: make many predictions with explicit probabilities, score them, and look at the pattern.
Calibration is the difference between an opinion and a forecast.