Glossarytaught in M9.04
Calibration
Also called forecast calibration, probability calibration.
Whether stated probabilities match observed frequencies. A well-calibrated forecaster is right about 70% of the time when they say 70%.
It's measurable, it's trainable, and it's rare. Most people are overconfident at high probabilities and underconfident at low ones.
Training it's simple and tedious. Make many predictions with explicit probabilities and score them, then look at the pattern.
Calibration is the difference between an opinion and a forecast.