Probability rules

Start with a sample space (the full list of things that could happen) and an event, some subset of it you care about (a bond defaults; a stock closes up on the day; a screen flags a filing). Probability assigns each event a number between 0 and 1, with the whole sample space getting exactly 1, and two housekeeping facts follow immediately: the complement rule, P(not A) = 1 − P(A), and the fact that probabilities of a full, non-overlapping partition of outcomes sum to 1.

What is the addition rule?

The addition rule answers "what is the probability that A or B happens?" In its general form,

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

• you subtract the overlap once, because a simple sum would double-count the outcomes where both happen. If A and B are mutually exclusive (they cannot both happen, a bond either defaults this year or it doesn't, not both), P(A ∩ B) = 0 and the rule collapses to the simple sum P(A) + P(B). This is the single most confused pairing in the whole subject, so fix it now, hard: mutually exclusive is not the same idea as independent, and the two are close to opposites. If A and B are mutually exclusive and both have positive probability, they are automatically dependent, learning that A happened tells you with certainty that B did not. Independence is the opposite kind of relationship: learning that A happened tells you nothing at all about B. Keep this distinction sharp; it is the first entry in Common Mistakes because it causes real errors, not just vocabulary slips.

What is conditional probability?

Conditional probability (the probability of A given that B has already happened) is defined as

P(A | B) = P(A ∩ B) / P(B),   for P(B) > 0

Rearranged, this is the multiplication rule: P(A ∩ B) = P(A | B) × P(B), the tool for building up a joint probability from a chain of conditional steps (a company has a going-concern qualification, and then, given that, it defaults within a year. Practice Problem 2 drills exactly this). Independence is defined formally by this rule collapsing: A and B are independent precisely when P(A ∩ B) = P(A) × P(B), equivalently when P(A | B) = P(A), knowing B occurred does not move your probability for A at all. Finally, the total probability rule: if a set of events B₁, B₂, …, Bₖ partitions the sample space (they are mutually exclusive and cover every possibility), then for any event A,

P(A) = P(A | B₁)P(B₁) + P(A | B₂)P(B₂) + … + P(A | Bₖ)P(Bₖ)

This rule is not decorative, it is the exact engine that computes the denominator in Bayes' theorem, and you will use it in every worked Bayes calculation in this module.