Introduction to Statistical Methods

Probability Axioms, Conditional Probability, and Total Probability

📅Published:2026-06-30

🎓Level:postgraduate

👥Audience:Postgraduate students in Statistical Methods

Prerequisite Knowledge

This lecture builds on the following concepts from earlier lectures. If any feel unfamiliar, review the linked notes before proceeding.

Previously Covered in This Subject

.1 Definition and Intuition— covered in Lecture 1
.1 Understanding Quartiles— covered in Lecture 2
.2 Formal Definition— covered in Lecture 1
.2 The Five-Point Summary— covered in Lecture 2
.3 Interquartile Range (IQR)— covered in Lecture 2

Probability Axioms, Conditional Probability, and Total Probability

What if I told you that most everyday reasoning rules are actually formal mathematical axioms? For example: "if A and B can't happen together, their combined chance is just the sum." And a single formula,

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}

, powers everything from Amazon recommendations to Google ad targeting. This lecture builds the foundation: the three axioms of probability, conditional probability, and the total probability theorem for pooling evidence from every possible scenario.

3.1 Review of Probability Fundamentals

Hook.A weather app says "80% chance of rain." What does that actually mean? Does it mean 80% of the sky will be covered? That it will rain 80% of the day? No — it means that under similar atmospheric conditions, rain occurs 8 times out of 10. Probability is a language for uncertainty, and this section builds its grammar from three unprovable but universally accepted rules.

3.1.1 Basic Definition and Notation

Intuition.Imagine a bag with 5 red marbles and 3 blue marbles. You close your eyes and pick one. The probability of picking red is 5/8 — you count what you want (5 red) and divide by everything possible (8 marbles). This is the simplest picture of probability:favorable divided by total.

Aprobability(the chance an event happens), written

P (A)

, is the ratio of favorable outcomes to all possible outcomes in thesample space

S

(the set of everything that could happen):

P (A) = \frac{Number of favorable outcomes for A}{Total number of outcomes in sample space S}

Thesample space $S$ is the set of all possible outcomes of an experiment.
The numerator counts only outcomes where event $A$ occurs — thefavorable cases.
If you flip a coin, $S = {Heads, Tails}$ . For event $A = Heads$ , $P (A) = \frac{1}{2}$ .

Every probability is a number in $[0, 1]$ .If your calculation gives a negative number or a number bigger than 1, you made a mistake — no exceptions.

3.1.2 The Three Axioms of Probability

Intuition.Anaxiomis a statement we accept without proof — like agreeing that

0! = 1

before doing any factorial arithmetic. We don't prove it; we build everything else on top of it. Probability rests on three such axioms. They are the foundation; every formula in this course traces back to them.

Think of axioms like the rules of chess. You don't prove that a bishop moves diagonally — you just accept it. Then every strategy, every opening, every endgame builds on that rule. Same with probability.

Axiom 1 — Sample space certainty:

P (S) = 1

Something must happen. The probability of the entire sample space is 1 (100%). When you roll a die, you're 100% sure some face will show up.

Axiom 2 — Complement rule:

P (A) + P (A^{c}) = 1

The probability an event happens plus the probability it doesnothappen is always 100%.

A^{c}

(also written

A ˉ

) is thecomplementof

A

— everything in

S

that is not

A

. If the chance of rain is 80%, the chance of no rain is 20%.Axiom 3 — Addition rule for mutually exclusive events:If events

A

and

B

aremutually exclusive(they cannot happen together), then:

P (A \cup B) = P (A) + P (B)

A \cap B = \emptyset

(empty set), or equivalently

P (A \cap B) = 0

. The two events share no outcomes. If you roll a die, getting a 1 and getting a 6 are mutually exclusive. — you can't get both on one roll, so

P (1 \cup 6) = \frac{1}{6} + \frac{1}{6} = \frac{1}{3}

Scope: Professor's axioms vs. standard axioms.The professor lists complement rule as Axiom 2. Standard textbooks list the axioms differently. Axiom 1 is

0 \leq P (A) \leq 1

. Axiom 2 is

P (S) = 1

. Axiom 3 is the addition rule for mutually exclusive events. The complement rule is aconsequenceof Axioms 1–3 in the standard treatment. For this course, use the professor's numbering — it's what the exam will test. Know that both formulations are equivalent; they just organize the same ground rules differently.

3.1.3 Mutually Exclusive Events vs. Independent Events

Intuition. Mutually exclusivemeans "these two things can't happen at the same time" — like being in Mumbai and Delhi simultaneously.Independentmeans "knowing about one tells you nothing about the other" — like the outcome of a coin flip in Mumbai. And a die roll in Delhi. These are completely different ideas, tested by completely different equations.Analogy:Two doors can't both be fully open at the same time — that's mutually exclusive. But whether your neighbor eats cereal for breakfast has no effect on your lunch choice — that's independence.

Property	Mutually Exclusive	Independent
Meaning	Events cannot happen together	One event does not influence the other
Condition	$P (A \cap B) = 0$	$P (A \cap B) = P (A) \cdot P (B)$
Set interpretation	$A \cap B = \emptyset$	No set restriction
Can they co-exist?	Only if at least one has probability 0	Yes, commonly

Joint probability:

P (A \cap B)

— the probability both

A

and

B

happen — is thejoint probability. In AI/ML notation:

P (A, B)

P (A B)

. All three mean the same thing: intersection.Naive assumption:In ML contexts,naivemeansindependent. "Naive Bayes" = "independent features, Bayes theorem." The naive assumption says: features don't influence each other.

Pitfall: Never judge independence by intuition alone.You must test the mathematical condition: compute

P (A \cap B)

and compare it to

P (A) \cdot P (B)

. They are equal → independent. They differ → dependent. Domain knowledge ("these should be related") is a hypothesis — the numbers confirm or reject it.Pitfall: Mutually exclusive ≠ independent.In fact, mutually exclusive events with non-zero probability are the mostdependentevents possible — knowing one happened tells you the other definitely did NOT happen.

3.1.4 The General Addition Rule

For any two events

A

and

B

(not necessarily mutually exclusive):

P (A \cup B) = P (A) + P (B) - P (A \cap B)

Why subtract?The overlap

A \cap B

is counted twice — once in

P (A)

and once in

P (B)

. Subtracting it once fixes the double-count. When

A

and

B

are mutually exclusive,

P (A \cap B) = 0

and the formula collapses to Axiom 3:

P (A \cup B) = P (A) + P (B)

Worked Example — Drawing a Card.From a standard deck of 52 cards, what is the probability of drawing a heart or a queen?

$P (Heart) = \frac{13}{52}$
$P (Queen) = \frac{4}{52}$
$P (Heart \cap Queen) = \frac{1}{52}$ (the Queen of Hearts)

P (Heart \cup Queen) = \frac{13}{52} + \frac{4}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13}

Sense-check: 16 cards are either hearts or queens (13 hearts + 3 other queens) out of 52 = 16/52 = 4/13. ✓

Recap.Probability measures chance on a 0-to-1 scale. Three axioms anchor it. Mutually exclusive events use straight addition; overlapping events need the general addition rule with subtraction. Independence is a special multiplicative relationship — don't confuse it with mutual exclusivity.Bridge.Now we put these rules to work on a real problem: a student facing two exams. Section 3.2 shows how the general addition rule unlocks

P (A \cap B)

when you're given

P (A)

P (B)

, and

P (A \cup B)

Real-World & Domain Connection.The general addition rule drivesinclusion-exclusioncounting in computer science,union-boundarguments in learning theory, anderror probabilitycalculations in reliability engineering. For example, a spacecraft with two redundant navigation systems: the probability at least one works is

P (A \cup B) = P (A) + P (B) - P (A \cap B)

. If each system fails independently with probability 0.01, both fail with probability

0.0 1^{2} = 0.0001

. So at least one works with probability 0.9999. This is why engineers design redundant systems.

3.2 Worked Example — Student Exam Probabilities

Hook.You have three numbers: pass rate for Statistics, fail rate for Mathematics, and the probability of passing at least one. Can you figure out the probability of passingboth— without any additional data? Yes. The general addition rule is a Swiss Army knife: given any three of the four quantities

P (A), P (B), P (A \cup B), P (A \cap B)

, you can solve for the fourth.

3.2.1 Problem Setup

Intuition.Think of two overlapping circles: "Passes Stats" and "Passes Math." We know how big each circle is. And we know how big the combined shape (union) is. The missing piece is the overlap — students who pass both.

A student faces two examinations: Statistics and Mathematics.

Probability of passing Statistics: $P (A) = \frac{2}{3}$
Probability of NOT passing Mathematics: $P (B^{c}) = \frac{5}{9}$
Probability of passing at least one examination: $P (A \cup B) = \frac{4}{5}$

Where:

$A$ = event "passes Statistics"
$B$ = event "passes Mathematics"

First, convert the given "not passing" probability:

P (B) = 1 - P (B^{c}) = 1 - \frac{5}{9} = \frac{4}{9}

3.2.2 Finding the Probability of Passing Both

"At least one" means

A \cup B

— the student passes Statistics, or Mathematics, or both. We need

P (A \cap B)

— passing both. Rearranging the general addition rule:

P (A \cap B) = P (A) + P (B) - P (A \cup B)

Substitute the known values:

P (A \cap B) = \frac{2}{3} + \frac{4}{9} - \frac{4}{5}

Convert to a common denominator (45):

P (A \cap B) = \frac{30}{45} + \frac{20}{45} - \frac{36}{45} = \frac{14}{45}

3.2.3 Validating the Result

Always check: is the probability between 0 and 1?

\frac{14}{45} \approx 0.311

Yes, this is valid. If you had mistakenly computed something like

\frac{5}{3}

(greater than 1), that would signal an error.

3.2.4 Checking for Independence

Test the condition

P (A \cap B) = P (A) \cdot P (B)

P (A) \cdot P (B) = \frac{2}{3} \cdot \frac{4}{9} = \frac{8}{27} \approx 0.296

P (A \cap B) = \frac{14}{45} \approx 0.311

Since

\frac{8}{27} \neq = \frac{14}{45}

, the events arenot independent. The events are dependent.

Derivation — full algebra for the independence check.Let's verify the inequality rigorously:

\frac{8}{27} = ? \frac{14}{45}

Cross-multiply:

8 \times 45 = 360

vs.

14 \times 27 = 378

. Since

360 \neq = 378

, the fractions differ. The events are dependent.

3.2.5 When Assumptions Are Needed

Scope.In this problem, all three probabilities were given, so no assumptions were needed. In real-world scenarios, you may not have complete information. When a key probability (like

P (A \cap B)

) is missing, you may need to assume independence to proceed.Always state the assumption explicitlywhen you make it. The appropriateness of that assumption depends on the domain context.

3.2.6 Alternative Notations for Intersection

In AI/ML courses, the intersection

A \cap B

is commonly written as:

$P (A, B)$ — read as "P of A comma B"
$P (A B)$ — read as "P of A B"

All three notations mean the same thing: the joint probability that both A and B occur. When you see these forms in ML papers or textbooks, interpret them as intersection.

Recap.The general addition rule is symmetric: given any three of

P (A), P (B), P (A \cup B), P (A \cap B)

, you can always find the fourth. Always check your answer is between 0 and 1. Always test independence numerically — don't guess.Bridge.Section 3.3 shows the same pattern applied to investment data. The numbers change; the logic does not.

3.3 Worked Example — Investment Choices

Hook.85% of investors put money in at least one of two options: stocks or annuities. But how many invest exclusively in stocks? Exclusively in annuities? In both? The pattern from Section 3.2 repeats — the general addition rule is your universal tool for these questions.

3.3.1 Problem Statement

A survey finds:

85% of investors put money in at least one of: stock market or traditional annuities
Let $A$ = invests in stock market, $B$ = invests in traditional annuities
$P (A \cup B) = 0.85$

"At least one" again means union: the investor is in

A

, or in

B

, or in both.

The principle for solving such problems is identical to the student exam example: use the general addition rule. And the relationships among

P (A)

P (B)

P (A \cup B)

, and

P (A \cap B)

. Given any three of these four quantities, the fourth is determined:

P (A \cap B) = P (A) + P (B) - P (A \cup B)

Extending the example with made-up numbers.Suppose we additionally know:

$P (A) = 0.60$ (60% invest in stocks)
$P (B) = 0.45$ (45% invest in annuities)

Then the overlap is:

P (A \cap B) = 0.60 + 0.45 - 0.85 = 0.20

So 20% invest in both. The "stock only" investors:

P (A) - P (A \cap B) = 0.60 - 0.20 = 0.40

(40%). The "annuity only" investors:

P (B) - P (A \cap B) = 0.45 - 0.20 = 0.25

(25%).

Sense-check: 40% (stocks only) + 25% (annuities only) + 20% (both) = 85% (at least one). ✓

Recap.The same formula, different domain. The general addition rule is a template — plug in any three of the four quantities and solve. This pattern of solving for the missing piece will recur throughout the course.Bridge.Now we shift from real-world surveys to the pure randomness of dice. Section 3.4 introduces an important subtlety: when events are defined by "maximum" and "minimum," the outcome counting requires careful thought.

3.4 Worked Example — Two Dice with Maximum and Minimum Conditions

Hook.Two dice. One event says "the maximum face value is 2." Another says "the minimum face value is 2." Are these events independent? Your intuition might say "probably not" — but probability demands a numerical test. And to run that test, you first need to count outcomes correctly. This example teaches a lesson: event definitions involving "max" and "min" are trickier than they look.

3.4.1 Problem Setup

Intuition.Imagine rolling two dice and only caring about thebiggestnumber showing. If you see (1,2), the max is 2. If you see (2,2), the max is also 2. But if you see (1,1), the max is 1 — even though both dice are small numbers, the max condition "is exactly 2" fails. Because 2 never appeared.

Two dice are thrown. Define:

Event $A$ : themaximumface value is 2
Event $B$ : theminimumface value is 2

Total possible outcomes:

6 \times 6 = 36

3.4.2 Determining the Outcomes

Event $A$ (max is 2):For the maximum to be exactly 2, every die must show at most 2, and at least one die must show exactly 2. The outcomes are:

A = {(1, 2), (2, 1), (2, 2)}

P (A) = \frac{3}{36} = \frac{1}{12}

Key reasoning from discussion:For "max is 2," the face value 2 must appear. An outcome like $(1, 1)$ has max = 1, not 2, so it does not belong to $A$ . The number 2 must be present in the outcome.

Event $B$ (min is 2):For the minimum to be exactly 2, every die must show at least 2, and at least one die must show exactly 2. The full enumeration is:

B = {(2, 2), (2, 3), (2, 4), (2, 5), (2, 6), (3, 2), (4, 2), (5, 2), (6, 2)}

P (B) = \frac{9}{36} = \frac{1}{4}

Math reconciliation note.The session recording notes

P (B) = \frac{3}{36}

, which appears to be a transcription error. The standard enumeration for "min is 2" with two dice yields9 outcomes(any pair where the smaller number is exactly 2). If the professor intended a different definition (e.g., "both dice show exactly 2"), that would give just

(2, 2)

. — one outcome, not three. With 9 outcomes,

P (B) = \frac{9}{36} = \frac{1}{4}

.Both interpretations lead to the same conclusion (events are dependent); only the numbers differ. For exam purposes, follow whatever definition the professor provides in slides. If only the 3/36 value is given in slides, use that — but know the standard enumeration.

Intersection $A \cap B$ :The only outcome satisfying both "max is 2" and "min is 2" is

(2, 2)

P (A \cap B) = \frac{1}{36}

3.4.3 Testing Independence

Check whether

P (A \cap B) = P (A) \cdot P (B)

:Using the standard enumeration(

P (B) = \frac{9}{36}

P (A) \cdot P (B) = \frac{3}{36} \cdot \frac{9}{36} = \frac{27}{1296} = \frac{3}{144} = \frac{1}{48}

P (A \cap B) = \frac{1}{36}

Since

\frac{1}{36} \neq = \frac{1}{48}

, the events arenot independent. They are dependent.Using the session\u2019s recorded value(

P (B) = \frac{3}{36}

P (A) \cdot P (B) = \frac{3}{36} \cdot \frac{3}{36} = \frac{9}{1296} = \frac{1}{144}

P (A \cap B) = \frac{1}{36}

Again,

\frac{1}{36} \neq = \frac{1}{144}

—same conclusion. The events are dependent regardless of which count you use.

3.4.4 Student Question — Clarifying "Max is 2"

Q:For event

A

(maximum is 2), why can we not include

(1, 1)

?A:The maximum of

(1, 1)

is 1, not 2. The event "maximum is 2" requires that 2 appears on at least one die and that no die shows a number greater than 2. In

(1, 1)

, the value 2 is absent entirely — so this outcome does not satisfy the condition. Think of it this way: if I say "the tallest person in the room is 6 feet," then someone must actually be 6 feet tall in the room.

3.4.5 Joint Probability and the Term "Joint"

The termjoint probability(used for

P (A \cap B)

) captures the idea of "what is common to both events." It is the probability that both

A

. And

B

happen together. This terminology becomes important in later topics involving joint distributions and conditional probability.

Recap."Max is 2" and "min is 2" are dependent events — knowing the max restricts what the min can be. And vice versa. Counting outcomes for max/min conditions requires checking all ordered pairs systematically. The independence test is always numerical: compute both sides of

P (A \cap B) = P (A) \cdot P (B)

and compare.Bridge.The term "joint probability" is more than vocabulary — it's the numerator in the conditional probability formula we build next. Section 3.5 motivates why conditional probability matters.

3.5 Conditional Probability — Introduction

Hook.A supermarket knows 80% of shoppers buy both bread and butter. Should they place butter next to the bread, or bread next to the butter? The joint probability (80%) doesn't answer this. You need to know: among bread-buyers, how many also buy butter? And vice versa. These two questions have different answers — and different business implications. This is conditional probability.

3.5.1 Motivation — Market Basket Analysis

Market Basket Analysisis a classic data mining model that studies purchasing patterns. The canonical example is bread and butter: customers often buy both together. But the business question is deeper:what is the pattern?

Do customers first pick bread, then butter? → $P (butter ∣ bread)$
Or do they first pick butter, then bread? → $P (bread ∣ butter)$

If you only know that 80% of baskets contain both, you cannot act on that. But if you know that 60% of bread-buyers also buy butter, while only 20% of butter-buyers also buy bread, you know which product drives the other. This is what conditional probability measures.

3.5.2 Motivation — Digital Marketing Attribution

A company runs digital marketing campaigns across multiple platforms. Suppose:

1000 people visit the website
300 people buy the product (become customers)

The company wants to know:which platform is most effective at converting visitors to customers?

Consider two platforms:

LinkedIn: 500 visitors → 100 reach the website → some become customers
Reels/Instagram: 300 visitors → 50 reach the website → some become customers

The raw count (100 vs. 50) does not tell the full story. The conversion rate — 100/500 vs. 50/300 — reveals which platform performs better. This is a conditional probability:probability of becoming a customer given that the visitor came from LinkedIn.

This is the essence ofattribution modeling— understanding which channel or touchpoint contributes most to the final conversion.

3.5.3 What Is a Conditional Event?

Intuition.Imagine you're at a party. The probability a random person speaks French might be 10%. But if I tell you the person is from Paris, that probability jumps to nearly 100%. The condition "from Paris" shrinks your world — you stop considering all 8 billion humans and focus only on Parisians. Conditional probability is zooming in.

Aconditional eventis an event whose probability depends on another event having already occurred. The key phrase is "already happened."

$P (B ∣ A)$ : "probability of $B$ given $A$ " — $A$ has already happened; now we want the probability of $B$
$P (A ∣ B)$ : "probability of $A$ given $B$ " — $B$ has already happened; now we want the probability of $A$

The event that happened firstreduces the sample space. We no longer consider all possible outcomes — only those consistent with the condition.

Pronunciation guidance:Pronounce

P (B ∣ A)

as "B given A," not "B by A" or "B followed by A." The word "given" makes it clear that

A

is the condition, not the consequence. Saying "B followed by A" incorrectly suggests

B

happens first.

3.5.4 Real-World Example — Medical Diagnosis (BP and Diabetes)

A common observation in medical practice: after age 40, many people have both hypertension (high BP) and diabetes. Public health authorities want to know:

$P (diabetes ∣ BP)$ : Among people with BP, what fraction develop diabetes?
$P (BP ∣ diabetes)$ : Among people with diabetes, what fraction develop BP?

Numerical illustration.Suppose in a population of 100,000:

20,000 have hypertension (BP)
15,000 have diabetes
9,000 have both

Then:

$P (diabetes ∣ BP) = \frac{9 , 000}{20 , 000} = 0.45$ — 45% of BP patients also have diabetes
$P (BP ∣ diabetes) = \frac{9 , 000}{15 , 000} = 0.60$ — 60% of diabetes patients also have BP

These are different numbers! The condition matters. If data shows 60% of BP patients develop diabetes, but only 40% of diabetes patients develop BP, that pattern suggests BP tends to precede diabetes. This guides public health messaging — controlling BP first may reduce diabetes incidence.

Note:These are conditional probabilities reflecting observed patterns. They describe association, not necessarily causation. But they provide the evidence base for public health decisions.

3.5.5 Identifying Conditional Probability in Problems

How to tell whether a problem requires conditional probability or simple probability:

Simple (unconditional) probability:The question has no "given that" clause. Example: "What is the probability a randomly selected person is a loan defaulter?" → Use total sample space.
Conditional probability:The question contains a condition. Example: "What is the probability a person will not default on the loangiven thathe or she is middle-aged?" → The sample space shrinks to middle-aged people only.

The phrase "given that" (or equivalent wording like "among," "if we know that," "of those who") is the trigger to use conditional probability.

Recap.Conditional probability answers "what's the chance of X, now that we know Y happened?" It shrinks the world to only outcomes consistent with Y. The notation

P (A ∣ B)

reads "A given B" — B is the condition, A is the event of interest. Market basket analysis, attribution modeling, and medical epidemiology all use this idea.Bridge.Section 3.6 gives the formula that makes this precise:

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}

. The intuition is simple — you're taking the overlap and dividing by the new, smaller world.

3.6 Conditional Probability — Formula and Derivation

Hook.You know 30% of your class passed both Statistics and Math, and 60% passed Math. Among Math-passers, what fraction also passed Statistics? The answer is 30/60 = 50%. You just computed a conditional probability — without realizing there's a formula for it.

3.6.1 The Formula

Intuition.Picture a dartboard split into regions. Event

B

is the bullseye ring. Event

A

is the left half.

P (A ∣ B)

asks: if I tell you the dart hit the bullseye ring, how likely is it also in the left half? You ignore the whole board and look only at the bullseye ring — then ask what fraction of that ring is also in the left half.

For two events

A

and

B

with

P (B) > 0

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}

Denominator $P (B)$ :The probability of the condition — the event that has already happened. This becomes the reduced sample space.
Numerator $P (A \cap B)$ :The probability that both $A$ and $B$ occur — the overlap between the event of interest and the condition.

Similarly:

P (B ∣ A) = \frac{P ( A \cap B )}{P ( A )}, P (A) > 0

Requirement:The conditioning event must have non-zero probability. If

P (B) = 0

P (A ∣ B)

is undefined — you can't condition on something that never happens.

3.6.2 Venn Diagram Explanation

Consider a sample space

S

with two overlapping events

A

and

B

.Visual Intuition — the shrinking-world picture.Draw a rectangle (the sample space

S

). Inside, draw two overlapping circles labeled

A

and

B

. The overlap region is

A \cap B

.For $P (B ∣ A)$ :Circle

A

lights up — it becomes the entire world. Everything outside

A

disappears from consideration. Inside this lit-up circle

A

, the part that also belongs to

B

is the overlap

A \cap B

. The conditional probability is the fraction of the lit-up world occupied by the overlap:

P (B ∣ A) = \frac{area of A \cap B}{area of A} = \frac{P ( A \cap B )}{P ( A )}

For $P (A ∣ B)$ :Circle

B

lights up. The overlap inside

B

that is also

A

is again

A \cap B

P (A ∣ B) = \frac{area of A \cap B}{area of B} = \frac{P ( A \cap B )}{P ( B )}

Thenumerator is always the intersection— the part common to both events. Thedenominator is always the condition— the event that has already occurred.

3.6.3 Concrete Illustration — Numbers on a Die

Consider throwing a fair die.

Event $A$ : the number isless than or equal to 5( $A = {1, 2, 3, 4, 5}$ , $P (A) = \frac{5}{6}$ )
Event $B$ : the number iseven( $B = {2, 4, 6}$ , $P (B) = \frac{3}{6} = \frac{1}{2}$ )
Intersection $A \cap B = {2, 4}$ , $P (A \cap B) = \frac{2}{6} = \frac{1}{3}$

Finding $P (B ∣ A)$ — even given ≤ 5:

The condition ( $A$ ) restricts us to ${1, 2, 3, 4, 5}$ — five equally likely outcomes
From these five, the even numbers are ${2, 4}$ — two outcomes
$P (B ∣ A) = \frac{2}{5} = 0.40$

Using the formula:

P (B ∣ A) = \frac{P ( A \cap B )}{P ( A )} = \frac{2/6}{5/6} = \frac{2}{5}

. ✓Finding $P (A ∣ B)$ — ≤ 5 given even:

The condition ( $B$ ) restricts us to ${2, 4, 6}$ — three equally likely outcomes
From these three, numbers ≤ 5 are ${2, 4}$ — two outcomes
$P (A ∣ B) = \frac{2}{3} \approx 0.667$

Using the formula:

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )} = \frac{2/6}{3/6} = \frac{2}{3}

. ✓Key observation:

P (B ∣ A) = \frac{2}{5} \neq = P (A ∣ B) = \frac{2}{3}

. The order of conditioning matters — swapping the condition changes the answer. This is because

A

and

B

are dependent (knowing the number is even changes the chance it's ≤ 5).

Pitfall:

P (A ∣ B)

is NOT the same as

P (B ∣ A)

. These are different probabilities with different meanings. The only exception is when

P (A) = P (B)

and the events are symmetric, or when events are independent (Section 3.8). Never swap them casually.Pitfall:The denominator can never be the event of interest.

P (A ∣ B)

has

B

in the denominator. Because

B

is the given condition — the world we're zooming into. If you find yourself writing

\frac{P ( A \cap B )}{P ( A )}

for

P (A ∣ B)

, stop — that's

P (B ∣ A)

Recap.

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}

. Numerator = overlap, denominator = condition (the reduced world). The formula is the same regardless of which version you use; what changes is which event plays the role of the condition.Bridge.From this formula, we can derive a powerful tool: the multiplication rule. Multiply both sides by

P (B)

and you get

P (A \cap B) = P (B) \cdot P (A ∣ B)

. Section 3.7 explores this.

3.7 The Multiplication Rule

Hook.You're at a junction. One-third of all drivers turn left. Among those who turn left, half then turn right. Among those who go left-then-right, 90% reach the destination. What fraction of all drivers reach the destination? You need the multiplication rule: multiply the probabilities along the path.

3.7.1 Two-Event Form

Intuition.The conditional probability formula

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}

tells you the overlap as a fraction of

B

. Multiply both sides by

P (B)

and you get

P (A \cap B) = P (B) \cdot P (A ∣ B)

. This is the multiplication rule — it reconstructs the joint probability from a marginal and a conditional. Think of it as: "first

B

happens (with probability

P (B)

), then, given

B

A

happens (with probability

P (A ∣ B)

)."

From the definition of conditional probability, multiply both sides by the denominator:

P (A \cap B) = P (A) \cdot P (B ∣ A)

Equivalently:

P (A \cap B) = P (B) \cdot P (A ∣ B)

This is called themultiplication rule. It expresses the joint probability as the product of a marginal probability and a conditional probability.

Derivation:

P (A ∣ B) P (B) \cdot P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )} = P (A \cap B) (definition of conditional probability) (multiply both sides by P (B))

3.7.2 Multi-Event Extension

The rule extends to any number of events. The order of conditioning follows the sequence in which events happen:

P (A_{1} \cap A_{2} \cap \dots \cap A_{n}) = P (A_{1}) \cdot P (A_{2} ∣ A_{1}) \cdot P (A_{3} ∣ A_{1} \cap A_{2}) \cdot \dots \cdot P (A_{n} ∣ A_{1} \cap A_{2} \cap \dots \cap A_{n - 1})

This is sometimes called thechain rule of probability. Each factor conditions on everything that came before.

Interpretation by sequence:

$P (A_{1})$ — probability the first event happens
$P (A_{2} ∣ A_{1})$ — probability the second event happens, given the first already happened
$P (A_{3} ∣ A_{1} \cap A_{2})$ — probability the third event happens, given the first two already happened
And so on...

The conditioning set grows as more events have occurred.

Example with three events (C happens first, then B, then A):

P (C \cap B \cap A) = P (C) \cdot P (B ∣ C) \cdot P (A ∣ B \cap C)

Concrete numbers.Suppose:

$P (C) = 0.5$ — 50% chance event C happens
$P (B ∣ C) = 0.4$ — given C happened, 40% chance B happens
$P (A ∣ B \cap C) = 0.75$ — given both B and C happened, 75% chance A happens

Then:

P (C \cap B \cap A) = 0.5 \times 0.4 \times 0.75 = 0.15

The joint probability of all three happening is15%.

Out of 1000 trials, C happens ~500 times. Of those 500, B happens ~200 times. Of those 200, A happens ~150 times. 150/1000 = 15%. ✓

Pitfall:The multiplication rule is NOT

P (A \cap B) = P (A) \cdot P (B)

unless

A

and

B

are independent. The general form uses conditional probability:

P (A \cap B) = P (A) \cdot P (B ∣ A)

. Using the product

P (A) \cdot P (B)

for dependent events gives the wrong answer.Pitfall:In the multi-event chain, the order of conditioning must match the actual sequence. You can't write

P (A ∣ C)

when

B

happened between

C

and

A

— the conditioning set must include everything that happened before.

Recap.The multiplication rule

P (A \cap B) = P (A) \cdot P (B ∣ A)

is the conditional formula rearranged. For

n

events, chain the conditionals: each event's probability is conditioned on everything that happened before it.Bridge.What if

A

and

B

are independent? Then

P (B ∣ A) = P (B)

, and the multiplication rule collapses to

P (A \cap B) = P (A) \cdot P (B)

. Section 3.8 explores this special case.

Real-World & Domain Connection.The chain rule is the backbone ofautoregressive modelsin NLP (like GPT). To generate a sentence, these models compute

P (w_{1}, w_{2}, \dots, w_{n}) = P (w_{1}) \cdot P (w_{2} ∣ w_{1}) \cdot P (w_{3} ∣ w_{1}, w_{2}) \dots

. — each word's probability is conditioned on all previous words. The multiplication rule, extended to thousands of events, is what makes modern language models work.

3.8 Conditional Probability When Events Are Independent

Hook.If I tell you "the coin landed heads," does that change the probability the die shows a 6? No. The coin and die are independent — knowing one tells you nothing about the other. In probability notation:

P (die = 6 ∣ coin = H) = P (die = 6) = \frac{1}{6}

. The condition is irrelevant.

3.8.1 The Simplification

Intuition.Independent events are like strangers passing on a street — neither affects the other. Mathematically, independence means the conditional probability equals the unconditional probability. The formula

P (A ∣ B) = P (A)

says: "even after learning

B

happened, my belief in

A

hasn't changed."

A

and

B

are independent, then

P (A \cap B) = P (A) \cdot P (B)

. Substituting into the conditional probability formula:

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )} = \frac{P ( A ) \cdot P ( B )}{P ( B )} = P (A)

Similarly:

P (B ∣ A) = P (B)

3.8.2 Meaning

When events are independent, knowing that one event occurred givesno informationabout the other. The conditional probability equals the unconditional (marginal) probability. The condition is irrelevant — it does not shift the probability at all.

Pitfall — the asymmetry trap.When

A

and

B

are dependent,

P (A ∣ B) \neq = P (B ∣ A)

(as the die example showed:

\frac{2}{5} \neq = \frac{2}{3}

). But when they are independent, BOTH conditional probabilities equal their respective marginals:

P (A ∣ B) = P (A)

AND

P (B ∣ A) = P (B)

. Independence is symmetric.

3.8.3 Student Question

Q:If

A

and

B

are not independent, what is the formula for

P (A \cap B)

?A:There is no special simplified formula. Independence is a special case that gives the product form

P (A) \cdot P (B)

. When events are dependent, you must use the general form:

P (A \cap B) = P (A) \cdot P (B ∣ A) = P (B) \cdot P (A ∣ B)

. There is no shortcut — you need the conditional probability.

Recap.Independence means conditioning doesn't change the probability:

P (A ∣ B) = P (A)

. This is equivalent to the product rule

P (A \cap B) = P (A) \cdot P (B)

. When independence fails, you must use the full multiplication rule with conditional probability.Bridge.Armed with conditional probability, we now tackle a rich real-world dataset: loan applicants categorized by age and default status. Section 3.9 shows how to read a contingency table, compute conditional probabilities from counts, and test independence.

3.9 Worked Example — Loan Default Contingency Table

Hook.A bank has data on 46,687 loan applicants. They know each person's age group and whether they defaulted. The raw numbers are overwhelming — but conditional probability turns them into actionable insights. What's the default rate among middle-aged applicants? And is age group independent of default behavior? The contingency table holds the answers.

3.9.1 The Data

Intuition.Acontingency tableis a grid that cross-tabulates two variables. Each cell counts how many people fall into both categories. The margins (row and column totals) give the counts for each category alone. From this grid, every probability — simple, joint, conditional — can be computed.

A dataset contains information about loan applicants:

	Loan Default = Yes	Loan Default = No	Total
Young	—	—	—
Middle-aged	4,851	27,368	32,219
Old	—	—	—
Total	8,557	38,130	46,687

Additional values mentioned:

Total population: 46,687
Total loan defaulters (Yes): 8,557
Total not defaulted (No): 38,130
Total angry individuals: 14,089
Total middle-aged: 32,219
Middle-aged AND loan default = No: 27,368
Middle-aged AND loan default = Yes: 4,851

3.9.2 Simple (Unconditional) Probabilities

Probability a randomly selected person is a loan defaulter:

P (Default = Yes) = \frac{8 , 557}{46 , 687} \approx 0.183

Probability a randomly selected person is angry:

P (Angry) = \frac{14 , 089}{46 , 687} \approx 0.302

These use the full sample space because there is no condition.

3.9.3 Conditional Probabilities

Probability of NOT defaulting given middle-aged:

The condition "middle-aged" reduces the sample space to 32,219 people. Among them, 27,368 did not default.

P (Default = No ∣ Middle-aged) = \frac{27 , 368}{32 , 219} \approx 0.849

Numerator: middle-aged AND not defaulted. Denominator: all middle-aged.

Probability of being middle-aged given NOT defaulted:

The condition "not defaulted" reduces the sample space to 38,130 people. Among them, 27,368 are middle-aged.

P (Middle-aged ∣ Default = No) = \frac{27 , 368}{38 , 130} \approx 0.718

Pitfall — swapped conditions.Notice

P (No Default ∣ Middle-aged) \approx 0.849

but

P (Middle-aged ∣ No Default) \approx 0.718

. These are different questions with different answers. The first asks "among the middle-aged, how many don't default?" The second asks "among non-defaulters, how many are middle-aged?" Always check: which group is the "given". — the one whose total goes in the denominator.

3.9.4 Venn Diagram Representation

Visual Intuition.Draw two overlapping circles in a large rectangle (the sample space of 46,687 people):

Circle $A$ : Middle-aged (total 32,219)
Circle $B$ : Not defaulted on loan (total 38,130)

The overlap

A \cap B

= 27,368 (middle-aged AND not defaulted). The part of

A

outside the overlap (middle-aged AND defaulted) = 32,219 − 27,368 = 4,851. The part of

B

outside the overlap (not middle-aged AND not defaulted) = 38,130 − 27,368 = 10,762.

The remaining region (neither middle-aged nor non-defaulter) = total − (4,851 + 27,368 + 10,762) = 46,687 − 42,981 = 3,706. These would be young/old people who defaulted.

3.9.5 Domain Knowledge vs. Mathematical Validation

The question arose: are age and loan default independent?

Withdomain knowledge, you might suspect dependence. Older people may have different default patterns than younger people. But in probability theory, youmust validateusing the mathematical condition:

Independent if P (A \cap B) = P (A) \cdot P (B)

Testing independence for middle-aged vs. not defaulted:

P (Middle-aged) \cdot P (Not Defaulted) = \frac{32 , 219}{46 , 687} \cdot \frac{38 , 130}{46 , 687} \approx 0.6901 \times 0.8167 \approx 0.5636

P (Middle-aged \cap Not Defaulted) = \frac{27 , 368}{46 , 687} \approx 0.5862

Since

0.5862 \neq = 0.5636

, the events aredependent. Domain intuition is confirmed by the numbers — but the numbers, not the intuition, are the final authority.

You cannot conclude independence or dependence from domain intuition alone. Even when you suspect dependence, compute both sides of the condition to confirm numerically. Domain knowledge informs your hypothesis; the mathematical condition confirms or rejects it.

3.9.6 Student Discussion — Filling the Venn Diagram

Q:In the Venn diagram for middle-aged and loan default, how do we compute the numbers for each region?A:Start with the overlap: the number that satisfies both conditions (27,368 for middle-aged AND not defaulted). Then subtract that overlap from each total to get the "only A" and "only B" regions. The remaining part of the sample space is everything else. This subtraction method works when you have totals and the intersection count. It is the same logic as the general addition rule:

P (A only) = P (A) - P (A \cap B)

Recap.A contingency table is probability in grid form. Row and column totals give unconditional probabilities. Cell counts divided by row/column totals give conditional probabilities. Venn diagrams make the relationships visual. Always test independence numerically — domain knowledge suggests, math confirms.Bridge.What if one event is entirely inside another? Section 3.10 explores conditional probability when events have subset relationships — a special case that simplifies the math dramatically.

Real-World & Domain Connection.Contingency tables are the workhorse ofcredit risk modeling. Banks compute

P (Default ∣ Age group)

P (Default ∣ Income bracket)

, and

P (Default ∣ Credit history)

to build scorecards. These conditional probabilities feed into logistic regression models that assign every loan applicant a default risk score. The 46,687-row table in this example is a miniature version of what real banks process daily with millions of records.

3.10 Conditional Probability with Subset Relationships

Hook.If every dog is a mammal, and I tell you "this animal is a dog," what's the probability it's a mammal? 100%. That's a subset relationship: dogs

\subseteq

mammals. When one event is entirely inside another, conditional probability simplifies dramatically.

3.10.1 When B Is a Subset of A

Intuition.Imagine a dartboard where the bullseye (

B

) is completely inside the red ring (

A

). If I tell you the dart hit the bullseye, it definitely hit the red ring too —

P (A ∣ B) = 1

. But if I tell you it hit the red ring, the chance it's also the bullseye is small. — the bullseye is only a fraction of the red area:

P (B ∣ A) = \frac{P ( B )}{P ( A )}

B \subseteq A

(

B

is entirely contained within

A

), then every outcome in

B

is also in

A

. Therefore:

A \cap B = B

This simplifies conditional probability:

P (B ∣ A) = \frac{P ( A \cap B )}{P ( A )} = \frac{P ( B )}{P ( A )}

Also:

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )} = \frac{P ( B )}{P ( B )} = 1

When

B

is a subset of

A

, knowing

B

happenedguarantees

A

happened — hence

P (A ∣ B) = 1

3.10.2 Using Venn Diagrams to Derive Complement Intersections

For computing probabilities involving complements, use the Venn diagram:

$P (A \cap B^{c})$ — $A$ but not $B$ :This is the part of

A

that lies outside

B

. From the Venn diagram:

P (A \cap B^{c}) = P (A) - P (A \cap B)

Take the total probability of

A

. And subtract the overlap with

B

. $P (A^{c} \cap B)$ — $B$ but not $A$ :

Similarly:

P (B \cap A^{c}) = P (B) - P (A \cap B)

Important:

P (A \cap B^{c}) \neq = P (A^{c} \cap B)

in general. They are different regions of the Venn diagram and are not equivalent. The first is "

A

only," the second is "

B

only."

Concrete example.

P (A) = 0.6

P (B) = 0.4

P (A \cap B) = 0.2

$P (A \cap B^{c}) = P (A) - P (A \cap B) = 0.6 - 0.2 = 0.40$ — "A only"
$P (A^{c} \cap B) = P (B) - P (A \cap B) = 0.4 - 0.2 = 0.20$ — "B only"

They are indeed different (0.40 ≠ 0.20).

3.10.3 When Information Is Insufficient

Scope.If the given data provides

P (A)

and

P (A \cap B)

but not

P (B)

P (B^{c})

, can you find

P (A \cap B^{c})

? Yes — use

P (A) - P (A \cap B)

as shown above. But what if you need

P (A^{c} \cap B)

and only

P (A)

and

P (A \cap B)

are given? You cannot compute it without

P (B)

. In such cases, you may need to make an assumption (e.g., independence) to proceed. The Venn diagram tells you what is derivable and what requires additional information.

3.10.4 Limitation — Two Events Only

These Venn diagram derivations assume the world consists of only two events of interest (plus their complements). If a third event

C

exists that is not captured in the data, the Venn diagram reasoning with only

A

. And

B

is incomplete. The information you have determines what you can compute.

Q:"What if there is a third event not in the data?"A:You can only work with the information available. If the data does not include a third event, it is outside the scope of the probability discussion. In practice, you work with the events for which data exists. This is analogous to predicting who will be the next Prime Minister — you work with the known contenders, not hypothetical candidates for whom no polling data exists.

Recap.Subset relationships simplify conditional probability: if

B \subseteq A

, then

P (A ∣ B) = 1

and

P (B ∣ A) = \frac{P ( B )}{P ( A )}

. For complements:

P (A \cap B^{c}) = P (A) - P (A \cap B)

. The Venn diagram is your visual calculator — use it.Bridge.Section 3.11 puts all these tools together in a worked example that asks: "Given a person has at least one credit card, what's the probability they have a Visa?" The condition itself is a union. — a twist that tests your event-identification skills.

3.11 Worked Example — At Least One Card

Hook."Given that a person has at least one credit card, what's the probability they have a Visa?" The condition itself. — "at least one card" — is a union of events. This problem tests whether you can identify events correctly when the condition is a compound event.

3.11.1 Problem

A

= has a Visa card,

B

= has a MasterCard. Find:

P (has Visa ∣ has at least one card)

3.11.2 Identifying the Events

"At least one card" means

A \cup B

— the person has Visa, or MasterCard, or both. This is the condition (the event that has already happened). "Has a Visa card" is the event of interest, which is

A

So we need:

P (A ∣ A \cup B)

3.11.3 Solving

P (A ∣ A \cup B) = \frac{P ( A \cap ( A \cup B ))}{P ( A \cup B )}

Now,

A \cap (A \cup B)

. — the intersection of

A

with the union — is simply

A

. Because

A

is entirely contained within

A \cup B

(everything in

A

is also in the union). This is the subset property from Section 3.10: since

A \subseteq (A \cup B)

, we have

A \cap (A \cup B) = A

Therefore:

P (A ∣ A \cup B) = \frac{P ( A )}{P ( A \cup B )}

And:

P (A \cup B) = P (A) + P (B) - P (A \cap B)

So the final formula is:

P (A ∣ A \cup B) = \frac{P ( A )}{P ( A ) + P ( B ) - P ( A \cap B )}

Concrete numbers.Suppose

P (A) = 0.40

P (B) = 0.30

P (A \cap B) = 0.10

P (A ∣ A \cup B) = \frac{0.40}{0.40 + 0.30 - 0.10} = \frac{0.40}{0.60} = \frac{2}{3} \approx 0.667

Among people who have at least one card, roughly 67% have a Visa.

Sense-check: 60% of the population has at least one card. Of those, 40% have Visa. So 40/60 = 2/3. ✓

3.11.4 Strategy for Conditional Probability Problems

Identify whether the problem is conditional.Look for "given that," "among," "of those," "if we know."
Identify which event is the condition (first) and which is the event of interest (second).The condition goes in the denominator.
Write the formula: $P (interest ∣ condition) = \frac{P ( interest \cap condition )}{P ( condition )}$ .
Check if the needed probabilities are directly given.If yes, substitute and compute.
If not directly given, use relationships:the addition rule, Venn diagram derivations, complement rules, or the multiplication rule to compute the missing pieces.
If still stuck, consider whether an independence assumption is justified.

Bridge.This six-step strategy is your algorithm for every conditional probability problem in the course. The next section introduces a theorem that handles the case where the condition can arise from multiple mutually exclusive causes. — the total probability theorem.

3.12 Total Probability Theorem

Hook.You have three email accounts. Each has its own spam rate. But what's the overall probability that a random email in your unified inbox is spam? You can't just average the three spam rates — the accounts receive different volumes of email. You need a weighted average, where each account's spam rate is weighted by how much email it receives. That's the total probability theorem.

3.12.1 The Conditions

Intuition.Imagine partitioning a pizza into slices. Every bite you take belongs to exactly one slice. The slices aremutually exclusive(no bite belongs to two slices) andexhaustive(every bite belongs to some slice). Total probability works the same way: you split the sample space into non-overlapping pieces that together cover everything.

The total probability theorem applies when you have a set of events

A_{1}, A_{2}, \dots, A_{k}

that are:

Mutually exclusive: $P (A_{i} \cap A_{j}) = 0$ for all $i \neq = j$ . No two can happen together.
Exhaustive: $A_{1} \cup A_{2} \cup \dots \cup A_{k} = S$ , the entire sample space. Equivalently, $P (A_{1}) + P (A_{2}) + \dots + P (A_{k}) = 1$ .

These arehard conditions— both must be satisfied for the theorem to apply.

3.12.2 The Theorem Statement

A_{1}, A_{2}, \dots, A_{k}

are mutually exclusive and exhaustive, then for any event

B

P (B) = i = 1 \sum k P (A_{i}) \cdot P (B ∣ A_{i})

The total probability of

B

is the weighted average of the conditional probabilities of

B

given each

A_{i}

, weighted by the probability of each

A_{i}

.Why it works:Every outcome in

B

must belong to exactly one

A_{i}

(by exhaustiveness and mutual exclusivity). So

B = (A_{1} \cap B) \cup (A_{2} \cap B) \cup \dots \cup (A_{k} \cap B)

, where these pieces are disjoint. Using the addition rule and the multiplication rule:

P (B) = P (A_{1} \cap B) + P (A_{2} \cap B) + \dots + P (A_{k} \cap B) = P (A_{1}) \cdot P (B ∣ A_{1}) + \dots + P (A_{k}) \cdot P (B ∣ A_{k}) (addition rule for disjoint events) (multiplication rule)

3.12.3 Worked Example — Email Spam

You sync three email accounts to your phone:

$A_{1}$ : Office mail — receives 60% of all emails ( $P (A_{1}) = 0.60$ ). Spam rate: 2% ( $P (Spam ∣ A_{1}) = 0.02$ )
$A_{2}$ : Bits (educational institution) mail — receives 20% of all emails ( $P (A_{2}) = 0.20$ ). Spam rate: 5% ( $P (Spam ∣ A_{2}) = 0.05$ )
$A_{3}$ : Personal Gmail — receives 20% of all emails ( $P (A_{3}) = 0.20$ ). Spam rate: 1% ( $P (Spam ∣ A_{3}) = 0.01$ )

These three accounts are mutually exclusive (an email goes to exactly one account) and exhaustive (60% + 20% + 20% = 100% of your emails).

What is the total probability that a randomly selected email is spam?

P (Spam) = P (A_{1}) \cdot P (Spam ∣ A_{1}) + P (A_{2}) \cdot P (Spam ∣ A_{2}) + P (A_{3}) \cdot P (Spam ∣ A_{3}) = (0.60) (0.02) + (0.20) (0.05) + (0.20) (0.01) = 0.012 + 0.010 + 0.002 = 0.024

The total spam probability is2.4%.

Sense-check: The overall spam rate (2.4%) is between the lowest (1%) and highest (5%) individual rates. It's pulled toward 2% because the office account (60% of email) has a 2% spam rate. ✓

3.12.4 Worked Example — CEO Succession

Three contenders for CEO:

$A_{1}$ : 30% chance of becoming CEO ( $P (A_{1}) = 0.30$ )
$A_{2}$ : 30% chance ( $P (A_{2}) = 0.30$ )
$A_{3}$ : 40% chance ( $P (A_{3}) = 0.40$ )

These are exhaustive (30% + 30% + 40% = 100%) and mutually exclusive (only one person becomes CEO).

Concrete numbers.Suppose:

$P (Benefits ∣ A_{1}) = 0.80$ — $A_{1}$ is very employee-friendly
$P (Benefits ∣ A_{2}) = 0.50$ — $A_{2}$ is neutral
$P (Benefits ∣ A_{3}) = 0.20$ — $A_{3}$ is unlikely to provide good benefits

P (Benefits) = (0.30) (0.80) + (0.30) (0.50) + (0.40) (0.20) = 0.24 + 0.15 + 0.08 = 0.47

There's a 47% chance of good benefits overall.

3.12.5 Understanding "Exhaustive"

Visual Intuition.Draw a large rectangle (the sample space

S

). Divide it into several non-overlapping regions labeled

A_{1}, A_{2}, A_{3}, A_{4}, A_{5}

. If the regions together fill the entire rectangle with no gaps, they are exhaustive. If there is white space not covered by any region, they are not exhaustive — there exists an outcome belonging to none of them. The test:

P (A_{1}) + P (A_{2}) + \dots + P (A_{k}) = 1

. If the sum is less than 1, the events are not exhaustive — there are other possibilities not accounted for.

3.12.6 Intuition — Aggregating All Past Experiences

The total probability theorem formalizes aggregating information from all possible scenarios. Think of an athlete preparing for an Olympic final. She reviews all past matches against the opponent: 10 previous encounters, of which she won 8 and lost 2. She also considers specific conditions — matches in China, matches after breaks, matches under pressure. By pooling all this conditional information (performance under each scenario weighted by how often that scenario occurs), she forms a total probability estimate of winning. This aggregation of evidence is what the total probability theorem captures mathematically.

Pitfall — forgetting exhaustiveness.If

P (A_{1}) + \dots + P (A_{k}) < 1

, the total probability formula gives the wrong answer. There are outcomes not covered by any

A_{i}

that could produce

B

, and you're ignoring them. Always verify both conditions before using the theorem. Pitfall — using non-mutually-exclusive events.If

A_{1}

and

A_{2}

overlap, the formula double-counts contributions from the overlap region. The events must partition the sample space cleanly, with no overlaps.

Recap.The total probability theorem computes

P (B)

as a weighted sum of conditional probabilities:

P (B) = \sum_{i} P (A_{i}) \cdot P (B ∣ A_{i})

. The

A_{i}

must be mutually exclusive and exhaustive — they must partition the sample space. Think of it as "pooling evidence from every possible scenario."Bridge.Total probability answers "what's the overall chance of

B

?" The natural next question is: "Given that

B

happened. , which

A_{i}

was the most likely cause?" That's Bayes theorem — the topic of Section 3.13.

Real-World & Domain Connection.Total probability powersensemble methodsin ML. A random forest averages predictions from many decision trees, each trained on a different data subset. The final prediction is a weighted vote — exactly the total probability pattern. In medical testing, total probability computes the overall disease prevalence by pooling age-group-specific rates weighted by population proportions. In reliability engineering, it computes overall system failure probability by summing failure rates of each component weighted by its usage frequency.

3.13 Preview — Bayes Theorem

Hook.You're a doctor. A patient tests positive for a disease that affects 1% of the population. The test is 95% accurate. What's the chance the patient actually has the disease? Hint: it's not 95%. Bayes theorem reveals why — and the answer might surprise you.

3.13.1 What Bayes Theorem Does

Intuition.Total probability answers "what's the overall chance of seeing this evidence?" Bayes theorem answers the inverse: "given that we saw this evidence. , which cause is most likely?" It's like a detective who, upon finding a fingerprint at a crime scene, updates their belief about each suspect. The detective doesn't ask "what's the chance of finding this fingerprint?" — they ask "given this fingerprint, who did it?"

Bayes theorem "reverses" conditional probabilities. Given

P (B ∣ A)

, you can compute

P (A ∣ B)

— provided you also know the individual probabilities

P (A)

and

P (B)

The theorem connects:

The total probability of $B$ (computed via the total probability theorem as denominator)
The conditional probabilities $P (B ∣ A_{i})$ (the "forward" probabilities)
The "reverse" conditional probabilities $P (A_{i} ∣ B)$ (what we want)

The formula (two-event case):

P (A ∣ B) = \frac{P ( A ) \cdot P ( B ∣ A )}{P ( B )}

The general form (k mutually exclusive, exhaustive events):

P (A_{j} ∣ B) = \frac{P ( A _{j} ) \cdot P ( B ∣ A _{j} )}{\sum _{i = 1}^{k} P ( A _{i} ) \cdot P ( B ∣ A _{i} )}, j = 1, \dots, k

Denominator = total probability of

B

. Numerator = contribution of

A_{j}

to that total.

3.13.2 The Hard Conditions

Bayes theorem inherits the same conditions as the total probability theorem:

The events $A_{1}, A_{2}, \dots, A_{k}$ must bemutually exclusive.
They must beexhaustive.

If either condition fails, Bayes theorem does not apply in its standard form.

3.13.3 The "Snake" Nickname

Bayes theorem was informally called the "snake theorem" in this session — an informal way to signal that it is tricky. And requires careful handling. The full treatment of Bayes theorem will be covered in the next session.

3.13.4 Connection to Machine Learning

The total probability and Bayes theorem together form the foundation of a class of ML algorithms. The structure uses mutually exclusive and exhaustive hypotheses, each with a prior probability. Evidence updates these probabilities through conditional formulas. This is the blueprint for the Naive Bayes classifier, covered later in the course.

Quick preview of Naive Bayes:

$P (A_{i})$ =prior— how likely each class is before seeing data
$P (B ∣ A_{i})$ =likelihood— how likely the observed features are, given the class
$P (A_{i} ∣ B)$ =posterior— updated belief about each class after seeing the evidence

The algorithm classifies by picking the class with the highest posterior probability.

Recap.Bayes theorem flips conditional probabilities:

P (A ∣ B) = \frac{P ( A ) \cdot P ( B ∣ A )}{P ( B )}

. It needs mutually exclusive, exhaustive hypotheses and uses total probability in the denominator. The "snake" nickname warns that it's subtle — watch your conditions.Bridge.Section 3.14 consolidates everything with a seven-step strategy for solving any conditional probability problem. The full treatment of Bayes theorem awaits in the next lecture.

3.14 Conditional Probability — Summary of Problem-Solving Strategy

The 7-step algorithm for any conditional probability problem:

Classify the problem:Is it a simple (unconditional) probability or a conditional probability? The word "given" (or equivalent phrasing) is the trigger.
Identify the sequence:Which event happened first (the condition)? Which event's probability we want (the interest)? The condition goes in the denominator.
Write the formula: $P (interest ∣ condition) = \frac{P ( interest \cap condition )}{P ( condition )}$ .
Check what is given:Are all needed probabilities directly available?
If not, derive:Use the addition rule, complement rule, Venn diagram relationships, or the multiplication rule.
If stuck, consider assumptions:In some real-world scenarios, you may need to assume independence to proceed. State the assumption explicitly.
Validate:Every computed probability must lie between 0 and 1. If not, something is wrong.

Toolkit reference — formulas at your disposal:

Addition rule: $P (A \cup B) = P (A) + P (B) - P (A \cap B)$
Complement rule: $P (A^{c}) = 1 - P (A)$
Conditional probability: $P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}$
Multiplication rule: $P (A \cap B) = P (A) \cdot P (B ∣ A)$
Independence test: $P (A \cap B) = ? P (A) \cdot P (B)$
Total probability: $P (B) = \sum_{i} P (A_{i}) \cdot P (B ∣ A_{i})$ (requires mutually exclusive, exhaustive $A_{i}$ )
Venn diagram derivation: $P (A \cap B^{c}) = P (A) - P (A \cap B)$

Bridge.This concludes the core of Lecture 3. The appendices that follow summarize exam guidance and key industry applications — keep them as your quick-reference cards.

Exam Guidance Summary

Calculator policy:Calculator is allowed in the exam. The exam application also provides a built-in calculator.Study priority:First, complete all slides and exercise problems given in the slides. Then, if time permits, go through the first textbook.Question types:Expect both conceptual questions (identifying events, stating conditions, interpreting results) and numerical problems (computing probabilities, applying formulas, filling contingency tables).

Key formulas to memorize:

Addition rule: $P (A \cup B) = P (A) + P (B) - P (A \cap B)$
Conditional probability: $P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}$
Multiplication rule: $P (A \cap B) = P (A) \cdot P (B ∣ A)$
Independence condition: $P (A \cap B) = P (A) \cdot P (B)$
Total probability: $P (B) = \sum_{i} P (A_{i}) \cdot P (B ∣ A_{i})$ (requires mutually exclusive and exhaustive $A_{i}$ )
Mutually exclusive condition: $P (A \cap B) = 0$

Exam note:These six formulas are the core toolkit. Write them on your scratch paper at the start of the exam so you don't blank under pressure.

Common mistake areas:

Confusing mutually exclusive with independent — they are tested by different conditions
Misidentifying the condition in a conditional probability problem (which event is "given")
Forgetting that total probability requires BOTH mutually exclusive AND exhaustive conditions
Computing the complement intersection incorrectly — use the Venn diagram, not guesswork

Exam note:The professor explicitly flagged these four traps. Expect at least one exam question designed to catch each one.

Validation habit:Always check that your computed probability is between 0 and 1. If you get a value outside this range, re-check your calculation. This 2-second check catches more errors than any other single technique.

Key Industry Applications

Market Basket Analysis:Classic data mining model for retail — understanding which products drive the purchase of other products. Uses conditional probability to identify purchase patterns (e.g., bread → butter vs. butter → bread). Retailers use this to design store layouts (place butter near bread if

P (butter ∣ bread)

is high) and targeted promotions.Attribution Modeling:Digital marketing — determining which channel (LinkedIn, Instagram, email, referral, etc.) is most effective at converting visitors to customers. Uses conditional probability:

P (conversion ∣ channel)

. The total probability theorem aggregates across all channels to compute overall conversion rate; Bayes theorem then attributes each conversion back to the most likely originating channel.Medical Epidemiology:Studying comorbidity patterns — e.g., whether hypertension tends to precede diabetes or vice versa. Conditional probabilities from patient records guide public health interventions. The total probability theorem computes overall disease burden by pooling age-group-specific rates; Bayes theorem computes the probability a patient has a disease given a positive test result.Credit Risk / Loan Default Analysis:Financial institutions use conditional probabilities to assess default risk. They look at demographic or behavioral characteristics like age group, income bracket, or credit history. Contingency tables (Section 3.9) are the industry-standard format for this analysis. The independence test

P (A \cap B) = P (A) \cdot P (B)

checks whether features like age and default are related. If they are, both must be included in risk models.

ISM Lecture 3 notes · Probability Axioms, Conditional Probability, and Total Probability

Introduction to Statistical Methods· postgraduate· 2026-06-30

Sections Breakdown

13.1 Review of Probability Fundamentals

3.1 Review of Probability Fundamentals

23.2 Worked Example — Student Exam Probabilities

3.2 Worked Example — Student Exam Probabilities

33.3 Worked Example — Investment Choices

3.3 Worked Example — Investment Choices

43.4 Worked Example — Two Dice with Maximum and Minimum Conditions

3.4 Worked Example — Two Dice with Maximum and Minimum Conditions

53.5 Conditional Probability — Introduction

3.5 Conditional Probability — Introduction

63.6 Conditional Probability — Formula and Derivation

3.6 Conditional Probability — Formula and Derivation

73.7 The Multiplication Rule

3.7 The Multiplication Rule

83.8 Conditional Probability When Events Are Independent

3.8 Conditional Probability When Events Are Independent

93.9 Worked Example — Loan Default Contingency Table

3.9 Worked Example — Loan Default Contingency Table

103.10 Conditional Probability with Subset Relationships

3.10 Conditional Probability with Subset Relationships

113.11 Worked Example — At Least One Card

3.11 Worked Example — At Least One Card

123.12 Total Probability Theorem

3.12 Total Probability Theorem

133.13 Preview — Bayes Theorem

3.13 Preview — Bayes Theorem

143.14 Conditional Probability — Summary of Problem-Solving Strategy

3.14 Conditional Probability — Summary of Problem-Solving Strategy

15Exam Guidance Summary

Exam Guidance Summary

16Key Industry Applications

Key Industry Applications

Postgraduate students in Statistical Methods

Exam Revision Notes

Below is the distilled, exam-ready core of this lecture. Every entry is built from the full textbook notes above. Use this section for rapid review — but if something doesn't make sense, go back to the full explanation in the main content.

Probability Axioms

Must-know:Probability is always in $[0, 1]$ . Three axioms: $P (S) = 1$ , $P (A) + P (A^{c}) = 1$ , and for mutually exclusive events $P (A \cup B) = P (A) + P (B)$ . Every formula in this course traces back to these.

$P (S) = 1 P (A) + P (A^{c}) = 1 P (A \cup B) = P (A) + P (B) e x t i f A \cap B = \emptyset$

⚠️ Top pitfall:Confusing the complement rule as a separate axiom vs. a consequence. Use the professor’s numbering for the exam.

Self-check:If $P (A) = 0.3$ , what is $P (A^{c})$ ?

Connects to:General addition rule, conditional probability.

General Addition Rule

Must-know:For any two events, $P (A \cup B) = P (A) + P (B) - P (A \cap B)$ . Subtract the overlap once to avoid double-counting. Given any three of the four quantities, solve for the fourth.

$P (A \cup B) = P (A) + P (B) - P (A \cap B)$

⚠️ Top pitfall:Forgetting to subtract the intersection when events overlap. Verify whether events are mutually exclusive first.

Self-check: $P (A) = 0.6, P (B) = 0.3, P (A \cap B) = 0.1$ . What is $P (A \cup B)$ ?

Connects to:Mutually exclusive events, conditional probability, Venn diagrams.

Mutually Exclusive vs. Independent Events

Must-know:Mutually exclusive ( $P (A \cap B) = 0$ ) is NOT the same as independent ( $P (A \cap B) = P (A) \cdot P (B)$ ). In fact, non-zero mutually exclusive events are maximally dependent.

$e x t M E : P (A \cap B) = 0 e x t I n d : P (A \cap B) = P (A) \cdot P (B)$

⚠️ Top pitfall:Assuming mutually exclusive events are independent. Never judge independence by intuition alone — test numerically.

Self-check:Can two events be both mutually exclusive and independent?

Connects to:Addition rule, multiplication rule, conditional probability.

Conditional Probability

Must-know: $P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}$ . The condition reduces the sample space. Numerator = overlap, denominator = condition. $P (A ∣ B) e q P (B ∣ A)$ in general.

$P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}, P (B) > 0$

⚠️ Top pitfall:Swapping the condition — $P (A ∣ B)$ and $P (B ∣ A)$ have different meanings and different values. The condition always goes in the denominator.

Self-check:If $P (A \cap B) = 0.2$ and $P (B) = 0.5$ , what is $P (A ∣ B)$ ?

Connects to:Multiplication rule, Bayes theorem, contingency tables.

Multiplication Rule

Must-know: $P (A \cap B) = P (A) \cdot P (B ∣ A)$ . The chain rule extends to $n$ events. This is NOT $P (A) \cdot P (B)$ unless events are independent.

$P (A \cap B) = P (A) \cdot P (B ∣ A) = P (B) \cdot P (A ∣ B)$

⚠️ Top pitfall:Using $P (A) \cdot P (B)$ for dependent events. Always check independence before using the product form.

Self-check: $P (A) = 0.4, P (B ∣ A) = 0.3$ . What is $P (A \cap B)$ ?

Connects to:Conditional probability, independence, chain rule (language models).

Independence and Conditional Probability

Must-know:If $A$ and $B$ are independent, $P (A ∣ B) = P (A)$ and $P (B ∣ A) = P (B)$ . Knowing one gives no information about the other.

$P (A ∣ B) = P (A) ⟺ P (A \cap B) = P (A) \cdot P (B) ⟺ e x t in d e p e n d e n t$

⚠️ Top pitfall:The asymmetry trap — dependence implies $P (A ∣ B) e q P (B ∣ A)$ ; independence is symmetric and both equal their marginals.

Self-check:Can dependent events have $P (A ∣ B) = P (A)$ ?

Connects to:Conditional probability, multiplication rule, Naive Bayes.

Total Probability Theorem

Must-know: $P (B) = \sum_{i} P (A_{i}) \cdot P (B ∣ A_{i})$ . Requires mutually exclusive AND exhaustive $A_{i}$ . Computes overall probability by pooling evidence from all scenarios.

$P (B) = i = 1 \sum k P (A_{i}) \cdot P (B ∣ A_{i})$

⚠️ Top pitfall:Forgetting exhaustiveness — if the $A_{i}$ don’t cover the whole sample space, the formula gives the wrong answer. Always verify $P (A_{1}) + \dots + P (A_{k}) = 1$ .

Self-check:Three mutually exclusive events have probabilities 0.2, 0.3, 0.5. Conditional probabilities of B given each are 0.1, 0.2, 0.3. What is P(B)?

Connects to:Bayes theorem, ensemble methods, medical testing.

Bayes Theorem (Preview)

Must-know:Bayes theorem reverses conditional probabilities: $P (A ∣ B) = \frac{P ( A ) \cdot P ( B ∣ A )}{P ( B )}$ . The denominator is the total probability of $B$ . Requires mutually exclusive, exhaustive hypotheses.

$P (A_{j} ∣ B) = \frac{P ( A _{j} ) \cdot P ( B ∣ A _{j} )}{\sum _{i} P ( A _{i} ) \cdot P ( B ∣ A _{i} )}$

⚠️ Top pitfall:Applying Bayes theorem without verifying the mutually exclusive and exhaustive conditions — inherited from total probability theorem.

Self-check:A disease affects 1% of the population. A test is 95% accurate. If a person tests positive, what is P(disease)? (Hint: use Bayes theorem.)

Connects to:Total probability theorem, conditional probability, Naive Bayes classifier.

Previous LectureLecture 02 · Measures of Variability, Dispersion, and Introduction to Probability

Next LectureLecture 04 · Bayes Theorem and Naive Bayes Classifier

Was this lecture useful?

Loading comments…

🤖

BitsNotes AI Assistant

Subject Notes Assistant

Configure AI Chat

Choose how to access the chatbot

Have your own API key?

Switch to "Bring Your Own Key" tab above for unlimited access with any OpenAI-compatible provider.

AI Provider Preset

API Endpoint URL (OpenAI-Compatible)

API Key

Model Name

🔑 Enter API key above to fetch live models from provider, or enter model name manually.

Remember key for this session (until tab closes)

OpenAI-Compatible API Support

Choose any provider preset (Gemini, DeepSeek, Kimi, GLM, MiniMax, Qwen, OpenAI, Groq, Ollama, etc.) or enter a custom endpoint URL.

Security & Privacy First

Your API key is sent directly from your browser to your specified provider. BitsNotes servers never store or see your key.