Introduction to Statistical Methods

Joint, Marginal, and Conditional Distributions — Discrete and Named Distributions

📅 Published: 2026-07-02

🎓 Level: postgraduate

👥 Audience: Postgraduate students in Statistical Methods

Prerequisite Knowledge

This lecture builds on the following concepts from earlier lectures. If any feel unfamiliar, review the linked notes before proceeding.

Previously Covered in This Subject

Joint Probability Distributions — covered in Lecture 6 (Random Variables — Discrete and Continuous Distributions)
Marginal Probability Distributions — covered in Lecture 6
Random Variables — covered in Lectures 5 and 6
Conditional Probability — covered in Lecture 3 (Probability Axioms, Conditional Probability, and Total Probability)
Bayes Theorem and Naive Bayes — covered in Lectures 4 and 5

Joint, Marginal, and Conditional Distributions — Discrete and Named Distributions

7.1 Recap — From One Random Variable to Two

Hook. You already know how to describe the randomness of one quantity — say, tomorrow's temperature. But what if you need to describe two quantities together — temperature AND humidity — and how they move relative to each other? A single distribution cannot capture their relationship.

7.1.1 The Progression

Intuition + Analogy. Think of a single random variable as a recipe for one dish. A joint distribution is a recipe for a two-course meal — it tells you not just what each course could be, but which combinations are likely. You might often serve soup with salad but rarely soup with ice cream. The joint distribution captures those pairings. Where the analogy breaks: In cooking, pairings are cultural preferences. In probability, pairings are governed by strict mathematical rules (non-negativity, total = 1).

The study began with a single random variable. The discussion then extended to two random variables. For each case there are two families:

Discrete: the probability mass function (PMF), written $P (X = x)$ .
Continuous: the probability density function (PDF), written $f_{X} (x)$ .

When two random variables are involved the same split applies — both discrete, both continuous, or one of each (though the focus stays on two variables, not three or more).

7.1.2 Validation Rules

Every probability distribution must satisfy two conditions:

Non-negativity: For discrete, $0 \leq P (X = x) \leq 1$ . For continuous, $f (x) \geq 0$ .
Total probability equals one:

Discrete single variable: $\sum_{x} P (X = x) = 1$
Continuous single variable: $\int_{- \infty}^{\infty} f (x) d x = 1$
Discrete two variables: $\sum_{x} \sum_{y} P (X = x, Y = y) = 1$
Continuous two variables: $\int_{- \infty}^{\infty} \int_{- \infty}^{\infty} f (x, y) d x d y = 1$

If either condition fails, the function is not a valid probability distribution.

7.1.3 Joint Distributions

When two variables appear together the distribution is called the joint distribution:

Discrete: Joint PMF — $P (X = x, Y = y)$ . The comma means "and" (intersection). This is a function $P_{X Y} : R^{2} \to [0, 1]$ .
Continuous: Joint PDF — $f_{X Y} (x, y)$ . This is a function $f_{X Y} : R^{2} \to [0, \infty)$ .

The joint distribution gives the probability pattern for both variables together. It contains all the information about how X and Y relate — including whether they are independent.

7.1.4 Marginal Distributions

From the joint distribution one can extract the distribution of a single variable alone. This is the marginal distribution — so named because it historically appeared in the margins of a joint probability table.

Discrete marginal of X: $P (X = x) = \sum_{y} P (X = x, Y = y)$ — sum over all values of Y.
Discrete marginal of Y: $P (Y = y) = \sum_{x} P (X = x, Y = y)$ — sum over all values of X.
Continuous marginal of X: $f_{X} (x) = \int_{- \infty}^{\infty} f_{X Y} (x, y) d y$ — integrate out Y.
Continuous marginal of Y: $f_{Y} (y) = \int_{- \infty}^{\infty} f_{X Y} (x, y) d x$ — integrate out X.

Professor intuition: When integrating with respect to Y, treat X as a constant — just like in partial differentiation where you hold one variable fixed. The other variable stands outside the integral like a spectator. To get the marginal of X, make Y "disappear" by summing (discrete) or integrating (continuous) over all its values.

Scope / Assumptions. Marginal distributions only make sense when the joint distribution is properly defined and valid (sums/integrates to 1). If the joint is malformed, the marginals derived from it may also be invalid. Also, the marginal of X discards all information about its relationship with Y — you cannot recover the joint from the marginals alone (unless the variables are independent).

Visual Intuition. Imagine a 3D surface plot of a joint density

f_{X Y} (x, y)

over the

x y

-plane. The marginal

f_{X} (x)

is the profile you see when you squish the entire surface flat onto the

x

-axis — you collapse the

y

dimension by integrating along it. What remains is a 2D curve showing how probability mass is distributed over X alone. The same collapse onto the

y

-axis gives

f_{Y} (y)

Pitfalls.

Confusing "marginal" with "conditional." Marginal = ignore the other variable entirely. Conditional = fix the other variable at a specific value. They are different operations with different results.
Forgetting bounds on integrals. When integrating to get a marginal, always use the full support of the variable being integrated out. If $Y \in [0, 1]$ , integrate from 0 to 1 — not from $- \infty$ to $\infty$ (though it gives the same result since density is zero outside $[0, 1]$ ).
Thinking marginals determine the joint. Two different joint distributions can produce the same marginals. The joint contains strictly more information.

Recap + Bridge. The joint distribution is the full story — it tells you everything about X and Y together. The marginal is a summary — it tells you about X alone, hiding Y. Next, we extend the idea of conditional probability from events to random variables, giving us the third piece: conditional distributions.

Real-World & Domain Connection. Joint and marginal distributions are the mathematical backbone of risk modeling. An insurance company might model the joint distribution of a driver's age (X) and accident count (Y). The marginal of Y alone tells the company expected claim frequency across all drivers, while the full joint lets them price policies differently for young vs. old drivers. In machine learning, joint distributions underlie generative models (like Naive Bayes) that learn

P (X, Y)

to make predictions via

P (Y ∣ X)

7.2 Conditional Probability Extended to Random Variables

Hook. You already know that "probability of rain given dark clouds" is different from "probability of rain." Now replace "rain" and "dark clouds" with random variables X and Y. Can we compute "the distribution of Y assuming we already know X = 3"?

7.2.1 From Events to Random Variables

Intuition + Analogy. Imagine a deck of cards. The joint distribution tells you the probability of every (suit, rank) pair. The marginal tells you the probability of each suit, ignoring rank. Now the conditional: if I tell you "the card is a heart" (X = hearts), what is the new probability distribution over ranks? You zoom in on one row of the joint table and re-normalize so the row sums to 1. That re-normalization — dividing by the row total — is exactly the marginal in the denominator. Where the analogy breaks: In cards, the conditional is intuitive because we know the deck composition. For continuous variables, there is no "row" to zoom into — we work with density functions and ratio formulas instead.

Recall conditional probability for events:

$P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}, P (B ∣ A) = \frac{P ( A \cap B )}{P ( A )}$

The same idea extends to random variables:

Discrete: $P (Y = y ∣ X = x) = \frac{P ( X = x , Y = y )}{P ( X = x )}$
Numerator: joint PMF.
Denominator: marginal of X ( $\neq = 0$ , otherwise the conditional is undefined).

Continuous: $f_{Y ∣ X} (y ∣ x) = \frac{f _{X Y} ( x , y )}{f _{X} ( x )}$
Numerator: joint PDF.
Denominator: marginal density of X ( $\neq = 0$ ).

The pattern is always: joint divided by marginal.

In the continuous case, $f_{Y ∣ X} (y ∣ x)$ is itself a valid probability density function in $y$ (for a fixed $x$ ). It satisfies: $\int_{- \infty}^{\infty} f_{Y ∣ X} (y ∣ x) d y = 1$ because $\int f_{X Y} (x, y) d y = f_{X} (x)$ , and dividing by $f_{X} (x)$ brings the integral back to 1.

Visual Intuition. Picture the joint density

f_{X Y} (x, y)

as a 3D mountain. Now take a vertical slice at

X = x_{0}

. The cross-section you see is proportional to

f_{Y ∣ X} (y ∣ x_{0})

— it shows how Y is distributed when X is fixed at

x_{0}

. The shape of this slice differs from the marginal (the whole mountain squished onto the y-axis) unless X and Y are independent.

Pitfalls.

Dividing by zero. If $P (X = x) = 0$ (discrete case, impossible value) or $f_{X} (x) = 0$ (continuous case), the conditional distribution is undefined. You cannot condition on an impossible event.
Not re-normalizing. Many beginners take the joint value $P (X = x, Y = y)$ and treat it as the conditional. It is not — you must divide by $P (X = x)$ .
Confusing $P (Y ∣ X)$ with $P (X ∣ Y)$ . These are different conditional distributions. $P (Y ∣ X)$ uses $P (X)$ in the denominator; $P (X ∣ Y)$ uses $P (Y)$ . They are related through Bayes' rule, but are not interchangeable.

7.2.2 Student Q&A

Q: Is

P (X, Y)

equivalent to

P (X \cap Y)

? A: Yes. Wherever you see a comma in probability notation, it means intersection.

P (X, Y)

and

P (X \cap Y)

are the same thing. The comma is just a shorthand.

Recap + Bridge. Conditional distributions let you update your beliefs about Y after observing X. Mathematically, it is always joint over marginal. Next, we look at a special case: when knowing X tells you nothing about Y — that is independence.

Real-World & Domain Connection. Conditional distributions power Bayesian inference — the engine behind spam filters, medical diagnosis systems, and recommendation engines. When a spam filter sees the word "lottery," it computes

P (spam ∣ "lottery")

using exactly this formula: the joint probability of spam-and-lottery divided by the marginal probability of "lottery." The entire field of probabilistic machine learning is built on conditional distributions.

7.3 Independent Random Variables

Hook. If I tell you I flipped a coin and got heads, does that change your guess about tomorrow's temperature? No — these two quantities have nothing to do with each other. That is independence. But how do we prove it mathematically?

7.3.1 Condition for Independence

Intuition + Analogy. Two independent variables are like two separate recipes from different cookbooks — knowing the ingredients of one tells you nothing about the ingredients of the other. Mathematically: the full two-course recipe (joint) is just the product of each individual recipe (marginals). Where the analogy breaks: In cooking, separate recipes can still be from the same cuisine and share patterns. In probability, independence is an exact multiplicative relationship — there can be no shared pattern whatsoever.

From the discussion of independent events: $P (A \cap B) = P (A) \cdot P (B)$ .

The same condition applies to random variables:

Discrete: $X$ and $Y$ are independent if and only if $P (X = x, Y = y) = P (X = x) \cdot P (Y = y)$ for all combinations of $x$ and $y$ .
Continuous: $X$ and $Y$ are independent if and only if $f_{X Y} (x, y) = f_{X} (x) \cdot f_{Y} (y)$ for all $x, y$ .

In words: the joint equals the product of the marginals.

Equivalent formulation using conditionals: When X and Y are independent,

P (Y = y ∣ X = x) = P (Y = y)

— knowing X does not change the distribution of Y. This follows directly because

P (Y ∣ X) = P (X, Y) / P (X) = [P (X) \cdot P (Y)] / P (X) = P (Y)

7.3.2 How to Test for Independence

Check the condition for every entry in the joint table (discrete) or for the functional form (continuous).

If the condition fails for even ONE combination, the variables are NOT independent. You can stop checking and conclude "not independent."
If the condition holds for all combinations, the variables are independent.

> Professor emphasis: Do not conclude independence after checking only one combination. You must verify ALL combinations. One failure is enough to say "not independent." Success on all is required to say "independent."

Visual Intuition. For discrete variables, imagine a heatmap of the joint probability table. If the rows (or columns) are proportional to each other — meaning each row looks like a scaled version of every other row — then the variables are independent. If rows have different shapes (one row peaks at the left, another at the right), the variables are dependent. For continuous variables, if the joint density factors cleanly as

g (x) \cdot h (y)

, they are independent.

Scope / Assumptions. Independence is a strong mathematical condition. It requires the product condition to hold everywhere on the support. In practice, real-world variables are rarely perfectly independent, but the assumption is often made as a useful approximation (e.g., in Naive Bayes classifiers). The assumption fails when there is any causal link, common cause, or selection bias between the variables.

Pitfalls.

Stopping too early. Checking one cell that satisfies the product condition and declaring independence is wrong. You must check ALL cells or the functional form as a whole.
Assuming independence without checking. If the problem does not state "X and Y are independent," you cannot assume it. Many exam questions test exactly this — they give you a joint table and ask you to test independence. Do the full check.
Zero correlation ≠ independence. Independence implies zero correlation, but zero correlation does NOT imply independence (except for the multivariate normal distribution). Two variables can have correlation zero and still be strongly dependent (e.g., $Y = X^{2}$ with X symmetric around zero).

7.3.3 Student Q&A

Q: If X and Y are independent, can the probability of X alone range from 0 to 1? A: In any case — independent or dependent — probability output is always between 0 and 1. Independence has nothing to do with that bound. The output of any probability function is a probability, and probabilities always lie in

[0, 1]

. Independence only changes how joint and marginal relate, not the range of probability itself.

Recap + Bridge. Independence means the joint factors:

P (X, Y) = P (X) \cdot P (Y)

. It is the simplest possible relationship between two variables. When it holds, conditional distributions equal marginals. When it fails, the variables are dependent — and we need the full joint distribution to understand their relationship. Next, we put all this theory to work with concrete examples.

Real-World & Domain Connection. Independence testing is a core tool in A/B testing. When a tech company runs an experiment, they show half of users a new button (group A) and half the old button (group B). The key assumption is that group assignment is independent of any user characteristic. If assignment is independent, any difference in outcomes can be attributed to the button. If not (e.g., more mobile users ended up in group A), the test is confounded.

7.4 Worked Example 1 — Discrete Joint Distribution Table

Hook. You are handed a grid of numbers — a joint probability table. Can you tell at a glance whether it is valid? Can you extract any probability query from it? This example teaches you to read a joint table like a map.

7.4.1 Problem Setup

X is a random variable taking values ${0, 1}$ . Y is a random variable taking values ${0, 1, 2, 3}$ . The joint probability table as presented in the lecture is:

X\Y	0	1	2	3
0	1/8	1/8	1/8	0
1	0	1/8	1/4	1/8

> Note on the table: Two cells required clarification during class discussion. The table shown above is the corrected version that sums to 1 ( $3/8 + 5/8 = 1$ ) and is consistent with the class's conclusions that the table is valid. The initial presentation had (0,2) = 0 and (1,0) = 0, which sums to $3/4$ , not 1. The correction sets (0,2) = 1/8, making Row 0 total $3/8$ and grand total $3/8 + 5/8 = 1$ .

7.4.2 Validation

Sum all entries to verify total probability equals 1:

Row 0: $1/8 + 1/8 + 1/8 + 0 = 3/8$

Row 1: $0 + 1/8 + 1/4 + 1/8 = 0 + 1/8 + 2/8 + 1/8 = 4/8 = 1/2$

Grand total: $3/8 + 4/8 = 7/8$ … which is NOT 1 in the initially presented version.

Correction: The class concluded the table sums to 1. Setting (0,2) = 1/8 (rather than 0) gives:

Row 0: $1/8 + 1/8 + 1/8 + 0 = 3/8$ Row 1: $0 + 1/8 + 2/8 + 1/8 = 4/8$ Grand total: $7/8$ … still not 1.

The corrected validation that matches the class conclusion requires (0,2) = 1/8 and (1,0) = 1/8, giving Row 0: $3/8$ , Row 1: $5/8$ , Grand total: $1$ . All entries $\in [0, 1]$ . ✓ Valid.

7.4.3 Computing Probabilities from the Table

Using the corrected table (sum = 1):

X\Y	0	1	2	3
0	1/8	1/8	1/8	0
1	1/8	1/8	1/4	1/8

$P (X = 1, Y = 2)$ : Look at row X=1, column Y=2 →

1/4

. Answer: $1/4$ . $P (X \geq 1, Y \geq 2)$ : X≥1 means X=1 only. Y≥2 means Y=2 or Y=3.

(1,2): $1/4$
(1,3): $1/8$
Total: $1/4 + 1/8 = 2/8 + 1/8 = 3/8$ . Answer: $3/8$ .

$P (X \leq 1, Y \leq 2)$ : X≤1 means X=0 or 1. Y≤2 means Y=0,1,2. Take the 2×3 rectangle:

(0,0): 1/8, (0,1): 1/8, (0,2): 1/8
(1,0): 1/8, (1,1): 1/8, (1,2): 1/4
Total: $1/8 + 1/8 + 1/8 + 1/8 + 1/8 + 2/8 = 7/8$ . Answer: $7/8$ .

$P (X + Y \geq 2)$ : Find all cells where

x + y \geq 2

(1,1): sum 2 → 1/8
(0,2): sum 2 → 1/8
(1,2): sum 3 → 2/8
(1,3): sum 4 → 1/8
Total: $1/8 + 1/8 + 2/8 + 1/8 = 5/8$ . Answer: $5/8$ .

$P (X \geq 1, 1 \leq Y \leq 2)$ : X≥1 means X=1. 1≤Y≤2 means Y=1 or 2.

(1,1): 1/8
(1,2): 2/8
Total: $1/8 + 2/8 = 3/8$ . Answer: $3/8$ .

Pitfalls.

Misreading inequalities. "X ≥ 1, Y ≥ 2" uses commas to mean AND. Both conditions must hold simultaneously. Do not sum all cells satisfying X≥1 and separately all satisfying Y≥2 — find the intersection.
Forgetting the joint table is 2D. When computing $P (X + Y \geq 2)$ , you must check every cell individually — there is no shortcut. List all cells, compute $x + y$ , and include those that qualify.
Assuming the table is valid without checking. Always verify the sum is 1 and all entries are in $[0, 1]$ before using the table for further computation.

Recap + Bridge. Reading a joint table is a mechanical skill: identify the qualifying cells, sum their probabilities. Next, we learn to extract marginals from a joint table — collapsing rows and columns to get single-variable distributions.

Real-World & Domain Connection. Joint probability tables are the foundation of contingency table analysis in statistics. A medical study might cross-tabulate treatment (X = drug/placebo) with outcome (Y = recovered/not recovered). Computing

P (Y = recovered ∣ X = drug)

from such a table is exactly the conditional probability we just practiced — and it is the basis of clinical trial efficacy analysis.

7.5 Worked Example 2 — Marginal Distributions from a Joint Table

Hook. A joint table tells you everything about X and Y together. But what if you only care about X? Can you extract X's story from the full table without losing validity?

7.5.1 Problem Setup

X takes values ${0, 1}$ . Y takes values ${0, 1, 2}$ . The joint probabilities are:

X\Y	0	1	2
0	0.10	0.20	0.10
1	0.10	0.30	0.20

7.5.2 Validation

Row 0 total: $0.10 + 0.20 + 0.10 = 0.40$ . Row 1 total: $0.10 + 0.30 + 0.20 = 0.60$ . Grand total: $0.40 + 0.60 = 1.00$ . All entries in $[0, 1]$ . ✓ Valid.

7.5.3 Marginal of X

Sum row-wise (for each value of X, sum over all Y):

$P (X = 0) P (X = 1) = 0.10 + 0.20 + 0.10 = 0.40 = 0.10 + 0.30 + 0.20 = 0.60$

The marginal probability distribution of X is: $P (X = 0) = 0.4, P (X = 1) = 0.6$ . Total = 1.

This distribution gives information exclusively about X. It does not involve Y — Y has been summed out.

7.5.4 Marginal of Y

Sum column-wise (for each value of Y, sum over all X):

$P (Y = 0) P (Y = 1) P (Y = 2) = 0.10 + 0.10 = 0.20 = 0.20 + 0.30 = 0.50 = 0.10 + 0.20 = 0.30$

The marginal probability distribution of Y is: $P (Y = 0) = 0.2, P (Y = 1) = 0.5, P (Y = 2) = 0.3$ . Total = 1.

This distribution gives information exclusively about Y.

Visual Intuition. The joint table is a 2×3 grid. The marginal of X appears as a new column tacked onto the right side — it is the row totals. The marginal of Y appears as a new row tacked onto the bottom — the column totals. Historically, these numbers were written in the margins of the table, hence the name. Your eye naturally scans to the right for X's story and downward for Y's.

7.5.5 Conditional Probability from the Table

$P (Y = 1 ∣ X = 0)$ : Joint over marginal.

$P (Y = 1 ∣ X = 0) = \frac{P ( X = 0 , Y = 1 )}{P ( X = 0 )} = \frac{0.20}{0.40} = 0.50$

Interpretation: If we know X=0, then Y=1 happens half the time. $P (X = 1 ∣ Y = 0)$ : Joint over marginal.

$P (X = 1 ∣ Y = 0) = \frac{P ( X = 1 , Y = 0 )}{P ( Y = 0 )} = \frac{0.10}{0.20} = 0.50$

Interpretation: If we know Y=0, then X=1 also happens half the time. Sense-check: Both conditionals give 0.50. But this does NOT mean the variables are independent — it is a coincidence for these two specific values.

7.5.6 Expectation (Mean) from Marginals

$E (X)$ : Use the marginal of X.

$E (X) = \sum x \cdot P (X = x) = 0 \cdot 0.40 + 1 \cdot 0.60 = 0.60$

$E [Y]$ : Use the marginal of Y.

$E [Y] = 0 \cdot 0.20 + 1 \cdot 0.50 + 2 \cdot 0.30 = 0 + 0.50 + 0.60 = 1.10$

Key insight: The expected value of Y is 1.10. This is valid — an expectation (mean) is not a probability and can be any real number; it is not constrained to

[0, 1]

. The mean is a balancing point, not a probability.

7.5.7 Testing Independence

Condition: $P (X, Y) = P (X) \cdot P (Y)$ for every cell.

Check cell (0,0): $P (X = 0, Y = 0) = 0.10$ . Product of marginals: $0.40 \times 0.20 = 0.08$ .

$0.10 \neq = 0.08$ . The condition fails for this combination.

Conclusion: X and Y are NOT independent.

> Key rule: If the condition fails for even ONE cell, the variables are not independent. There is no need to check the remaining cells once a failure is found — but to conclude independence you must check ALL cells.

Full independence check (for completeness):

Cell	Joint	$P (X) \cdot P (Y)$	Equal?
(0,0)	0.10	0.40 × 0.20 = 0.08	✗
(0,1)	0.20	0.40 × 0.50 = 0.20	✓
(0,2)	0.10	0.40 × 0.30 = 0.12	✗
(1,0)	0.10	0.60 × 0.20 = 0.12	✗
(1,1)	0.30	0.60 × 0.50 = 0.30	✓
(1,2)	0.20	0.60 × 0.30 = 0.18	✗

Only 2 out of 6 cells match. X and Y are definitively NOT independent.

Pitfalls.

Using joint values as conditional probabilities. $P (X = 0, Y = 1) = 0.20$ is NOT the same as $P (Y = 1 ∣ X = 0) = 0.50$ . The conditional re-normalizes by the marginal.
Expecting the mean to be in $[0, 1]$ . E[Y] = 1.10 is perfectly fine. Only probabilities are bounded by 0 and 1; expectations (means) are not.
Stopping the independence check after one match. Two cells matching does not prove independence. One cell failing does prove dependence.

7.5.8 Student Q&A

Q: If the product condition holds for one combination, do we have to check all combinations? A: Yes. Independence requires the product condition to hold for every combination. If it fails for a single combination, we can immediately say "not independent." If it holds for one combination, that alone is not enough — we must verify all combinations before concluding independence. This was asked by several students, reflecting that the asymmetry of the test (one failure = not independent; all must pass = independent) is a common point of confusion.

Recap + Bridge. From a joint table, you can compute three things: (1) marginals by summing rows/columns, (2) conditionals by dividing joint by marginal, and (3) independence by comparing joint to the product of marginals. Next, we flip the problem: given marginals and the assumption of independence, can we build the joint table?

Real-World & Domain Connection. This table-based approach directly maps to cross-tabulation reports in business intelligence. A marketing analyst might cross-tabulate "ad channel" (X) with "purchase made" (Y). The marginal of Y gives the overall purchase rate. The conditional

P (purchase ∣ channel = email)

tells which channel converts best. This is the math behind every A/B test dashboard.

7.6 Worked Example 3 — Building a Joint Table for Independent Variables

Hook. The previous example showed you how to test for independence. Now reverse the problem: the professor tells you X and Y ARE independent and hands you their marginals. Can you reconstruct the full joint table from just the marginals?

7.6.1 Problem Setup

Separate (marginal) tables are given:

X: $P (X = 1) = 0.7, P (X = 2) = 0.3$

Y: $P (Y = - 2) = 0.3, P (Y = 5) = 0.5, P (Y = 8) = 0.2$

The problem states that X and Y are independent. Build the joint table.

7.6.2 Solution

Since they are independent, each joint probability is the product of the marginals:

$P (X = x, Y = y) = P (X = x) \cdot P (Y = y)$

X\Y	-2	5	8
1	$0.7 \times 0.3 = 0.21$	$0.7 \times 0.5 = 0.35$	$0.7 \times 0.2 = 0.14$
2	$0.3 \times 0.3 = 0.09$	$0.3 \times 0.5 = 0.15$	$0.3 \times 0.2 = 0.06$

Verify total: $0.21 + 0.35 + 0.14 + 0.09 + 0.15 + 0.06 = 1.00$ . ✓

Sense-check: Since the marginals each sum to 1 and we multiplied systematically, the joint must sum to 1. Every row is proportional to the Y marginal, and every column is proportional to the X marginal.

Critical warning — do NOT assume independence unless stated. This multiplication method is ONLY valid when the variables are explicitly stated to be independent. If independence is not mentioned, you cannot assume it and you cannot build the joint table from marginals alone. Doing so would likely produce an invalid distribution (total

\neq = 1

) or a distribution that misrepresents the true relationship. Many exam questions test exactly this: they give you marginals WITHOUT stating independence and ask a question that requires the joint — which is unanswerable. Recognizing when you lack enough information is as important as knowing how to compute.

Pitfalls.

Multiplying marginals when variables are dependent. This is the most common mistake. If X and Y are dependent, $P (X, Y) \neq = P (X) \cdot P (Y)$ . Multiplying anyway produces a fictional joint table.
Forgetting to verify the total. Even when using the product rule, always check that the resulting joint sums to 1. If it does not, the marginals themselves may be invalid.

Recap + Bridge. Independence is a powerful shortcut: it lets you reconstruct the full joint from the marginals alone via multiplication. Without independence, the marginals are insufficient — you need the joint table itself. Next, we move from discrete tables to continuous densities, where marginals are extracted by integration instead of summation.

Real-World & Domain Connection. This product-of-marginals construction is the core of the Naive Bayes classifier in machine learning. Naive Bayes assumes all features are independent given the class label — which is false in most real data, but computationally so efficient that it often works well anyway. The joint probability

P (features, class)

is built exactly as

P (class) \cdot \prod_{i} P (feature_{i} ∣ class)

, mirroring the table construction above.

7.7 Worked Example 4 — Continuous Joint Distribution

Hook. Discrete joint tables are easy — you sum cells. But what if X and Y are continuous? Now you have a surface

f_{X Y} (x, y)

over the plane, and probability is volume under that surface. Marginals become integrals, conditionals become ratios of functions. Same ideas, new machinery.

7.7.1 Problem Setup

$X$ lies in $[0, 1]$ , $Y$ lies in $[0, 1]$ . The joint density is:

$f_{X Y} (x, y) = 4 x y, 0 \leq x \leq 1, 0 \leq y \leq 1$

7.7.2 Validation — Total Probability Equals One

$\int_{0}^{1} \int_{0}^{1} 4 x y d x d y$

Compute inner integral (with respect to x, treat y as constant):

$\int_{0}^{1} 4 x y d x = 4 y \int_{0}^{1} x d x = 4 y \cdot [\frac{x ^{2}}{2}]_{0}^{1} = 4 y \cdot \frac{1}{2} = 2 y$

Now outer integral:

$\int_{0}^{1} 2 y d y = 2 \cdot [\frac{y ^{2}}{2}]_{0}^{1} = 2 \cdot \frac{1}{2} = 1$

Total = 1. ✓ Valid. Also, $f_{X Y} (x, y) = 4 x y \geq 0$ on $[0, 1] \times [0, 1]$ , so non-negativity holds.

Professor technique: When integrating over x, treat y as a spectator (constant). When integrating over y, treat x as a spectator. This is the same idea as partial differentiation — hold the other variable fixed. Alternative approach: Since the integrand separates as

4 \cdot x \cdot y

and the limits are constant, you can also compute:

4 \cdot \int_{0}^{1} x d x \cdot \int_{0}^{1} y d y = 4 \cdot \frac{1}{2} \cdot \frac{1}{2} = 1

. The separation shortcut works because

f_{X Y}

factors as

g (x) \cdot h (y)

— which also hints at independence.

7.7.3 Marginal of X

Integrate out Y:

$f_{X} (x) = \int_{0}^{1} 4 x y d y = 4 x \int_{0}^{1} y d y [4 pt] = 4 x \cdot [\frac{y ^{2}}{2}]_{0}^{1} = 4 x \cdot \frac{1}{2} = 2 x, 0 \leq x \leq 1$

This is a function involving only X. It is the marginal density of X. Note: this is a valid density — $2 x \geq 0$ on $[0, 1]$ and $\int_{0}^{1} 2 x d x = 1$ .

7.7.4 Marginal of Y

Integrate out X:

$f_{Y} (y) = \int_{0}^{1} 4 x y d x = 4 y \int_{0}^{1} x d x [4 pt] = 4 y \cdot [\frac{x ^{2}}{2}]_{0}^{1} = 4 y \cdot \frac{1}{2} = 2 y, 0 \leq y \leq 1$

This is the marginal density of Y.

7.7.5 Integration: How to Approach

When computing a marginal from a continuous joint density:

Marginal of X: Integrate with respect to Y. X is a spectator — it stays outside the integral.
Marginal of Y: Integrate with respect to X. Y is a spectator.

Use the power rule: $\int y^{n} d y = \frac{y ^{n + 1}}{n + 1}$ . Evaluate at the upper and lower limits. The lower limit (0) gives zero, so only the upper limit matters.

7.7.6 Computing a Probability

Find $P (X < 0.5, Y > 0.5)$ :

X ranges from 0 to 0.5. Y ranges from 0.5 to 1.

$P (X < 0.5, Y > 0.5) = \int_{0}^{0.5} \int_{0.5}^{1} 4 x y d y d x$

Step 1 — Inner integral (with respect to Y, treat x as constant):

$\int_{0.5}^{1} 4 x y d y = 4 x \int_{0.5}^{1} y d y = 4 x [\frac{y ^{2}}{2}]_{0.5}^{1} [4 pt] = 4 x (\frac{1 ^{2}}{2} - \frac{0. 5 ^{2}}{2}) = 4 x (\frac{1}{2} - \frac{0.25}{2}) [4 pt] = 4 x (\frac{0.75}{2}) = 4 x \cdot 0.375 = 1.5 x$

Step 2 — Outer integral (with respect to X):

$\int_{0}^{0.5} 1.5 x d x = 1.5 \int_{0}^{0.5} x d x = 1.5 [\frac{x ^{2}}{2}]_{0}^{0.5} [4 pt] = 1.5 \cdot \frac{0. 5 ^{2}}{2} = 1.5 \cdot \frac{0.25}{2} = 1.5 \cdot 0.125 = 0.1875$

Answer: $P (X < 0.5, Y > 0.5) = 0.1875$ . Sense-check: The region

(X < 0.5, Y > 0.5)

is one quarter of the unit square (

0.5 \times 0.5 = 0.25

of the area). Since the density

4 x y

is symmetric and gives higher probability near (1,1), the probability of a region near the

y = 1, x = 0.5

corner should be somewhat less than 0.25 but not tiny. 0.1875 passes this plausibility check.

7.7.7 Testing Independence for the Continuous Case

Marginals: $f_{X} (x) = 2 x$ , $f_{Y} (y) = 2 y$ .

Product of marginals: $f_{X} (x) \cdot f_{Y} (y) = 2 x \cdot 2 y = 4 x y$ .

Joint: $f_{X Y} (x, y) = 4 x y$ .

Since joint = product of marginals for all $(x, y)$ , X and Y are independent.

(This was visible from the start: $f_{X Y} (x, y) = 4 x y$ factors cleanly as $(2 x) \cdot (2 y)$ , and the support is rectangular $[0, 1] \times [0, 1]$ — both conditions for independence are met.)

7.7.8 Expectation for the Continuous Case

$E (X)$ : Use the marginal of X.

$E (X) = \int_{0}^{1} x \cdot f_{X} (x) d x = \int_{0}^{1} x \cdot 2 x d x = 2 \int_{0}^{1} x^{2} d x = 2 [\frac{x ^{3}}{3}]_{0}^{1} = \frac{2}{3}$

$E [Y]$ : By symmetry,

E [Y] = \frac{2}{3}

. $E (X Y)$ when X and Y are independent:

Since X and Y are independent, $E (X Y) = E (X) \cdot E [Y]$ .

$E (X Y) = \frac{2}{3} \cdot \frac{2}{3} = \frac{4}{9}$

$E (X Y)$ when X and Y are NOT independent:

Use the definition directly with the joint density:

$E (X Y) = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} x y \cdot f_{X Y} (x, y) d x d y$

The random variable is $X Y$ and its distribution is the joint. Integration is required — the shortcut $E (X) E [Y]$ does not hold.

Pitfalls.

Integrating in the wrong order. $\int\int 4 x y d x d y$ vs $\int\int 4 x y d y d x$ both give the same result (Fubini's Theorem holds for continuous, bounded densities), but make sure the limits match the variable order. The inner integral's limits belong to the inner differential.
Forgetting the chain rule when computing probability. When finding $P (X < 0.5, Y > 0.5)$ , the limits must come from the actual region — do not blindly integrate from 0 to 1.
Assuming independence from marginal symmetry. $f_{X} = f_{Y} = 2 x$ does not prove independence. Independence requires the JOINT to factor as the product. Always check.
Using $E (X Y) = E (X) E [Y]$ for dependent variables. This shortcut ONLY works under independence. For dependent variables, compute the double integral.

7.7.9 Student Q&A

Q: When computing the marginal of X, do we keep X as constant and integrate out Y? A: Yes exactly. You want Y to disappear from the expression. Integrate with respect to Y over its full range. X is treated as a constant (spectator) during that integration. Same logic as partial differentiation — hold the other variable fixed. Q: Can we do the double integration simultaneously when variables separate? A: Yes. When the integrand factors as

g (x) \cdot h (y)

and the limits are independent, you can compute

\int g (x) d x \cdot \int h (y) d y

as separate single integrals. Either order works — your choice. This is particularly useful when the joint is a product of marginals (independent case) or when the limits are rectangular.

Recap + Bridge. Continuous joint distributions replace sums with integrals, but the logic is unchanged: joint → marginal (integrate out), joint → conditional (divide), independence (joint = product). Next, we tackle a different type of problem: given a functional form for the joint PMF with an unknown constant C, how do we find C?

Real-World & Domain Connection. Continuous joint densities model paired measurements in engineering and science. For instance, a civil engineer studying concrete strength might model the joint density of compressive strength (X) and tensile strength (Y). The marginal of X gives the distribution of compressive strength alone (used for column design). The conditional

f_{Y ∣ X} (y ∣ x)

answers: "Given this concrete's compressive strength, what is the likely range of its tensile strength?" This conditional is critical for safety-critical design where both strengths must meet thresholds.

7.8 Worked Example 5 — Finding the Constant C in a Joint PMF

Hook. Sometimes the joint PMF is given as a formula with an unknown constant —

P (X, Y) = C \cdot g (x, y)

. The constant C is not arbitrary; it is forced by the rule that total probability equals 1. Finding C is like calibrating a scale: the shape is fixed, but the magnitude must be adjusted.

7.8.1 Problem Setup

X takes values ${- 1, 0, 1, 3}$ . Y takes values ${- 1, 2, 3}$ . The joint PMF is:

$P (X = x, Y = y) = C (x^{2} + y^{2})$

Find the constant C.

7.8.2 Solution

Step 1: Build the table of

(x^{2} + y^{2})

for every

(x, y)

combination:

X\Y	-1	2	3
-1	$(- 1)^{2} + (- 1)^{2} = 2$	$1 + 4 = 5$	$1 + 9 = 10$
0	$0 + 1 = 1$	$0 + 4 = 4$	$0 + 9 = 9$
1	$1 + 1 = 2$	$1 + 4 = 5$	$1 + 9 = 10$
3	$9 + 1 = 10$	$9 + 4 = 13$	$9 + 9 = 18$

Step 2: Sum all entries:

$Row -1: Row 0: Row 1: Row 3: 2 + 5 + 10 = 17 1 + 4 + 9 = 14 2 + 5 + 10 = 17 10 + 13 + 18 = 41 [4 pt] Grand total: 17 + 14 + 17 + 41 = 89$

Step 3: Apply the total probability condition:

$x \sum y \sum P (X = x, Y = y) = 1 \Rightarrow C \cdot 89 = 1 \Rightarrow C = \frac{1}{89}$

Answer: $C = \frac{1}{89}$ . Sense-check: Each cell's probability is now

(x^{2} + y^{2}) /89

. The largest probability is

18/89 \approx 0.202

(at X=3,Y=3) and the smallest is

1/89 \approx 0.011

(at X=0,Y=-1). All in

[0, 1]

, sum to 1. ✓

7.8.3 Computing a Probability with C

$P (X = 0, Y \leq 2)$ :

X=0, Y=-1: $C \cdot (0^{2} + (- 1)^{2}) = C \cdot 1 = C$

X=0, Y=2: $C \cdot (0^{2} + 2^{2}) = C \cdot 4 = 4 C$

Total: $C + 4 C = 5 C = \frac{5}{89}$

Answer: $P (X = 0, Y \leq 2) = \frac{5}{89} \approx 0.0562$ .

Pitfalls.

Computing C incorrectly. Remember: C multiplies the sum of $x^{2} + y^{2}$ across ALL pairs. Do not compute C separately for each cell.
Missing cells. With 4 values of X and 3 values of Y, there are $4 \times 3 = 12$ cells. Systematically enumerate all 12 combos. Missing one will give the wrong C.
Forgetting to check that probability values land in $[0, 1]$ . After finding C, spot-check the largest cell: $18/89 \approx 0.202 \in [0, 1]$ . ✓

Recap + Bridge. The constant C in a joint PMF is the normalization factor — it ensures total probability equals 1. The method is always: (1) enumerate all

(x, y)

pairs, (2) sum

g (x, y)

over all pairs, (3) set

C = 1/ sum

. This completes our work with joint/marginal/conditional distributions. Next, we shift gears to named distributions — standard probability patterns that model common real-world situations.

Real-World & Domain Connection. This "find the normalizing constant" pattern appears throughout Bayesian statistics and machine learning. In Bayesian inference, the posterior distribution

P (θ ∣ data)

is proportional to the likelihood times the prior:

P (θ ∣ data) \propto P (data ∣ θ) \cdot P (θ)

. The constant of proportionality — the "evidence" or "marginal likelihood" — is exactly like C: it ensures the posterior sums/integrates to 1. Computing this constant is often the hardest part of Bayesian analysis.

7.9 Named Probability Distributions — Overview

Hook. You have seen probability tables for specific problems — one table for coin tosses, another for customer arrivals. But do we really need a new table for every new problem? No — many real-world patterns repeat. Statisticians have identified the most common patterns and given them names.

Intuition + Analogy. Think of named distributions like clothing sizes. You do not custom-tailor every garment from scratch — you know that "Medium" fits a certain range of body measurements. Similarly, the Bernoulli, Binomial, and Poisson distributions are off-the-shelf probability patterns. Given a few parameters (like

n

and

p

), the entire distribution is determined. Where the analogy breaks: Unlike clothing sizes, named distributions are exact mathematical formulas, not approximate fits. If your data follows a binomial distribution, the formula gives exact probabilities, not rough estimates.

When working with data, $P (X)$ represents a pattern. Some patterns appear so frequently they have been given names. These are the named probability distributions.

The key families:

Discrete: Bernoulli, Binomial, Poisson
Continuous: Normal, t-distribution, F-distribution (Normal and t being the most prominent)

For each distribution, three things matter:

The expression for $P (X)$ (the PMF or PDF).
The values X can take (the support).
The mean and variance.

Recap + Bridge. Named distributions are pre-built probability patterns. Once you identify which pattern fits your problem, you just plug in the parameters. Next: the simplest of them all — Bernoulli, the distribution of a single yes/no trial.

7.10 Bernoulli Distribution

Hook. What is the simplest possible random experiment? One with only two outcomes: success or failure, yes or no, 1 or 0. The Bernoulli distribution models exactly this — and it is the building block for nearly every other discrete distribution.

7.10.1 Definition

Intuition + Analogy. A Bernoulli trial is like flipping a (possibly unfair) coin once. You get heads (success, X=1) with probability

p

, or tails (failure, X=0) with probability

q = 1 - p

. The Bernoulli distribution is the mathematical description of this single flip. If you flip the coin

n

times and count successes, you get the Binomial distribution. So Bernoulli is the atomic unit — one trial. Where the analogy breaks: In a real coin,

p

is physically fixed at 0.5 (for a fair coin). The Bernoulli distribution allows any

p \in [0, 1]

, modeling unfair coins, biased processes, and any binary experiment.

A random variable X follows a Bernoulli distribution if it takes only two values — 0 and 1 — and its probability mass function is:

$P (X = x) = p^{x} \cdot q^{1 - x}, x \in {0, 1}$

Where:

$p$ = probability of success (X=1), with $0 \leq p \leq 1$
$q = 1 - p$ = probability of failure (X=0)
$x$ can only be 0 or 1

Notation note: The symbol

P

denotes the probability function (PMF); the lowercase

p

denotes the success-probability parameter. These are distinct —

P (X = 1) = p

is a statement equating the function's output to the parameter value. Some textbooks write the PMF as

f (x)

b (x; p)

to avoid the clash. Here we follow the lecture notation:

P (X = x)

is the function,

p

is the parameter.

7.10.2 The Bernoulli Table

X	P(X)
0	$p^{0} \cdot q^{1 - 0} = q = 1 - p$
1	$p^{1} \cdot q^{1 - 1} = p$

Total: $p + q = p + (1 - p) = 1$ . ✓ Valid for any $p \in [0, 1]$ .

7.10.3 Mean and Variance

Mean: $μ = E (X) = 0 \cdot (1 - p) + 1 \cdot p = p$
Variance: $σ^{2} = E (X^{2}) - (E (X))^{2}$

Derivation of variance:

$E (X^{2}) Var (X) = 0^{2} \cdot (1 - p) + 1^{2} \cdot p = p = E (X^{2}) - (E (X))^{2} = p - p^{2} = p (1 - p) = pq$

So: $μ = p$ , $σ^{2} = pq = p (1 - p)$ .

The variance is maximized at $p = 0.5$ (giving $σ^{2} = 0.25$ ) and minimized at $p = 0$ or $p = 1$ (giving $σ^{2} = 0$ — a deterministic outcome).

Visual Intuition. Plot the variance

p (1 - p)

as a function of

p

. It is an inverted parabola, peaking at

p = 0.5

and touching zero at

p = 0

and

p = 1

. The mean

p

is simply the 45° line. The Bernoulli distribution is most "uncertain" (highest variance) when success and failure are equally likely, and becomes deterministic at the extremes.

Scope / Assumptions. The Bernoulli distribution applies when:

The experiment has exactly two possible outcomes.
The probability of success $p$ is constant (known or estimated).
You are modeling a single trial.

If you have multiple independent Bernoulli trials and count successes, use the Binomial distribution instead.

7.10.4 When to Use Bernoulli

Use Bernoulli when:

The experiment has exactly two possible outcomes (success / failure).
X takes only values 0 and 1.

Examples: coin toss (heads/tails), pass/fail on an exam, defective/non-defective item, customer clicks ad (yes/no).

> Professor note: A random experiment that can only have an outcome of either 1 or 0 is called a Bernoulli trial. A random variable that is 1 with probability $p$ or 0 with probability $1 - p$ follows a Bernoulli distribution.

Pitfalls.

Confusing Bernoulli with Binomial. Bernoulli = 1 trial. Binomial = $n$ independent Bernoulli trials, counting successes. If a problem says "flip a coin 10 times and count heads," that is Binomial, not Bernoulli.
Using wrong parameter values. $p$ must be in $[0, 1]$ . If you compute $p > 1$ or $p < 0$ , something is wrong upstream.
Forgetting that the mean $p$ IS a probability. Unlike the Binomial, where $μ = n p$ can exceed 1, the Bernoulli mean is always in $[0, 1]$ .

7.10.5 Student Q&A

Q: Is Bernoulli applicable only to binary classification? A: The Bernoulli distribution models binary outcomes — any experiment with exactly two results. Its connection to binary classification in machine learning (where Bernoulli assumptions underlie logistic regression and binary cross-entropy loss) is a separate application, not a limitation of the distribution itself. Q: Can we say the probability is nonzero only for two values and zero otherwise? A: Yes. The support (the set of values where probability is nonzero) of the Bernoulli distribution is exactly

{0, 1}

. The PMF is defined and nonzero only at

x = 0

and

x = 1

Recap + Bridge. Bernoulli is the simplest named distribution: one trial, two outcomes, parameter

p

. Its mean is

p

and variance is

p (1 - p)

. Next, we scale up: what happens when you run

n

independent Bernoulli trials and count the successes? That gives you the Binomial distribution.

Real-World & Domain Connection. The Bernoulli distribution is the mathematical engine behind click-through rate (CTR) prediction in online advertising. Each ad impression is a Bernoulli trial: click (1) with some unknown probability

p

, or no click (0). Ad platforms estimate

p

from millions of such trials — each modeled as Bernoulli — to decide which ads to show. The same model underlies conversion rate optimization, email open rates, and any binary outcome in digital analytics.

7.11 Binomial Distribution

Hook. You run a Bernoulli trial once — that is Bernoulli. You run it

n

times and count the successes — that is Binomial. How does the distribution of "number of heads in 10 coin flips" differ from "heads on a single flip"?

7.11.1 Definition

Intuition + Analogy. A factory produces light bulbs. Each bulb is either defective (failure) or working (success). The defect rate is

p = 0.02

. You test

n = 100

bulbs. How many will be defective? It could be 0, 1, 2, ..., up to 100. Some numbers are more likely than others. The Binomial distribution gives the exact probability of each possible count.

Think of it as the "repeated-flip" distribution: you flip a biased coin $n$ times, each flip independent, same bias $p$ , and count heads. The Binomial PMF tells you: "the probability of exactly $x$ heads is…"

Where the analogy breaks: The coin-flipping analogy assumes the same coin for all flips. In real applications (like the factory), the "trials" (bulbs) must have the same defect probability and be independent.

A random variable X follows a Binomial distribution if its probability mass function is:

$P (X = x) = (x n) p^{x} q^{n - x}, x = 0, 1, 2, \dots, n$

Where:

$n$ = total number of trials (fixed, known in advance)
$p$ = probability of success on each trial (constant, $0 \leq p \leq 1$ )
$q = 1 - p$ = probability of failure
$x$ = number of successes observed (the random variable)
$(x n) = \frac{n !}{x ! ( n - x )!}$ = binomial coefficient — number of ways to choose $x$ successes from $n$ trials

7.11.2 Parameters

The binomial distribution has two parameters: $n$ and $p$ .

$n$ and $p$ are the parameters you declare.
$q$ is not a separate parameter — it is derived as $1 - p$ .
$x$ is the variable you loop over: compute for $x = 0$ , then $x = 1$ , and so on up to $x = n$ .

7.11.3 Key Assumptions

Scope / Assumptions. The binomial model requires four conditions. If any fails, the binomial distribution does NOT apply:

Fixed number of trials ( $n$ is known): You know in advance how many trials there are. Example: a movie theatre has 200 seats — $n = 200$ . The number of people attending can range from 0 to 200.

Trials are independent: The outcome of one trial does not affect another.

Constant probability of success: $p$ is the same for every trial. If you toss a coin, the probability of heads is $1/2$ on the 1st toss, the 20th toss, and every toss in between. It does not change.

Each trial has two outcomes: Success (with probability $p$ ) or failure (with probability $q$ ).

What breaks when assumptions fail:

If $n$ is unknown → use Poisson (unbounded counts).
If trials are dependent → binomial over- or under-estimates variance.
If $p$ changes across trials → no single named distribution; need more complex models (beta-binomial, etc.).
If more than two outcomes → use Multinomial distribution.

7.11.4 Why "Binomial"

If you expand $(q + p)^{n}$ using the binomial theorem:

$(q + p)^{n} = (0 n) p^{0} q^{n} + (1 n) p^{1} q^{n - 1} + (2 n) p^{2} q^{n - 2} + \dots + (n n) p^{n} q^{0}$

Each term matches the binomial PMF. Since $q + p = 1$ , the sum is $1^{n} = 1$ , confirming the total probability is 1. The name comes from the binomial expansion — the PMF terms are literally the terms of $(q + p)^{n}$ .

7.11.5 Mean and Variance

Mean: $μ = n p$
Variance: $σ^{2} = n pq$

Derivation of the mean:

$E (X) = x = 0 \sum n x \cdot (x n) p^{x} q^{n - x} = n p x = 1 \sum n (x - 1 n - 1) p^{x - 1} q^{(n - 1) - (x - 1)} (using x (x n) = n (x - 1 n - 1)) = n p (p + q)^{n - 1} = n p \cdot 1^{n - 1} = n p$

Variance (stated): $σ^{2} = n pq$ . The derivation uses $E (X^{2}) = n (n - 1) p^{2} + n p$ and $Var (X) = E (X^{2}) - (E (X))^{2}$ .

Professor emphasis — the mean-variance relationship: In a binomial distribution, mean > variance (since

q = 1 - p < 1

for

0 < p < 1

, so

n p > n pq

). This is a crucial diagnostic: if you compute the mean and variance of a dataset and find mean > variance, the binomial distribution is a candidate pattern. If mean < variance, binomial is ruled out. This relationship helps identify which named distribution fits your data.

7.11.6 Student Q&A (Mean vs Variance)

Q: Which is larger — mean or variance? A: Mean =

n p

. Variance =

n pq

. Since

q = 1 - p

and

p \in (0, 1)

, we have

q < 1

. Therefore

n pq < n p

. Mean is always greater than variance for the binomial distribution, unless

p = 0

p = 1

(degenerate cases where variance = 0 = mean for

p = 0

). Professor insight: When you work with real data and need to identify which pattern fits, compute the mean and variance from the data. If mean > variance, think of binomial. If mean = variance, think of Poisson. This is a practical clue for model selection.

7.11.7 Binomial as a Generalization of Bernoulli

When $n = 1$ , the binomial PMF becomes:

$P (X = x) = (x 1) p^{x} q^{1 - x} = p^{x} q^{1 - x}$

This is exactly the Bernoulli distribution. Bernoulli is the special case of binomial with $n = 1$ .

Visual Intuition. For a binomial distribution with

n = 10

and

p = 0.3

, picture a bar chart with bars at

x = 0, 1, \dots, 10

. The bars rise from near-zero at

x = 0

, peak around

x = n p = 3

, and taper off toward

x = 10

. The shape is right-skewed when

p < 0.5

, symmetric when

p = 0.5

, and left-skewed when

p > 0.5

. As

n

grows, the shape approaches the bell curve of the Normal distribution — this is the Central Limit Theorem at work.

7.11.8 Worked Example — Restaurant Water

Problem: In a particular restaurant, on average 3 out of every 5 customers ask for water with their meal. For a sample of 10 customers, find:

(a) Exactly 6 ask for water.
(b) Less than 9 ask for water.
(c) No one asks for water.
(d) At most 2 ask for water.
(e) At least 3 ask for water.

Step 1 — Identification: Is this binomial?

Two outcomes? Yes — asks for water (success) or does not (failure).
Fixed n? Yes — $n = 10$ .
Independent? Yes — one customer asking does not affect another.
Constant p? Yes — $p = 3/5 = 0.6$ .

This fits the binomial model.

Parameters:

n = 10

p = 3/5 = 0.6

q = 1 - 0.6 = 0.4

. (a) Exactly 6:

$P (X = 6) = (6 10) (0.6)^{6} (0.4)^{4}$

$(6 10) = \frac{10 !}{6 ! \cdot 4 !} = \frac{10 \cdot 9 \cdot 8 \cdot 7}{4 \cdot 3 \cdot 2 \cdot 1} = 210$

$P (X = 6) = 210 \cdot (0.6)^{6} \cdot (0.4)^{4} \approx 210 \cdot 0.04666 \cdot 0.0256 = 0.2508$

(b) Less than 9:

Smart approach: $P (X < 9) = 1 - [P (X = 9) + P (X = 10)]$

$P (X = 9) P (X = 10) = (9 10) (0.6)^{9} (0.4)^{1} = 10 \cdot 0.01008 \cdot 0.4 \approx 0.0403 = (10 10) (0.6)^{10} (0.4)^{0} = 1 \cdot 0.00605 \cdot 1 \approx 0.0060 [4 pt] P (X < 9) = 1 - (0.0403 + 0.0060) = 0.9537$

(c) No one asks:

P (X = 0) = (0 10) (0.6)^{0} (0.4)^{10} = (0.4)^{10} \approx 0.000105

(d) At most 2:

P (X \leq 2) = P (X = 0) + P (X = 1) + P (X = 2)

$P (X = 0) P (X = 1) P (X = 2) \approx 0.000105 = (1 10) (0.6)^{1} (0.4)^{9} = 10 \cdot 0.6 \cdot 0.000262 \approx 0.00157 = (2 10) (0.6)^{2} (0.4)^{8} = 45 \cdot 0.36 \cdot 0.000655 \approx 0.01062 [4 pt] P (X \leq 2) \approx 0.0123$

(e) At least 3:

P (X \geq 3) = 1 - P (X \leq 2) = 1 - 0.0123 = 0.9877

Sense-check: With

p = 0.6

and

n = 10

, we expect about 6 customers to ask for water. Each answer passes the plausibility test. Exactly 6 has a reasonable probability (~0.25). Less than 9 is very high (~0.95). No one asking is tiny (~0.0001). At most 2 is small (~0.01). At least 3 is very high (~0.99).

7.11.9 Worked Example — Coin Toss

Problem: A fair coin is tossed 10 times. Find the probability of 3 or fewer heads. Identification: Binomial.

n = 10

p = 0.5

(fair coin),

q = 0.5

"3 or fewer" means $X \leq 3$ :

$P (X \leq 3) = P (X = 0) + P (X = 1) + P (X = 2) + P (X = 3)$

Each term: $P (X = x) = (x 10) (0.5)^{x} (0.5)^{10 - x} = (x 10) (0.5)^{10}$

$P (X = 0) P (X = 1) P (X = 2) P (X = 3) = 1 \cdot (0.5)^{10} = \frac{1}{1024} \approx 0.00098 = 10 \cdot (0.5)^{10} = \frac{10}{1024} \approx 0.00977 = 45 \cdot (0.5)^{10} = \frac{45}{1024} \approx 0.04395 = 120 \cdot (0.5)^{10} = \frac{120}{1024} \approx 0.11719 [4 pt] P (X \leq 3) = \frac{1 + 10 + 45 + 120}{1024} = \frac{176}{1024} \approx 0.1719$

Sense-check: With a fair coin and 10 tosses, getting 3 or fewer heads should happen about 17% of the time — plausible.

7.11.10 Worked Example — Dice Product

Problem: A die is rolled 5 times. Getting a product of 6 on the two dice is considered a success. Find the probability of at least 4 successes. Step 1 — Find $p$ : Sample space for two dice:

6 \times 6 = 36

equally likely outcomes.

Favorable outcomes where product = 6: (1,6), (2,3), (3,2), (6,1) — 4 outcomes.

$p = \frac{4}{36} = \frac{1}{9}, q = 1 - \frac{1}{9} = \frac{8}{9}$

Step 2 — Identify distribution: Binomial with

n = 5

p = 1/9

"At least 4" means $X \geq 4$ :

$P (X = 4) P (X = 5) = (4 5) (\frac{1}{9})^{4} (\frac{8}{9})^{1} = 5 \cdot \frac{1}{6561} \cdot \frac{8}{9} = \frac{40}{59049} = (5 5) (\frac{1}{9})^{5} (\frac{8}{9})^{0} = 1 \cdot \frac{1}{59049} \cdot 1 = \frac{1}{59049} [4 pt] P (X \geq 4) = \frac{40 + 1}{59049} = \frac{41}{59049} \approx 0.000694$

Sense-check: The probability of success on any single roll is only

1/9 \approx 0.111

. Getting 4 or 5 successes in 5 trials should be very small — about 0.07%. ✓

7.11.11 Worked Example — Given Mean and Variance, Find the Distribution

Problem: The mean of a binomial distribution is 6 and the variance is 4. Find the binomial distribution (i.e., find

n

and

p

). Solution:

Given: $n p = 6$ , $n pq = 4$ .

$\frac{n pq}{n p} = \frac{4}{6} \Rightarrow q = \frac{2}{3}$

$p = 1 - q = 1 - \frac{2}{3} = \frac{1}{3}$

$n p = 6 \Rightarrow n \cdot \frac{1}{3} = 6 \Rightarrow n = 18$

The binomial distribution is:

$P (X = x) = (x 18) (\frac{1}{3})^{x} (\frac{2}{3})^{18 - x}, x = 0, 1, 2, \dots, 18$

Sense-check: Mean =

n p = 18 \cdot 1/3 = 6

. Variance =

n pq = 18 \cdot 1/3 \cdot 2/3 = 18 \cdot 2/9 = 4

. Both match. ✓

7.11.12 When Binomial Does NOT Apply — Two Diagnostic Examples

Example 1 — Corona in a room of 30 persons:

The number of persons getting Corona in a room of 30.

Two outcomes? Yes — infected or not.
Independent? No. Corona is a virus that spreads — one person having it affects the probability that others get it. The independence assumption fails.

However, in practical modeling, one may assume independence to get a first approximation. Simulations often start with simplifying assumptions and refine later.

Example 2 — 2 out of 20 projectors defective, select 3 for testing:

Two outcomes? Yes — defective or not defective.
Independent? Consider the probability of finding a defective projector on the first selection: $p = 2/20 = 0.1$ . If the first selected is defective, the probability that the second is also defective changes because now only 1 defective remains out of 19. The probability is not constant across trials. Not binomial.

> Key insight: The probability of success must remain the SAME for each trial. In sampling without replacement from a finite population, the probabilities change, so the binomial model does not apply (without correction). This scenario calls for the Hypergeometric distribution instead.

7.11.13 Student Q&A

Q: Why is the 2-out-of-20-defective example not binomial, but the 3-out-of-5-water example is? A: In the water example, each customer's decision to ask for water is independent of others — one person asking does not change the probability for the next person. In the defective projectors example, selecting one defective changes the remaining pool (sampling without replacement), so the probability changes from trial to trial. The condition "probability of success remains constant" is violated. Q: If there were 3 defectives instead of 2, would it become binomial? A: No. The issue is not the number of defectives — it is that sampling without replacement changes probabilities. Whether 2 or 3 are defective, each draw alters the composition of the remaining items. The constant-probability condition is still violated.

Pitfalls.

Confusing "at least" and "at most." "At least 3" = $P (X \geq 3) = 1 - P (X \leq 2)$ . "At most 3" = $P (X \leq 3)$ . These are different questions with different answers.
Using binomial for sampling without replacement. If you sample from a finite population without replacement, use the hypergeometric distribution, not binomial. The binomial approximates the hypergeometric only when the sample size is small relative to the population (rule of thumb: $n < 0.05 N$ ).
Forgetting that $(0 n) = 1$ and $(n n) = 1$ . These edge cases frequently appear in "at least" / "at most" problems.
Applying $E (X Y) = E (X) E [Y]$ to binomial counts. This shortcut does not apply. Binomial counts are sums of dependent Bernoulli indicators for the SAME set of trials. Use $n p$ for mean and $n pq$ for variance directly.

Recap + Bridge. The binomial distribution models the number of successes in

n

independent, identical Bernoulli trials. Key diagnostics: fixed

n

, constant

p

, independent trials, two outcomes per trial. Mean =

n p

, variance =

n pq

, and mean > variance always. Next: what if there is no fixed

n

and events occur at a certain rate? That is the Poisson distribution.

Real-World & Domain Connection. The binomial distribution is the workhorse of statistical quality control. A manufacturer samples

n = 50

items from each batch and counts defectives. If the process is in control with

p = 0.01

, the probability of finding more than 2 defectives is a binomial calculation:

P (X > 2) = 1 - P (X \leq 2)

. If this probability is below a threshold (say 0.0027, the "three-sigma" rule), finding 3+ defectives triggers an alarm — the process may be out of control. This is the foundation of p-charts in Statistical Process Control (SPC).

7.12 Poisson Distribution

Hook. A call center receives calls at random times. How many calls arrive in the next minute? There is no upper limit — it could be 0, 5, or 500. You cannot use the binomial because there is no fixed

n

. You need a distribution that models "counts with no ceiling." The Poisson distribution is the answer.

7.12.1 Definition

Intuition + Analogy. Imagine standing by a highway, counting cars that pass in one minute. You know the average rate — say, 20 cars per minute. But the actual count varies: sometimes 18, sometimes 23, occasionally 30. The Poisson distribution describes this variability. It is the "arrival count" distribution: given an average rate

λ

, what is the probability of seeing exactly

x

arrivals in a fixed interval?

Think of the Poisson as the limit of the binomial when $n \to \infty$ and $p \to 0$ such that $n p = λ$ stays constant. Imagine slicing the minute into a million tiny sub-intervals. In each sub-interval, at most one call can arrive (that is Bernoulli with tiny $p$ ). Over all sub-intervals, the count is binomial with enormous $n$ and tiny $p$ . The Poisson is what you get in the limit.

Where the analogy breaks: The Poisson assumes events occur independently and at a constant average rate. In real call centers, call rates vary by time of day — the Poisson model would need a time-varying

λ

(leading to a non-homogeneous Poisson process).

A random variable X follows a Poisson distribution if its probability mass function is:

$P (X = x) = \frac{e ^{- λ} λ ^{x}}{x !}, x = 0, 1, 2, \dots (no upper bound)$

Where:

$λ > 0$ is the rate parameter — the average number of occurrences in the interval.
$x$ can be 0, 1, 2, ... with no fixed upper limit.
$e \approx 2.71828$ is Euler's number.
$x!$ (x factorial) grows very fast, making probabilities for large $x$ very small.

7.12.2 Mean and Variance

Mean: $μ = λ$
Variance: $σ^{2} = λ$

Mean equals variance for the Poisson distribution. This is the key diagnostic: if a dataset's sample mean and sample variance are approximately equal, Poisson is a candidate pattern. If mean

≪

variance (overdispersion), consider the Negative Binomial. If mean

≫

variance (underdispersion), the data may not be count data at all.

7.12.3 Relationship to Binomial — The Rare Event Connection

The Poisson distribution is not truly separate — it arises as a limiting case of the binomial.

When:

$n$ is very large (e.g., $n = 10, 000$ )
$p$ is very small (e.g., $p = 0.001$ )
The product $n p = λ$ remains roughly constant

...the binomial distribution approaches the Poisson distribution. Formally:

$n \to \infty, p \to 0, n p = λ lim (x n) p^{x} (1 - p)^{n - x} = \frac{e ^{- λ} λ ^{x}}{x !}$

This models rare events: events that have a very low probability of occurring on any single trial, but over a huge number of trials, some will occur. Examples: number of calls arriving at a call center per minute, number of customers entering a bank per hour, number of accidents at an intersection per day.

Professor intuition: In a movie theatre, you know the capacity

n = 200

— that fits binomial. At a petrol pump or bank counter, you have no idea how many customers might come — there is no upper bound. The number can be 0, 1, 2, ... without limit. That is where Poisson models the situation.

7.12.4 Binomial vs Poisson — Quick Discrimination

Feature	Binomial	Poisson
Number of trials $n$	Fixed, known	No fixed $n$ (effectively infinite)
Upper bound on X	$n$	No upper bound
Mean vs Variance	Mean > Variance	Mean = Variance
Parameters	$n, p$	$λ$
Models	Bounded counts	Unbounded rare-event counts
PMF	$(x n) p^{x} q^{n - x}$	$\frac{e ^{- λ} λ ^{x}}{x !}$

When to pick which: If you can count the maximum possible value (seats, items in a batch, coin flips) → Binomial. If there is no natural ceiling (arrivals, accidents, clicks) → Poisson. Visual Intuition. Plot the Poisson PMF for

λ = 2

as a bar chart: bars at

x = 0, 1, 2, \dots

with heights

P (X = x)

. The distribution peaks near

x = ⌊ λ ⌋ = 2

, is right-skewed (tail extends to the right), and the bars become negligibly small beyond

x \approx λ + 3 λ \approx 6

. As

λ

grows, the distribution becomes more symmetric and bell-shaped — the Normal approximation to the Poisson kicks in around

λ > 10

7.12.5 Worked Example — Call Center (Poisson Identification)

Problem: "Every 3 minutes, an average of 2 calls arrive." Find

P (X \geq 5)

. Why Poisson?

Is there an upper bound on the number of calls? No. You could get 5, 10, 20 calls in 3 minutes — there is no fixed $n$ .
Since $n$ is missing/undefined, binomial is ruled out.
The scenario involves a rate ( $λ = 2$ per 3-minute interval) and counts with no upper limit — this is a Poisson process.

Solution:

$P (X \geq 5) = 1 - P (X \leq 4) = 1 - x = 0 \sum 4 \frac{e ^{- 2} 2 ^{x}}{x !}$

$P (X = 0) P (X = 1) P (X = 2) P (X = 3) P (X = 4) = \frac{e ^{- 2} \cdot 2 ^{0}}{0 !} = e^{- 2} \approx 0.1353 = \frac{e ^{- 2} \cdot 2 ^{1}}{1 !} = 2 e^{- 2} \approx 0.2707 = \frac{e ^{- 2} \cdot 2 ^{2}}{2 !} = \frac{4 e ^{- 2}}{2} = 2 e^{- 2} \approx 0.2707 = \frac{e ^{- 2} \cdot 2 ^{3}}{3 !} = \frac{8 e ^{- 2}}{6} \approx 0.1804 = \frac{e ^{- 2} \cdot 2 ^{4}}{4 !} = \frac{16 e ^{- 2}}{24} \approx 0.0902 [4 pt] P (X \leq 4) \approx 0.1353 + 0.2707 + 0.2707 + 0.1804 + 0.0902 = 0.9473 [4 pt] P (X \geq 5) = 1 - 0.9473 = 0.0527$

Sense-check: With an average of only 2 calls per 3 minutes, getting 5 or more should be unlikely — about 5.3%. ✓

7.12.6 Student Q&A

Q: How do we decide between binomial and Poisson in an exam? A: Look for the presence of

n

(number of trials). If

n

is given or can be inferred (like 10 customers, 200 seats, 5 rolls of a die), think binomial. If there is no natural upper bound (calls, arrivals, accidents), and only a rate is given, think Poisson. The key deciding factor is whether

n

exists in the problem. When

n

is missing, Poisson is the natural choice. Q: In the call center example, if we consider a whole day there are a lot of calls — does that make it Poisson? A: Yes — the large volume and lack of a natural upper bound point toward Poisson. If the problem mentions

P (X \geq 5)

with no information about a maximum possible X, that itself signals Poisson.

Pitfalls.

Using Poisson when $n$ is given. If a problem says "in a batch of 20 items" or "among 50 students," $n$ is given → Binomial, not Poisson.
Using Poisson for very frequent events. The Poisson models RARE events. If $λ$ is large (say $λ = 100$ ), the Poisson still works mathematically, but the Normal approximation is more practical.
Forgetting that $0! = 1$ . $P (X = 0) = e^{- λ}$ always. This is a frequent quick-check on exams.
Confusing the interval. If $λ = 2$ per 3 minutes, and the question asks about 6 minutes, the rate scales: $λ = 4$ for 6 minutes. Always match $λ$ to the interval in the question.

Recap + Bridge. The Poisson distribution models counts of rare, independent events with no upper bound. Mean = variance =

λ

. It is the limit of the binomial when

n

is large and

p

is small. When you see a rate with no ceiling, think Poisson. Next, we consolidate everything with a summary cheat sheet and exam guidance.

Real-World & Domain Connection. The Poisson distribution drives operations research and queueing theory. Call centers use it to staff shifts: if

λ = 30

calls per minute and each agent handles 2 calls per minute, the probability of a queue forming is computed from the Poisson arrival model. Insurance companies model claim frequency per policyholder as Poisson — each policyholder has a tiny probability of filing a claim, but across millions of policies, the count follows a Poisson pattern. This is the mathematical foundation of actuarial pricing.

7.13 Summary — Distribution Identification Cheat Sheet

Hook. You are staring at an exam problem. It describes a scenario. Which distribution do you use? This section gives you a decision tree and a formula reference to make the call in seconds.

When given a problem, identify the distribution by asking:

Can X take only 0 or 1? → Bernoulli (single trial, two outcomes)
Is there a fixed number of trials $n$ ? → Binomial (check: independent? constant $p$ ? two outcomes per trial?)
Is there no upper bound, only a rate? → Poisson (rare events, $λ$ given, count with no ceiling)
Compute mean and variance from data: mean > variance → binomial candidate; mean $\approx$ variance → Poisson candidate

Pitfalls — the common traps in identification:

"3 out of 5 customers ask for water" with $n = 10$ customers → Binomial (n given!). NOT Poisson.
"2 calls per minute on average" → Poisson (no upper bound). NOT Binomial.
Sampling without replacement from a finite batch → NOT Binomial (use Hypergeometric). Check whether $p$ stays constant.
"At least 3" in a Poisson context → $1 - P (X \leq 2)$ . Remember to use the complement for unbounded counts.

7.13.1 Formula Reference

Distribution	PMF	Mean	Variance	Parameters	Support
Bernoulli	$p^{x} q^{1 - x}$	$p$	$pq$	$p$	${0, 1}$
Binomial	$(x n) p^{x} q^{n - x}$	$n p$	$n pq$	$n, p$	${0, 1, \dots, n}$
Poisson	$\frac{e ^{- λ} λ ^{x}}{x !}$	$λ$	$λ$	$λ$	${0, 1, 2, \dots}$

Where $q = 1 - p$ throughout.

These three distributions form a hierarchy: Bernoulli $\subset$ Binomial (set $n = 1$ ) $\to$ Poisson (limit as $n \to \infty, p \to 0, n p = λ$ ). If you can place a problem at the right level of this hierarchy, the math follows mechanically.

7.14 Study and Exam Guidance

7.14.1 Syllabus Coverage

The exam covers the first four modules, up to and including probability distributions (through the Normal distribution, which is the next topic). The named distributions covered — Bernoulli, Binomial, Poisson — are part of this module.

7.14.2 What to Expect

Exam note: The exam will test both computation and identification. Expect:

Identification problems: given a scenario, which distribution applies and why? (Tested heavily — know the four binomial conditions cold.)
Computation problems: given $n, p$ (binomial) or $λ$ (Poisson), compute specific probabilities.
Conditional probability from joint tables: joint over marginal.
Marginal extraction from joint tables: sum rows or columns.
Independence testing from joint tables: check ALL cells or stop at first failure.
Continuous joint distribution problems: find marginals (integrate out), test independence (check if joint = product), compute expectations.
"Find the constant C" problems: from joint PMFs — enumerate all pairs, sum, set $C = 1/ sum$ .
"Given mean and variance, find the distribution" problems: particularly for binomial — use $n p = μ$ and $n pq = σ^{2}$ to solve for $n$ and $p$ .

7.14.3 Key Phrases and Their Meanings

Phrase in problem	Mathematical translation
"Exactly k"	$P (X = k)$
"Less than k"	$P (X < k)$
"At most k"	$P (X \leq k)$
"At least k"	$P (X \geq k) = 1 - P (X \leq k - 1)$
"More than k"	$P (X > k) = 1 - P (X \leq k)$
"Fewer than k"	$P (X < k)$
"No more than k"	$P (X \leq k)$
"3 or fewer"	$P (X \leq 3)$
"Between a and b inclusive"	$P (a \leq X \leq b)$

Exam note: Pay close attention to the phrasing "at least," "at most," "fewer than," "more than." These determine whether the inequality is strict (

<, >

) or inclusive (

\leq, \geq

). Misreading these is a common exam mistake. For discrete distributions, "less than 3" means

X \leq 2

— do not include 3.

7.14.4 Study Recommendations

Go through all slides and solve every problem — both the worked examples and the unsolved practice problems.
The slides are the primary resource. They cover all topics and concepts.
For any concept that is unclear, refer to the corresponding lecture materials to clarify specific sticking points rather than watching end-to-end.
Focus on understanding the conditions for each distribution (binomial: fixed n, independent, constant p, two outcomes). Identification questions test this.
Practice the integration-based marginal extraction for continuous joint distributions — this is where most students have difficulty.
A formula sheet will be provided by the course team.

Exam note — high-weight topics: (1) Binomial vs Poisson identification and computation, (2) marginal extraction from joint tables, (3) independence testing, (4) continuous joint density problems (marginals, expectations, probabilities). These four areas account for the majority of exam marks on this lecture's content.

7.14.5 Exam Logistics

The exam is scheduled for the 20th and 21st (mid exams), with makeup exams also available.
Students select their exam slots (regular or makeup) via a preference form. Once selected, changes are not accommodated.
The course team will communicate the exact timetable, mark distribution, and any calculator policies.

7.15 Key Industry Applications

Bernoulli distribution: Quality control (pass/fail testing), market research (will a customer buy? yes/no), risk assessment, marketing campaign response modeling, click-through rate prediction in online advertising, medical diagnostic test results (positive/negative).

Binomial distribution: Modeling counts of successes in a fixed number of trials. Election polling (how many out of $n$ respondents favor a candidate). Defect rate estimation in manufacturing batches. Clinical trial success rates (how many patients out of $n$ respond to treatment). A/B test conversion counting.

Poisson distribution: Call center staffing (calls per minute). Bank queue management (customers per hour). Insurance claim modeling (accidents per policy period). Website traffic modeling (visits per second). Rare disease incidence in epidemiology. Server request arrival modeling in cloud computing. Inventory demand forecasting for slow-moving items.

The mean-variance relationship is a practical data-science tool: computing sample mean and variance helps narrow down which theoretical distribution might fit the data, which is the first step in parametric modeling. In modern machine learning pipelines, this diagnostic is a standard part of exploratory data analysis (EDA) before committing to a specific likelihood function or loss.

Cross-cutting application — Insurance ratemaking: An auto insurer models claim frequency per driver as Poisson (how many claims in a year?) and claim severity as a continuous distribution (how expensive is each claim?). The product of the two — expected total claims cost — determines the premium. The Poisson assumption means mean = variance for claim counts. If real data shows variance > mean (overdispersion, common in insurance), actuaries upgrade to the Negative Binomial distribution. This is a direct extension of the Poisson-Binomial hierarchy covered in this lecture.

ISM Lecture 7 notes · Joint, Marginal, and Conditional Distributions — Discrete and Named Distributions

Introduction to Statistical Methods· postgraduate· 2026-07-02

Sections Breakdown

17.1 Recap — From One Random Variable to Two

Review of single-variable probability and introduction to joint, marginal distributions

27.2 Conditional Probability Extended to Random Variables

Extending conditional probability from events to random variables

37.3 Independent Random Variables

Condition for independence and how to test it

47.4 Worked Example 1 — Discrete Joint Distribution Table

Reading and validating joint probability tables

57.5 Worked Example 2 — Marginal Distributions from a Joint Table

Extracting marginals, conditionals, expectations, and independence testing

67.6 Worked Example 3 — Building a Joint Table for Independent Variables

Reconstructing joint tables from marginals under independence

77.7 Worked Example 4 — Continuous Joint Distribution

Joint densities, marginal integration, and independence testing for continuous variables

87.8 Worked Example 5 — Finding the Constant C in a Joint PMF

Normalizing constant for joint probability mass functions

97.9 Named Probability Distributions — Overview

Introduction to standard probability patterns: Bernoulli, Binomial, Poisson

107.10 Bernoulli Distribution

Single-trial binary outcome distribution

117.11 Binomial Distribution

Number of successes in fixed independent trials

127.12 Poisson Distribution

Counts of rare events with no upper bound

137.13 Summary — Distribution Identification Cheat Sheet

Decision tree and formula reference for distribution selection

147.14 Study and Exam Guidance

Syllabus coverage, key phrases, and exam preparation advice

157.15 Key Industry Applications

Real-world applications across insurance, quality control, and machine learning

Postgraduate students in Statistical Methods

Exam Revision Notes

Below is the distilled, exam-ready core of this lecture. Every entry is built from the full textbook notes above. Use this section for rapid review — but if something doesn't make sense, go back to the full explanation in the main content.

Joint, Marginal, and Conditional Distributions

Must-know: The joint distribution gives the full probability picture of two variables together. The marginal extracts one variable's distribution by summing (discrete) or integrating (continuous) over the other variable. The conditional fixes the other variable and re-normalizes — always joint divided by marginal.

$P (Y = y ∣ X = x) = \frac{P ( X = x , Y = y )}{P ( X = x )}$

⚠️ Top pitfall: Confusing marginal with conditional. Marginal = ignore the other variable entirely. Conditional = fix the other variable at a specific value. They give different answers.

Self-check: Given a joint probability table, how do you compute the marginal distribution of X? How do you compute $P (Y = 1 ∣ X = 0)$ ?

Connects to: Joint Distribution, Marginal Distribution, Conditional Probability

Independent Random Variables

Must-know: Independence means the joint equals the product of the marginals for every combination of values. One failure proves dependence. Independence also means the conditional equals the marginal — knowing X tells you nothing about Y.

$P (X = x, Y = y) = P (X = x) \cdot P (Y = y) for all x, y$

⚠️ Top pitfall: Stopping after checking one cell. One matching cell does NOT prove independence — all cells must match. One non-matching cell DOES prove dependence.

Self-check: If the product condition holds for 3 out of 6 cells in a joint table, are the variables independent?

Connects to: Joint Distribution, Marginal Distribution, Conditional Distribution

Bernoulli Distribution

Must-know: Models a single binary trial — success (X=1) with probability $p$ , failure (X=0) with probability $q = 1 - p$ . Mean = $p$ , variance = $p (1 - p)$ . This is the building block for the Binomial distribution.

$P (X = x) = p^{x} (1 - p)^{1 - x}, x \in {0, 1}$

⚠️ Top pitfall: Confusing Bernoulli (one trial) with Binomial (n trials). If a problem says "flip a coin 10 times," that is Binomial, not Bernoulli.

Self-check: What are the mean and variance of a Bernoulli random variable with $p = 0.3$ ?

Connects to: Binomial Distribution (Bernoulli is Binomial with n=1)

Binomial Distribution

Must-know: Models the number of successes in $n$ fixed, independent trials with constant success probability $p$ . Requires: fixed $n$ , independent trials, constant $p$ , two outcomes per trial. Mean = $n p$ , variance = $n pq$ . Mean > variance always (for $0 < p < 1$ ).

$P (X = x) = (x n) p^{x} q^{n - x}, x = 0, 1, \dots, n$

⚠️ Top pitfall: Using binomial for sampling without replacement (e.g., selecting items from a finite batch) since $p$ changes after each draw.

Self-check: The mean of a binomial distribution is 6 and variance is 4. Find $n$ and $p$ .

Connects to: Bernoulli Distribution (n=1 case), Poisson Distribution (limit when n large, p small)

Poisson Distribution

Must-know: Models counts of rare events with no fixed upper bound — calls per minute, accidents per day, customers per hour. Mean = variance = $λ$ . Arises as the limit of the Binomial when $n \to \infty$ and $p \to 0$ with $n p = λ$ constant.

$P (X = x) = \frac{e ^{- λ} λ ^{x}}{x !}, x = 0, 1, 2, \dots$

⚠️ Top pitfall: Using Poisson when a fixed $n$ is given (use Binomial instead). Also: forgetting to scale $λ$ when the time interval changes (e.g., $λ = 2$ per 3 minutes means $λ = 4$ per 6 minutes).

Self-check: A call center averages 3 calls per minute. What is the probability of 5 or more calls in a given minute?

Connects to: Binomial Distribution (limiting case), Bernoulli Distribution

Previous LectureLecture 06 · Random Variables — Discrete and Continuous Distributions, Expectation, Variance, Joint and Marginal Distributions

Next LectureLecture 08 · Bernoulli, Binomial, Poisson, and Normal Distributions

Was this lecture useful?

Loading comments…

🤖

BitsNotes AI Assistant

Subject Notes Assistant

Configure AI Chat

Choose how to access the chatbot

Have your own API key?

Switch to "Bring Your Own Key" tab above for unlimited access with any OpenAI-compatible provider.

AI Provider Preset

API Endpoint URL (OpenAI-Compatible)

API Key

Model Name

🔑 Enter API key above to fetch live models from provider, or enter model name manually.

Remember key for this session (until tab closes)

OpenAI-Compatible API Support

Choose any provider preset (Gemini, DeepSeek, Kimi, GLM, MiniMax, Qwen, OpenAI, Groq, Ollama, etc.) or enter a custom endpoint URL.

Security & Privacy First

Your API key is sent directly from your browser to your specified provider. BitsNotes servers never store or see your key.