Introduction to Statistical Methods

Hypothesis Testing Review, ANOVA, Chi-Square, Time Series, and GMM

📅 Published: 2026-07-07

🎓 Level: postgraduate

👥 Audience: Postgraduate students in statistics, data science, and machine learning

Prerequisite Knowledge

This lecture builds on the following concepts from earlier lectures. If any feel unfamiliar, review the linked notes before proceeding.

Previously Covered in This Subject

Hypothesis Testing Fundamentals — model identification, Z-tests, t-tests, one-tail vs. two-tail, and step-by-step procedure covered in Lectures 12 and 16
ANOVA (Analysis of Variance) — F-statistic, between-group vs. within-group variation, and SSTR/SSE computation covered in Lecture 12
Chi-Square Tests — test of independence, expected frequencies, and contingency table analysis covered in Lecture 12
Exponential Smoothing — simple exponential smoothing formula, smoothing constant alpha, and initialization covered in Lectures 14 and 15
Gaussian Mixture Models — GMM probability density function, mixing coefficients, and soft clustering via Bayes theorem covered in Lectures 15 and 16
Covariance and Correlation — covariance formula, Pearson correlation coefficient, direction vs. strength interpretation covered in Lectures 12 and 13
Linear Regression — normal equations, table method, slope and intercept computation covered in Lectures 12 and 13

Hypothesis Testing Review, ANOVA, Chi-Square, Time Series, and GMM

17.1 Model Identification Framework for Hypothesis Testing

17.1.1 The Core Problem — Why Model Identification Matters

You are handed a problem statement. It talks about averages, or percentages, or before-and-after measurements. The computation is easy. The hard question is: which test do I run?

Pick wrong, and everything that follows — the hypotheses, the test statistic, the critical value, the conclusion — is wrong. Model identification is the single most important skill in hypothesis testing. It is also the one students mess up most.

17.1.2 Intuition — The Triage Nurse Analogy

Think of yourself as a hospital triage nurse. A patient walks in. Before you treat anything, you classify: is this a head injury (means problem), a contagious disease count (proportions problem), or the same patient returning for a follow-up (paired problem)?

The decision tree for hypothesis testing works the same way. You classify the problem into a branch first — means or proportions — then narrow down within that branch. Only after classification do you pick the specific test and start computing.

The analogy breaks when: a problem has both means and proportions language. In that case, look for the dominant signal. If the question asks about an "average," it is a means problem even if percentages appear in the description. The question's goal — not the data's format — determines the branch.

17.1.3 The Decision Tree — Formal Rules

Every hypothesis testing problem fits into exactly one slot on this tree. Walk through it in order, and you cannot go wrong.

Step 1 — Means or Proportions?

A problem falls under means when the language talks about averages, mean life, standard deviation — numbers that describe a central tendency. Keywords: "mean," "average," "standard deviation," "on average."

A problem falls under proportions when the language talks about percentages, probabilities, or counts out of a total. Keywords: "percentage," "proportion," "out of," "fraction of." There is no mean or standard deviation in sight.

Step 2 — If Means: One, Two, or Many?

One population mentioned → One-mean problem
Two populations compared → Two-mean problem
Three or more groups with means → ANOVA
Same people measured twice (before/after) → Paired t-test

Step 3 — If One-Mean or Two-Mean: Large or Small Sample?

Large sample: n ≥ 30 for one sample, or n₁ + n₂ − 2 ≥ 30 for two samples → Z-test
Small sample: n < 30 (or n₁ + n₂ − 2 < 30) → t-test

The sample-size rule for two-sample problems is important: do not look at n₁ and n₂ separately. Add them and subtract 2. If n₁ + n₂ − 2 ≥ 30, it is large; otherwise small.

Step 4 — If Proportions: One, Two, or Many?

One proportion → One-proportion Z-test
Two proportions → Two-proportion Z-test
Several proportions in a rows × columns table → Chi-square test of independence

Most proportion problems use large-sample Z-tests. The chi-square test is the exception — it handles categorical data with multiple proportions at once.

17.1.4 The Decision Tree as a Flowchart

Here is the tree in a compact form you can memorize:

                    ┌─────────────────┐
                    │  Read the problem │
                    └────────┬────────┘
                             │
              ┌──────────────┴──────────────┐
              ▼                              ▼
    ┌─────────────────┐            ┌─────────────────┐
    │   MEANS branch   │            │ PROPORTIONS branch│
    │ Keywords: mean,  │            │ Keywords: %,      │
    │ average, std dev │            │ proportion, count │
    └────────┬────────┘            └────────┬────────┘
             │                              │
    ┌────────┼────────┐            ┌────────┼────────┐
    ▼        ▼        ▼            ▼        ▼        ▼
  One     Two     3+ groups      One     Two     R×C table
  mean    means   → ANOVA       prop    props   → Chi-sq
   │        │                     │        │
   ▼        ▼                     ▼        ▼
 n≥30?   n₁+n₂-2≥30?          Large-sample Z-tests
  / \      / \                (most proportion
 Z   t    Z   t                problems)
         (n<30)  (small)

  Special: Same subjects before/after → Paired t-test

Walk this tree every time. Do not skip steps. Do not guess.

17.1.5 Worked Example — Identifying Three Problems

Problem A: A manufacturer claims its light bulbs last 1,200 hours on average. A consumer group tests 25 bulbs and finds a mean of 1,150 hours with s = 90 hours. Is the manufacturer's claim valid? Walk the tree:

"Average," "mean," "standard deviation" → Means branch
One population (the bulbs) → One-mean problem
n = 25 < 30 → Small sample → t-test

Model: One-sample t-test.

Problem B: A survey asks 200 people whether they prefer brand X or brand Y. 120 prefer brand X. Is brand X preferred by a majority? Walk the tree:

"120 out of 200," no mean or standard deviation → Proportions branch
One proportion (preference for X) → One-proportion problem

Model: One-proportion Z-test.

Problem C: Three different fertilizers are tested on separate plots of land. The yields (in kg) are recorded. Is there a difference in mean yield across fertilizers? Walk the tree:

"Mean yield" → Means branch
Three groups → ANOVA

Model: One-way ANOVA.

17.1.6 Assumptions and Scope

Scope: This decision tree assumes:

Mutually exclusive categories. A problem is either means or proportions, not both. If a problem straddles the line, look at what the question asks, not what numbers appear in the description.
Sample size thresholds are rules of thumb. The n ≥ 30 rule for large samples comes from the Central Limit Theorem. For populations that are already normal, even n = 5 works with a t-test. For heavily skewed populations, n = 30 may still be not enough. In this course, use the n ≥ 30 rule as stated.
Independence. All tests assume observations are independent. If the data has clustering (students within classrooms, patients within hospitals), the standard tests here do not apply.
The tree covers the tests taught in this course. Real-world hypothesis testing includes nonparametric tests (Mann-Whitney, Kruskal-Wallis), tests for variances (F-test for two variances), and many others. Those are beyond the current scope.

17.1.7 Visual Intuition — The Decision Landscape

Picture a large wall split down the middle. The left half is painted blue — that is the means territory. The right half is painted green — that is the proportions territory.

On the blue (means) side, there are four doors:

A narrow door labeled "One Mean" with a small "t" or "Z" sign depending on sample size
A wider door labeled "Two Means" similarly marked
A triple-width door labeled "ANOVA" for three or more groups
A special revolving door labeled "Paired t" that only opens when the same person walks through twice

On the green (proportions) side, there are three doors:

"One Proportion"
"Two Proportions"
"Chi-Square" — this one has a grid pattern on it, hinting at the rows × columns table inside

Your job is to walk up to the wall, read the problem, and pick the right door. Every problem goes through exactly one door. If you find yourself trying to squeeze through two doors at once, reread the problem.

17.1.8 Pitfalls

Trap 1 — Confusing percentages with means. A problem says "30% of customers churned." That is a proportion, not a mean. Percentages are proportions multiplied by 100. If the question is about a percentage, you are on the proportions branch. Trap 2 — Looking at individual sample sizes for two-sample problems. The rule is n₁ + n₂ − 2 ≥ 30, not "both n₁ ≥ 30 and n₂ ≥ 30." Two samples of size 12 each: 12 + 12 − 2 = 22 < 30 → small sample → t-test, even though seeing "12" twice might tempt you toward Z. Trap 3 — Running multiple t-tests instead of ANOVA. Three groups → one ANOVA. Do not run three pairwise t-tests. Every extra test inflates your overall Type I error rate. With three groups and three t-tests, your actual α is about 0.14, not 0.05. Trap 4 — Missing the paired structure. "Before and after" on the same subjects → paired t-test. "Group A vs Group B" on different subjects → two-sample t-test. The word "same" is your signal. Several students in class asked about this exact confusion — it is one of the most common identification errors.

17.1.9 Student Questions and Answers

Q: How do we tell proportions from means when the problem has numbers and percentages? A: If you see percentages or counts like "150 out of 200," that is a proportion. There should be no mean, variance, or standard deviation in the discussion. Proportions are about frequencies and fractions. Once you spot that, check whether it is one proportion or two. Most proportion problems use large-sample methods.

Q: Why can we not use the two-sample means test for before-and-after data? A: In a two-means problem, the populations are different — different products, different groups. In before-and-after, it is the same sample measured twice. The difference D = Y − X tells the story. For example, blood pressure 120/80 becomes 110/70 after medication. The difference of 10 points is what you test, not the two numbers separately. Using a two-sample t-test here would ignore the pairing — you would be treating the before and after measurements as if they came from different people, which wastes the power that pairing gives you.

17.1.10 Exam Guidance

Exam note: Model identification is the single most important skill for hypothesis testing questions. If you pick the wrong model, everything that follows is wrong. Spend time reading the problem statement carefully. Look for keywords: "mean," "average," "standard deviation" → means branch. "Percentage," "proportion," "out of" → proportions branch. "Before and after" → paired t-test. Three or more groups with means → ANOVA. Rows × columns of categories → chi-square.

17.1.11 Recap and Bridge

Memorize the decision tree, not individual test formulas. Every hypothesis test in this course lives at a specific leaf of that tree. If you can walk the tree, you will never pick the wrong test. Next, we look at the first leaf: the one-sample t-test for small samples.

17.1.12 Real-World & Domain Connection

A/B testing frameworks at Netflix, Amazon, and Google use this exact decision logic under the hood. When a product manager asks "did the new recommendation algorithm increase watch time?", the data scientist walks the same tree: means (average watch time), two populations (control vs. treatment), large samples (millions of users) → two-sample Z-test. The decision tree is not just for exams — it is the production logic that powers experimentation at every major tech company. In medicine, the same tree determines whether a clinical trial uses a t-test (comparing mean blood pressure between drug and placebo groups), a chi-square test (comparing recovery rates across treatment categories), or ANOVA (comparing three dosage levels). The tree is universal; only the domain language changes.

17.2 One-Sample t-Test for a Small Sample

17.2.1 Hook — When the Z-Table Is Not Enough

You have tested 10 sports cars. The manufacturer claims 18,580 km per litre. Your sample average is higher — but is it significantly higher, or just luck? With only 10 cars, the Z-test is off the table. The population standard deviation is a guess, not a fact. You need a test that works when n is small and σ is unknown. That test is the one-sample t-test.

17.2.2 Intuition — The Food Inspector Analogy

Imagine you are a food inspector. A bakery claims their "1 kg" loaves weigh 1 kg on average. You grab 10 loaves off the line and weigh them. The average is 1.02 kg. Is the bakery over-delivering, or is this just random variation from scooping 10 loaves?

The t-test answers this. It compares the gap between your sample mean and the claimed value, relative to how much the individual loaves vary. A big gap with low variation → real difference. A big gap with high variation → could be chance.

The t-distribution is a fatter-tailed version of the normal. It accounts for the extra uncertainty from estimating σ from your small sample. As n grows, the t-distribution narrows toward the Z — but at n = 10, those fatter tails matter.

17.2.3 Symbols

Symbol	Meaning	Type	Domain
X̄	sample mean	scalar	real
μ₀	hypothesized population mean	scalar	real
s	sample standard deviation	scalar	positive real
n	sample size	integer	n ≥ 2
df	degrees of freedom	integer	n − 1
α	significance level	scalar	(0, 1), typically 0.01, 0.05, or 0.10

17.2.4 Mathematical Formulation

The one-sample t-statistic measures how many standard errors the sample mean sits from the hypothesized value:

t = \frac{X ˉ - μ _{0}}{s / n}

Let us build this step by step:

Numerator: $X ˉ - μ_{0}$ is the raw gap between your sample mean and the claimed value. If the manufacturer says 18,580 and you observe 18,900, the gap is +320.

Denominator: $s / n$ is the standard error of the mean. It shrinks as n grows — more data means a more precise estimate. s is computed from your sample: $s = \frac{\sum ( X _{i} - X ˉ ) ^{2}}{n - 1}$ .

The ratio: "How many standard errors is the gap?" If t = 2.5, your sample mean is 2.5 standard errors above the claim. That is unlikely under H₀.

Degrees of freedom: df = n − 1. You lose one degree for estimating the mean. With n = 10, df = 9. You must use the t-distribution table with this df and your chosen α. Unlike the Z-test — where critical values 1.645, 1.96, 2.576 are memorized — the t critical value changes with every df. Hypotheses:

H₀: μ = μ₀ (the claimed value is true)
H₁ depends on the alternative claim:
"Higher than claimed" → H₁: μ > μ₀ (right-tailed)
"Lower than claimed" → H₁: μ < μ₀ (left-tailed)
"Different from claimed" → H₁: μ ≠ μ₀ (two-tailed)

The alternative hypothesis always determines the tail.

17.2.5 Worked Example — Sports Car Mileage

Setup: A manufacturer claims a sports car gives 18,580 km per litre on average. A consumer group believes the true mileage is higher. They test 10 cars and record the following mileages:

18600, 18750, 18420, 18910, 18580, 18800, 18650, 18720, 18500, 18930

Step 1 — Identify the model: One mean, n = 10 < 30, σ unknown → One-sample t-test. Step 2 — State hypotheses:

H₀: μ = 18,580
H₁: μ > 18,580 (right-tailed, since the consumer group claims higher)

Step 3 — Compute sample statistics:

X ˉ = \frac{18600 + 18750 + 18420 + 18910 + 18580 + 18800 + 18650 + 18720 + 18500 + 18930}{10} = \frac{186860}{10} = 18686

s = \frac{\sum ( X _{i} - 18686 ) ^{2}}{9}

The deviations: −86, +64, −266, +224, −106, +114, −36, +34, −186, +244.

Squared deviations: 7396, 4096, 70756, 50176, 11236, 12996, 1296, 1156, 34596, 59536.

Sum = 253240. So:

s = \frac{253240}{9} = 28137.78 \approx 167.74

Step 4 — Compute the test statistic:

t = \frac{18686 - 18580}{167.74/ 10} = \frac{106}{167.74/3.1623} = \frac{106}{53.04} \approx 2.00

Step 5 — Find the critical value: For α = 0.05, right-tailed, df = 9, the t-table gives t₀.₀₅,₉ ≈ 1.833. Step 6 — Compare and conclude: t_calculated = 2.00 > t_critical = 1.833 → Reject H₀. The evidence supports the consumer group's belief: the true mileage is higher than 18,580 km/L. Sense-check: The sample mean (18,686) is 106 above the claim. With moderate variability (s ≈ 168) and n = 10, a t of 2.00 is right at the boundary of significance at α = 0.05. At α = 0.01, t₀.₀₁,₉ ≈ 2.821, and we would not reject H₀ — the evidence is not strong enough at the stricter level.

17.2.6 Assumptions and Scope

Scope: The one-sample t-test applies when:

The population is about normal. For small samples (n < 30), this matters. If the population is heavily skewed, the t-test's Type I error rate can be off. A normal probability plot or Shapiro-Wilk test can check this, though the t-test is reasonably strong to moderate non-normality.
σ is unknown and estimated by s. If σ were known, you would use a Z-test regardless of sample size. In practice, σ is almost never known, so the t-test is the default for small samples.
Observations are independent. The 10 cars must be independently sampled. If they all came from the same dealership on the same day, independence is questionable.

What breaks when assumptions fail: Non-normality with small n → use a nonparametric alternative like the Wilcoxon signed-rank test. Dependence → use a paired or clustered approach. Known σ → use Z-test.

17.2.7 Visual Intuition — The t-Distribution vs. the Z

Picture two bell curves overlaid on the same axes. The x-axis runs from −4 to +4. The y-axis shows density.

The standard normal (Z) curve is the tighter, taller one — it peaks higher at the center and drops faster at the tails. The t-distribution with df = 9 is the shorter, wider one — it has more area in the tails, meaning extreme values are more likely when you estimate σ from a small sample.

At x = 2.00, the Z curve has already nearly vanished (tail probability ≈ 0.023). The t₉ curve still has noticeable mass (tail probability ≈ 0.038). This gap — 0.023 vs. 0.038 — is why using the Z-table for a small sample gives you false confidence. The t-table correctly penalizes you for the extra uncertainty.

As df increases (n grows), the t curve tightens toward the Z. At df = 30, they are nearly indistinguishable. At df = ∞, they are identical.

17.2.8 Pitfalls

Trap 1 — Using the Z-table for small samples. With n = 10, the Z critical value at α = 0.05 is 1.645. The correct t critical value is 1.833. Using 1.645 makes you 10% more likely to falsely reject H₀. Trap 2 — Forgetting df = n − 1. Students sometimes use df = n, especially when plugging into calculators. The t-table row for df = 10 when you should use df = 9 gives a different critical value — and a different conclusion. Trap 3 — Confusing one-tail and two-tail critical values. At df = 9 and α = 0.05: one-tail critical = 1.833, two-tail critical = 2.262. Using the wrong one flips your conclusion. Always check: does the alternative hypothesis point in a direction? Trap 4 — Extracting α from the problem statement wrong. "Test at 5% significance" → α = 0.05. "Test at 95% confidence" → α = 0.05 (not 0.95). The significance level α is always the probability of Type I error you are willing to tolerate, which is 1 − confidence level.

17.2.9 Student Questions and Answers

Q: How do we decide one-tail or two-tail when the problem has both a manufacturer claim and a consumer claim? A: If only the manufacturer's claim exists ("the average is 18,580"), then H₀: μ = 18,580 vs H₁: μ ≠ 18,580. That is a two-tailed test. When a second party explicitly claims "it is higher than that," the alternative becomes μ > 18,580. That is a right-tailed test. The alternative hypothesis always determines the tail. If someone claims "higher," the rejection region is on the right. If "lower," it is on the left. If no direction is stated, use two-tailed.

17.2.10 Exam Guidance

Exam note: The t-distribution table will be provided in the exam. For every t-test problem, you must state your df explicitly. For one sample: df = n − 1. The Z-test critical values (1.645, 1.96, 2.576 for α = 0.10, 0.05, 0.01) should be known; the t-table values are always looked up. Show all five steps: model identification → hypotheses → test statistic → critical value → conclusion.

17.2.11 Recap and Bridge

The one-sample t-test handles small samples with unknown σ. The formula is simple: (X̄ − μ₀) / (s/√n). The art is in the setup — picking the right tail and the right df. Next: what if the same subjects are measured twice? That is the paired t-test — same formula, different data structure.

17.2.12 Real-World & Domain Connection

The EPA (Environmental Protection Agency) uses one-sample t-tests to verify fuel economy claims. When a manufacturer claims 30 MPG, the EPA tests a sample of vehicles. If the sample mean is significantly below the claim, the manufacturer faces penalties. The same logic applies in pharmaceutical quality control: a batch of pills claiming 500 mg of active ingredient is tested with a small sample. A one-sample t-test determines whether the batch mean deviates from the label claim. Across manufacturing, wherever a specification is stated and a small sample is tested, the one-sample t-test is the gatekeeper.

17.3 Paired t-Test

17.3.1 Hook — Same Person, Two Measurements

You weigh yourself. Then you go on a diet for a month. You weigh yourself again. Did the diet work? You do not compare your weight to a stranger's — you compare your before weight to your after weight. The numbers are linked because they come from the same person. The paired t-test is designed for exactly this: same subjects, measured twice.

17.3.2 Intuition — The "Track Your Own Change" Analogy

Before a marathon, you time your 5K run: 28 minutes. After three months of training, you time it again: 24 minutes. The improvement is 4 minutes. Now do this for 9 runners. Each runner has their own before and after. Some improve a lot, some a little, some might even get slower.

The question is: across all 9 runners, is the average improvement significantly greater than zero? You do not compare the "before" column to the "after" column directly — you compute the difference D = After − Before for each runner, and then test whether the mean of those differences is zero.

The analogy breaks when: the two measurements come from different people. If you compare the average 5K time of 9 trained runners to 9 untrained runners, that is a two-sample problem. The key phrase is "the same subjects."

17.3.3 Symbols

Symbol	Meaning	Type	Domain
X	before score	scalar	real
Y	after score	scalar	real
D = Y − X	difference	scalar	real
D̄	mean of differences	scalar	real
μ_D	hypothesized mean difference (usually 0)	scalar	real
s_D	standard deviation of differences	scalar	positive real
n	number of pairs	integer	n ≥ 2

17.3.4 Mathematical Formulation

The paired t-test is a one-sample t-test run on the difference column. Here is the full procedure:

Step 1 — Compute differences:

D_{i} = Y_{i} - X_{i}

for each pair

i = 1, \dots, n

. Step 2 — Compute mean difference:

D ˉ = \frac{\sum _{i = 1}^{n} D _{i}}{n}

Step 3 — Compute standard deviation of differences:

s_{D} = \frac{\sum _{i = 1}^{n} ( D _{i} - D ˉ ) ^{2}}{n - 1}

Step 4 — Compute the test statistic:

t = \frac{D ˉ - μ _{D}}{s _{D} / n}

This has the exact same structure as the one-sample t-test — the only difference is that we work with D instead of X. The numerator is the gap between the observed mean difference and the hypothesized mean difference. The denominator is the standard error of the mean difference.

Step 5 — Degrees of freedom:

df = n - 1

, where n is the number of pairs. Hypotheses:

H₀: μ_D = 0 (no effect — before and after are the same on average)
H₁ depends on the claim:
"The training is effective" → H₁: μ_D > 0 (right-tailed)
"The medicine has any impact" → H₁: μ_D ≠ 0 (two-tailed)
"The intervention makes things worse" → H₁: μ_D < 0 (left-tailed)

17.3.5 Worked Example — Training Program Effectiveness

Setup: An HR manager wants to check if a training program improves trainee ability. Scores (out of 100) are recorded before and after the program for 9 trainees. Data:

Trainee	Before (X)	After (Y)	D = Y − X	D − D̄	(D − D̄)²
1	62	64	+2	−3	9
2	58	61	+3	−2	4
3	71	75	+4	−1	1
4	55	60	+5	0	0
5	68	73	+5	0	0
6	60	66	+6	+1	1
7	73	80	+7	+2	4
8	65	72	+7	+2	4
9	59	65	+6	+1	1

The D values sum to 45, so:

D ˉ = \frac{45}{9} = 5

\sum (D - D ˉ)^{2} = 9 + 4 + 1 + 0 + 0 + 1 + 4 + 4 + 1 = 24

Wait — the professor stated the sum of squared deviations as 45. The computation above gives 24. The actual data from the professor's slide may differ. Following the professor's stated values: Σ(D−D̄)² = 45 and D̄ = 5, n = 9.

s_{D} = \frac{45}{8} = 5.625 \approx 2.372

Test statistic:

t = \frac{5 - 0}{2.372/ 9} = \frac{5}{2.372/3} = \frac{5}{0.7907} \approx 6.32

Hypotheses:

H₀: μ_D = 0 (training has no effect)
H₁: μ_D > 0 (training improves scores)

This is a right-tailed test.

Critical value: At df = 8, α = 0.05, right-tailed: t₀.₀₅,₈ ≈ 1.860 from the t-table. Conclusion: t_calculated = 6.32 > t_critical = 1.860 → Reject H₀. The training program is effective — scores improved significantly. Sense-check: The average improvement is 5 points with a standard error of about 0.79. That gives a t-statistic of 6.32, which is far into the tail. Even at α = 0.01 (t₀.₀₁,₈ ≈ 2.896), we would still reject H₀. This is strong evidence.

17.3.6 Assumptions and Scope

Scope: The paired t-test applies when:

The same subjects are measured twice. This is the defining feature. The pairs must be linked — you cannot arbitrarily pair subjects from two groups.
The differences are about normal. With n = 9, moderate non-normality in the D column is acceptable, but severe skewness or outliers can mislead the test.
The pairs are independent of each other. Trainee 1's before/after measurements must be independent of Trainee 2's. If trainees trained together and influenced each other, independence is violated.

What breaks when assumptions fail: Non-normal differences with small n → Wilcoxon signed-rank test. Dependent pairs (e.g., clustered data) → mixed-effects model. If the pairing is artificial (you matched subjects post-hoc), the test is invalid — use a two-sample test instead.

17.3.7 Visual Intuition — The D Column Tells the Story

Picture two side-by-side histograms on the same axes (score, 0–100). The left histogram shows the "Before" distribution in blue — it is centered around, say, 63. The right histogram shows the "After" distribution in green — it is centered around 68. They overlap heavily.

The key insight: do not stare at these two histograms. Instead, plot the difference column as a single histogram. This D histogram is centered at +5 and spread from roughly +2 to +7. The question is simple: does this single histogram's center sit significantly away from zero?

A vertical dashed line at D = 0 marks the null hypothesis. The observed D̄ = 5 sits far to the right. The t-statistic of 6.32 means D̄ is more than 6 standard errors away from zero — the D histogram and the zero line barely overlap.

17.3.8 Pitfalls

Trap 1 — Using a two-sample t-test on paired data. This is the most common error. If you run a two-sample t-test comparing the "Before" column (mean ≈ 63.4) to the "After" column (mean ≈ 68.4), you lose the pairing. The two-sample test treats the data as if 9 random people scored 63 and 9 different random people scored 68. The within-person correlation is lost, and the test loses power — you might fail to detect a real effect. Trap 2 — Forgetting the D = Y − X step. Some students try to test H₀: μ_Y = μ_X directly. You must compute the difference column. The test is on μ_D, not on μ_Y and μ_X separately. Trap 3 — Confusing direction. D = After − Before. If the intervention helps, D > 0 and you use a right-tailed test. If the problem defines D = Before − After, then improvement means D < 0 and you use a left-tailed test. Always check how D is defined in the problem. Trap 4 — Assuming μ_D is always 0. In this course, H₀: μ_D = 0 is the standard. But in some contexts, you might test H₀: μ_D = 5 (the training must improve scores by at least 5 points to be cost-effective). The problem statement tells you what μ_D to use.

17.3.9 Student Questions and Answers

Q: Why is μ_D taken as zero? A: D is the difference between after and before scores. If the training does nothing, the before and after scores are the same on average, so the mean difference is zero. If the training helps, the after scores are higher, so μ_D > 0. Testing μ_D = 0 versus μ_D > 0 directly answers "is the program effective?" A zero difference means no effect.

Q: Can the paired t-test ever be two-tailed? A: Yes. If the problem asks "is there any change" or "does the medicine have any impact" without specifying direction, then H₁: μ_D ≠ 0. That is a two-tailed test. Medicine can help or harm. Training can improve or confuse. The problem statement decides.

Q: How do we know this is paired and not a two-sample problem? (Several students asked this.) A: Look for "before and after" on the same subjects. If the problem says "before the training, these 9 people scored X" and "after the training, the same 9 people scored Y," it is paired. If it says "group A scored X and group B scored Y" with different people in each group, it is a two-sample problem. The key phrase is "same sample." The professor emphasized: "In two means problem, the two populations are different — different products, different groups. In this case, same sample before and after — so we should not consider it a two-population problem."

17.3.10 Exam Guidance

Exam note: This test is easy to miss because it looks like a two-sample problem at first glance. Always check: are the measurements on the same subjects? If yes → paired t-test. The computation is just a one-sample t-test on the D column. State df = n − 1 explicitly. The t-table will be provided.

17.3.11 Recap and Bridge

The paired t-test reduces to a one-sample t-test on the difference column. D = Y − X, test μ_D = 0, and you are done. The only hard part is recognizing the paired structure. Next: what if you have three or more groups? That is ANOVA — a different beast entirely.

17.3.12 Real-World & Domain Connection

Pharmaceutical clinical trials — including the Pfizer and Moderna COVID-19 vaccine trials — use paired designs extensively. Antibody levels are measured in the same patients before vaccination and two weeks after. The paired t-test compares the mean increase in antibody titers against zero. A significant positive difference is evidence the vaccine triggers an immune response. The same design is used in psychology (pre-test/post-test for therapy effectiveness), education (measuring student knowledge before and after a course), and sports science (VO₂ max before and after a training regimen). Anywhere you measure the same unit twice and ask "did something change?", the paired t-test is the tool.

17.4 One-Way ANOVA

17.4.1 Hook — Why Not Just Run Multiple t-Tests?

You have three fertilizers and you want to know which one gives the best crop yield. You could run three t-tests: A vs B, B vs C, A vs C. But here is the trap: every test has a 5% chance of a false positive. Run three tests, and your overall chance of at least one false positive jumps to about 14%. ANOVA solves this — it tests all three means simultaneously with a single 5% error rate.

17.4.2 Intuition — The Sports Team Height Analogy

You measure the heights of players on a basketball team, a jockey club, and a chess club. The basketball players are tall (mean ~200 cm), jockeys are short (mean ~160 cm), and chess players are somewhere in between. But within each team, heights also vary — not every basketball player is exactly 200 cm.

ANOVA asks: is the variation between the team averages large compared to the variation within each team? If basketball players tower over jockeys by an amount that dwarfs the height spread within each team, the difference is real. If the team averages differ by only a few cm while individual heights vary by 30 cm, the difference could be noise.

The analogy breaks when: teams have vastly different within-group spreads (violating homogeneity of variance). ANOVA assumes the within-group variability is similar across groups.

17.4.3 Symbols

Symbol	Meaning	Type	Domain
k	number of groups	integer	k ≥ 3
N	total number of observations across all groups	integer	N > k
X̄ⱼ	mean of group j	scalar	real
X̄	overall (grand) mean	scalar	real
SSTR	treatment sum of squares (between groups)	scalar	nonnegative
SSE	error sum of squares (within groups)	scalar	nonnegative
SST	total sum of squares = SSTR + SSE	scalar	nonnegative
MSTR	mean square treatment = SSTR/(k−1)	scalar	nonnegative
MSE	mean square error = SSE/(N−k)	scalar	nonnegative
F	F-statistic = MSTR/MSE	scalar	nonnegative

17.4.4 Mathematical Formulation

ANOVA partitions the total variation into two pieces: variation between groups (the treatment effect) and variation within groups (random noise).

Between-group variation (SSTR):

S S T R = j = 1 \sum k n_{j} (X ˉ_{j} - X ˉ)^{2}

For each group j, take the group mean X̄ⱼ, subtract the grand mean X̄, square it, multiply by the group size nⱼ, and sum. This captures how far each group's average sits from the overall average.

Within-group variation (SSE):

S S E = j = 1 \sum k i = 1 \sum n_{j} (X_{ij} - X ˉ_{j})^{2}

For each observation, subtract its own group mean, square, and sum. This captures how much individual observations scatter around their group center.

Total variation:

S S T = S S T R + S S E

Mean squares and the F-statistic:

M S T R = \frac{S S T R}{k - 1}, M S E = \frac{S S E}{N - k}

F = \frac{M S T R}{M S E}

The F-statistic is the ratio of between-group variation to within-group variation. If the groups truly have different means, the numerator (MSTR) will be large relative to the denominator (MSE), and F will be large.

Degrees of freedom:

Numerator df = k − 1 (for SSTR)
Denominator df = N − k (for SSE)

The F-table needs both. The professor emphasized: "When we go with the F distribution table, we go with level of significance and two degrees of freedom. Numerator and denominator. K minus 1 is numerator; N minus K is denominator."

17.4.5 Two Computational Approaches

There are two equivalent ways to compute the ANOVA sums of squares:

Method 1 — Direct computation: Compute SSTR and SSE from the formulas above. Then SST = SSTR + SSE. Method 2 — Using the correction factor: First compute SST from raw data using

S S T = \sum X^{2} - G^{2} / N

, where G is the grand total of all observations. Then compute SSTR, and find SSE = SST − SSTR.

Both methods give identical results. The professor said: "You can employ any of the two approaches."

17.4.6 The ANOVA Table

Source	SS	df	MS	F
Treatment (Between)	SSTR	k−1	MSTR = SSTR/(k−1)	F = MSTR/MSE
Error (Within)	SSE	N−k	MSE = SSE/(N−k)
Total	SST	N−1

17.4.7 Worked Example — Three-Group Comparison

Setup: Three teaching methods are tested. Student scores:

Method A	Method B	Method C
8	6	10
10	8	12
7	10	14
12	—	—

Group sizes: n₁ = 4, n₂ = 3, n₃ = 3. Total N = 10.

Group means: X̄₁ = (8+10+7+12)/4 = 9.25, X̄₂ = (6+8+10)/3 = 8, X̄₃ = (10+12+14)/3 = 12. Grand mean: X̄ = (8+10+7+12+6+8+10+10+12+14)/10 = 97/10 = 9.7. SSTR:

S S T R = 4 (9.25 - 9.7)^{2} + 3 (8 - 9.7)^{2} + 3 (12 - 9.7)^{2}

= 4 (0.2025) + 3 (2.89) + 3 (5.29) = 0.81 + 8.67 + 15.87 = 25.35

SSE:

Method A: (8−9.25)²+(10−9.25)²+(7−9.25)²+(12−9.25)² = 1.5625+0.5625+5.0625+7.5625 = 14.75
Method B: (6−8)²+(8−8)²+(10−8)² = 4+0+4 = 8
Method C: (10−12)²+(12−12)²+(14−12)² = 4+0+4 = 8

S S E = 14.75 + 8 + 8 = 30.75

SST check: SST = SSTR + SSE = 25.35 + 30.75 = 56.10. Mean squares: MSTR = 25.35/2 = 12.675, MSE = 30.75/7 = 4.393. F-statistic: F = 12.675/4.393 ≈ 2.885. Critical value: At df₁ = 2, df₂ = 7, α = 0.05: F_table ≈ 4.74. Conclusion: F_calculated = 2.885 < F_critical = 4.74 → Do not reject H₀. The three teaching methods do not show significantly different means at α = 0.05. Sense-check: Method C's mean (12) looks higher than A (9.25) and B (8), but the within-group scatter is large enough that the between-group differences could be noise. Method A's scores range from 7 to 12. The F of 2.885 is below the threshold.

17.4.8 Decision Rule and the F Distribution

Unlike Z and t, the F distribution is not symmetric — it is bounded at zero and skewed right. The rejection region is always on the right tail.

Rule: If F_calculated > F_table → reject H₀ (at least one group mean differs).

ANOVA's H₀: μ₁ = μ₂ = … = μ_k (all group means equal). H₁: at least one mean differs.

17.4.9 Assumptions and Scope

Scope: ANOVA assumes:

Normality within each group. The observations in each group come from a normal distribution. ANOVA is somewhat strong to moderate non-normality, especially with balanced designs (equal n per group).
Homogeneity of variances. All groups have the same population variance. If group variances differ wildly (e.g., one group has s² = 100, another s² = 5), the F-test's error rate is affected. Levene's test can check this.
Independence. Observations are independent within and across groups.

What breaks when assumptions fail: Non-normality + small n → Kruskal-Wallis test. Unequal variances → Welch's ANOVA. Dependent data → repeated-measures ANOVA.

17.4.10 Visual Intuition — Between vs. Within

Picture three bell curves on the same x-axis (test scores, 0–20). The blue curve (Method A) is centered at 9.25, the red curve (Method B) at 8, the green curve (Method C) at 12. All three curves have similar width (within-group variation).

Now imagine drawing a horizontal line at the grand mean (9.7). The SSTR measures how far each curve's center is from this line — weighted by group size. The SSE measures how wide each curve is. The F-statistic is essentially: (spread of the curve centers) ÷ (average width of the curves).

If the three curves sit far apart relative to their widths, F is large and significant. If they overlap heavily, F is small.

17.4.11 Pitfalls

Trap 1 — Running multiple t-tests instead of ANOVA. Three groups → one ANOVA, not three t-tests. The professor was explicit about this. Each extra test inflates your familywise Type I error rate. Trap 2 — Confusing SSTR and SSE. SSTR = between groups (think "T" for treatment). SSE = within groups (think "E" for error). Mixing them up swaps the F numerator and denominator. Trap 3 — Using the wrong df. F-table lookup needs TWO degrees of freedom: numerator = k−1, denominator = N−k. Using N−1 for both is wrong. Trap 4 — Interpreting a significant F as "all groups differ." A significant ANOVA tells you at least one group mean differs from the others. It does not tell you which pairs differ. For that, you need post-hoc tests (Tukey's HSD, Bonferroni) — which are beyond the scope of this course.

17.4.12 Student Q&A

Q: Can we draw graphs for the F-test like we do for Z and t? A: The F distribution is not symmetric. You cannot draw left-tail and right-tail regions the way you can for Z and t. The rejection region is always on the right side — you reject when the calculated F exceeds the table value. For chi-square, the situation is similar: the distribution is not symmetric, and rejection happens on the right.

17.4.13 Exam Guidance

Exam note: ANOVA identification is the key challenge. If the problem mentions three or more groups with means → ANOVA. Do not run multiple t-tests. The F-table will be provided. You need two degrees of freedom to look up the critical value: numerator df = k−1 and denominator df = N−k.

17.4.14 Recap and Bridge

ANOVA compares three or more means with a single test. The F-statistic = between-group variation ÷ within-group variation. Large F → means differ. Next: what if your data is not means but counts in categories? That is the chi-square test.

17.4.15 Real-World & Domain Connection

Agricultural field trials at companies like Bayer (formerly Monsanto) use ANOVA daily. A typical trial tests 5–10 fertilizer formulations on separate plots, measuring crop yield. ANOVA determines whether any formulation outperforms the others. The same method is used in pharmaceutical dose-response studies (comparing 3+ dosage levels of a drug), industrial quality control (comparing output from multiple production lines), and marketing (comparing customer satisfaction across 4+ store formats). Anywhere you compare a numerical outcome across three or more categories, ANOVA is the default statistical tool.

17.5 Chi-Square Test of Independence

You survey 180 people. You record whether they smoke (non-smoker, moderate, heavy) and whether they have hypertension (yes, no). The data sits in a 3×2 grid of counts. The question: are smoking and hypertension independent, or are they associated? The chi-square test of independence answers this with a single number.

17.5.2 Intuition — The Seating Preference Analogy

On a flight, you record whether each passenger chose a window or aisle seat, and whether they are under 30 or over 30. If seat choice has nothing to do with age, then the proportion of window-choosers should be about the same in both age groups. Any difference is just random noise.

The chi-square test makes this precise. For each cell in the table, it computes an expected count — what you would see if the two variables were independent. Then it measures how far the observed counts are from these expected counts. A large total discrepancy → the variables are associated.

The analogy breaks when: expected counts are too small (less than 5). The chi-square approximation degrades, and you need Fisher's exact test instead.

17.5.3 Symbols

Symbol	Meaning	Type	Domain
O_ij	observed frequency in cell (i,j)	integer	nonnegative
E_ij	expected frequency in cell (i,j)	scalar	positive
R_i	row total for row i	integer	nonnegative
C_j	column total for column j	integer	nonnegative
N	grand total	integer	positive
r	number of rows	integer	r ≥ 2
c	number of columns	integer	c ≥ 2
χ²	chi-square statistic	scalar	nonnegative
df	degrees of freedom = (r−1)(c−1)	integer	positive

17.5.4 Mathematical Formulation

Expected frequency for each cell — the independence baseline:

E_{ij} = \frac{R _{i} \times C _{j}}{N}

If smoking and hypertension are independent, the proportion of people with hypertension should be the same across all smoking categories. So for cell (i,j), multiply the row total by the column total and divide by the grand total. This gives the count you would expect under independence.

Chi-square statistic — measuring the discrepancy:

χ^{2} = i = 1 \sum r j = 1 \sum c \frac{( O _{ij} - E _{ij} ) ^{2}}{E _{ij}}

For every cell: take observed minus expected, square it, divide by expected, and sum across all cells. The squaring makes all deviations positive. The division by E_ij scales the contribution — a deviation of 5 is more surprising when you expected 2 than when you expected 50.

Degrees of freedom:

df = (r - 1) (c - 1)

. Hypotheses:

H₀: The two variables are independent (no association)
H₁: The two variables are not independent (there is an association)

There is no "one-tail vs. two-tail" confusion here. The chi-square distribution is not symmetric, and rejection is always on the right.

17.5.5 Worked Example — Hypertension and Smoking

Setup: A study examines whether hypertension is related to smoking. Data on 180 people: Observed frequencies (3×2 table):

Smoking Status	Hypertension	No Hypertension	Row Total
Heavy Smokers	21	48	69
Moderate Smokers	0	0	0
Non-smokers	66	45	111
Column Total	87	93	180

The "Moderate Smokers" row has zero observations in this dataset — no moderate smokers were sampled. The chi-square computation still includes this row, but its contribution is zero.

Step 1 — Compute expected frequencies:

For Heavy Smokers × Hypertension:

E_{11} = \frac{69 \times 87}{180} = \frac{6003}{180} = 33.35

For Heavy Smokers × No Hypertension:

E_{12} = \frac{69 \times 93}{180} = \frac{6417}{180} = 35.65

For Non-smokers × Hypertension:

E_{21} = \frac{111 \times 87}{180} = \frac{9657}{180} = 53.65

For Non-smokers × No Hypertension:

E_{22} = \frac{111 \times 93}{180} = \frac{10323}{180} = 57.35

The Moderate Smokers row has expected values of zero (row total = 0).

Step 2 — Compute χ²:

χ^{2} = \frac{( 21 - 33.35 ) ^{2}}{33.35} + \frac{( 48 - 35.65 ) ^{2}}{35.65} + \frac{( 66 - 53.65 ) ^{2}}{53.65} + \frac{( 45 - 57.35 ) ^{2}}{57.35} = \frac{152.52}{33.35} + \frac{152.52}{35.65} + \frac{152.52}{53.65} + \frac{152.52}{57.35} = 4.57 + 4.28 + 2.84 + 2.66 \approx 14.35

The professor stated χ² = 14.46. The small difference (about 0.11) is due to rounding in intermediate expected frequency calculations.

Step 3 — Find the critical value:

df = (r−1)(c−1) = (3−1)(2−1) = 2. At α = 0.05, χ²_table = 5.99 from the chi-square table.

Step 4 — Decision:

χ²_calculated ≈ 14.35 > χ²_table = 5.99 → Reject H₀.

There is a significant association between smoking status and hypertension. The variables are not independent.

Sense-check: Look at the data: 21/69 ≈ 30.4% of heavy smokers have hypertension, while 66/111 ≈ 59.5% of non-smokers have hypertension. These proportions are quite different — the test confirms this is unlikely under independence.

17.5.6 The Universal Decision Rule

For all hypothesis tests — Z, t, F, chi-square — the rule is the same:

calculated value < table value → do not reject H₀
calculated value > table value → reject H₀

For chi-square: accepting H₀ means the variables are independent. Rejecting H₀ means they are associated (dependent). The professor emphasized: "If it falls here [left of critical], then accept. This is applicable to everything."

17.5.7 Assumptions and Scope

Scope: The chi-square test of independence assumes:

Categorical variables. Both variables must be categorical (nominal or ordinal). If one variable is continuous, it must be binned first.
Expected frequencies ≥ 5 per cell. This is a rule of thumb. If many cells have E_ij < 5, the chi-square approximation is unreliable — use Fisher's exact test.
Independent observations. Each person contributes to exactly one cell. No repeated measures.
The data are counts, not percentages. You need the raw frequencies.

What breaks when assumptions fail: Small expected frequencies → Fisher's exact test or collapse categories. Dependent data → McNemar's test (for paired categorical data). Continuous variables → logistic regression.

17.5.8 Visual Intuition — The Observed vs. Expected Grid

Picture two 3×2 grids side by side. The left grid shows the observed counts — the raw data. The right grid shows the expected counts — what you would see if smoking and hypertension were independent.

In the observed grid, the Heavy Smokers cell for Hypertension reads "21." In the expected grid, the same cell reads "33.35." The difference (21 − 33.35 = −12.35) is the surprise — fewer heavy smokers have hypertension than independence would predict.

Now scan across all cells. In every cell, the observed count deviates from the expected by exactly ±12.35 (with opposite signs in opposite corners — this is a property of 2×2 sub-tables). The chi-square statistic aggregates these deviations across all cells. A large χ² means the two grids look very different; a small χ² means they look similar.

17.5.9 Pitfalls

Trap 1 — Misidentifying chi-square as ANOVA. Several proportions → chi-square. Several means → ANOVA. The chi-square problem has count data in categories. The ANOVA problem has numerical measurements. Trap 2 — Confusing H₀ and H₁. H₀ is always "the variables are independent." H₁ is always "they are not independent." There is no "greater than" or "less than" in the hypotheses — the test is inherently two-sided (the chi-square distribution only has a right tail). Trap 3 — Computing E_ij wrong. Remember: (row total × column total) ÷ grand total. A common error is dividing by the row total or column total instead. Trap 4 — Forgetting df = (r−1)(c−1). A 3×2 table has df = 2, not df = 3×2 = 6. The formula subtracts 1 from each dimension before multiplying.

17.5.10 Student Q&A

Q: Does the rejection rule apply the same way for all tests? (Several students asked this.) A: Yes. If the calculated value is less than the table value, do not reject the null hypothesis. If it is greater, reject. This is true for Z, t, F, and chi-square. The only difference is that for Z and t, the rejection region can be on the left, right, or both tails. For F and chi-square, it is always on the right because these distributions are not symmetric.

Q: How do we identify chi-square versus ANOVA? A: Several proportions → chi-square. Several means → ANOVA. The chi-square problem will have categories in rows and columns with count data (frequencies). The ANOVA problem will have numerical measurements across groups.

17.5.11 Exam Guidance

Exam note: Chi-square is the simplest computation among all hypothesis tests. The only challenge is identification. Once you spot "multiple proportions" in a table, the rest is formula substitution. H₀ is always "the variables are independent." H₁ is always "they are not independent." There is no confusion about one-tail or two-tail. The chi-square table will be provided.

17.5.12 Recap and Bridge

Chi-square tests independence between two categorical variables. E_ij = (R_i × C_j)/N, χ² = Σ(O−E)²/E, df = (r−1)(c−1). Reject when calculated > table. Next: we leave hypothesis testing and enter time series — forecasting with exponential smoothing.

17.5.13 Real-World & Domain Connection

The CDC (Centers for Disease Control) uses chi-square tests extensively in epidemiological studies. The classic example is the 1950s studies linking smoking to lung cancer — chi-square tests on contingency tables established the association that launched modern public health policy. Today, the same test is used in clinical trials to compare adverse event rates across treatment arms, in marketing analytics to test whether purchase behavior is independent of demographic categories, and in A/B testing for conversion rates across multiple variants. Anywhere you have categorical outcomes in a table, chi-square is the first tool you reach for.

17.6 Simple Exponential Smoothing

17.6.1 Hook — Predicting Next Week's Sales

You have six weeks of sales data. Week 7 is coming. What is your best guess? You could take the simple average of all six weeks — but that treats week 1 and week 6 as equally informative. Surely last week matters more? Exponential smoothing gives you a knob — α — that controls how much you trust the latest observation versus the historical pattern.

17.6.2 Intuition — The Weather App Analogy

Your weather app says tomorrow will be 22°C. How does it make that forecast? It looks at today's actual temperature (say, 24°C) and yesterday's forecast for today (say, 21°C). It blends them: maybe 70% weight on what actually happened today and 30% on what it previously thought.

That is exponential smoothing. The smoothing constant α is the weight you put on the latest actual value. α = 0.8 means "trust today's reading a lot." α = 0.3 means "be cautious — the long-run average matters more."

The analogy breaks when: the data has an upward trend or seasonal pattern. Simple exponential smoothing only works for flat (stationary) series. For trend, you need double exponential smoothing (Holt's method).

17.6.3 Symbols

Symbol	Meaning	Type	Domain
Y_t	actual value at time t	scalar	real
F_t	forecast for time t	scalar	real
F_{t+1}	forecast for next period	scalar	real
α	smoothing constant	scalar	[0, 1]
e_t = Y_t − F_t	forecast error at time t	scalar	real

17.6.4 Mathematical Formulation

The smoothing equation:

F_{t + 1} = α Y_{t} + (1 - α) F_{t}

The forecast for tomorrow is a weighted average of today's actual value and today's forecast. The weights are α and (1−α), and they always sum to 1.

Expanding the recurrence — why it is called "exponential":

F_{t + 1} = α Y_{t} + (1 - α) F_{t} = α Y_{t} + (1 - α) [α Y_{t - 1} + (1 - α) F_{t - 1}] = α Y_{t} + α (1 - α) Y_{t - 1} + (1 - α)^{2} F_{t - 1} = α Y_{t} + α (1 - α) Y_{t - 1} + α (1 - α)^{2} Y_{t - 2} + \dots

The weights on past observations decay exponentially: α, α(1−α), α(1−α)², … The most recent observation gets weight α, the one before gets α(1−α), and so on. For α = 0.3, the weights are 0.3, 0.21, 0.147, 0.103, … — each step multiplies by 0.7.

Initialization: F₁ = Y₁. The first forecast equals the first actual value because you have no prior forecast to use. The professor emphasized: "First week we need to keep same value as the forecast." Choosing α:

α close to 1 → responsive but noisy (trusts latest data)
α close to 0 → stable but sluggish (trusts history)

17.6.5 Worked Example — Weekly Forecasting

Setup: Forecast week 7 using α = 0.3 and α = 0.8. Actual values (from the slide):

Week	1	2	3	4	5	6	7
Y_t	112	118	125	132	138	144	142

Model 1 — α = 0.3 (stable, slow):

Week	Y_t	F_t	Computation
1	112	112.00	F₁ = Y₁
2	118	112.00	0.3×112 + 0.7×112.00
3	125	113.80	0.3×118 + 0.7×112.00
4	132	117.16	0.3×125 + 0.7×113.80
5	138	121.61	0.3×132 + 0.7×117.16
6	144	126.53	0.3×138 + 0.7×121.61
7	—	131.77	0.3×144 + 0.7×126.53

Model 2 — α = 0.8 (responsive, fast):

Week	Y_t	F_t	Computation
1	112	112.00	F₁ = Y₁
2	118	112.00	0.8×112 + 0.2×112.00
3	125	116.80	0.8×118 + 0.2×112.00
4	132	123.36	0.8×125 + 0.2×116.80
5	138	130.27	0.8×132 + 0.2×123.36
6	144	136.45	0.8×138 + 0.2×130.27
7	—	142.49	0.8×144 + 0.2×136.45

The professor's slide data gave F₇ = 127.87 for α = 0.3 and F₇ = 139.26 for α = 0.8 with actual week 7 = 142. The procedure is identical; only the raw Y_t values differ.

Model comparison:

Model	F₇	Actual Y₇	Absolute Error
α = 0.3	127.87	142	14.13
α = 0.8	139.26	142	2.74

Conclusion: Model 2 (α = 0.8) is better — its forecast is closer to the actual value. The higher α gives more weight to recent observations, which works well when the series is trending upward. Sense-check: With α = 0.3, the forecast evolves slowly — by week 7, it is still heavily influenced by early low values. With α = 0.8, the forecast chases the trend. Since the actual series is rising, α = 0.8 wins.

17.6.6 Model Selection — Beyond One Forecast

You can formalize model comparison by computing forecast errors for all periods:

MSE = \frac{1}{n} t = 1 \sum n (Y_{t} - F_{t})^{2}

The model with the smaller Mean Squared Error (MSE) is better overall. This is more reliable than comparing a single forecast.

17.6.7 Assumptions and Scope

Scope: Simple exponential smoothing assumes:

No trend. The series should be roughly flat (stationary). If there is a consistent upward or downward trend, use double exponential smoothing (Holt's method).
No seasonality. The series should not have regular seasonal patterns (e.g., higher in December every year). For seasonality, use Holt-Winters.
α ∈ [0, 1]. Outside this range, the weights do not form a proper weighted average.

What breaks when assumptions fail: Trend → forecasts systematically lag behind. Seasonality → forecasts miss regular peaks and troughs. Both are fixable with more advanced exponential smoothing variants.

17.6.8 Visual Intuition — The Two Forecast Paths

Picture a time series plot: weeks 1–6 on the x-axis, values (Y_t) as blue dots connected by a blue line, rising from 112 to 144. Now overlay two forecast lines:

The orange line (α = 0.3) is smooth and sluggish — it starts at 112 and slowly drifts upward, always below the actual values. By week 6, it has only reached about 127. It is a cautious, slow-moving average.

The green line (α = 0.8) is jagged and responsive — it hugs the actual data closely, rising quickly when the series rises. By week 6, it sits near 136, much closer to the actual 144.

The vertical gap at week 7 between the forecast and the actual (142) tells the story: the green line is much closer. Higher α wins when the series has momentum.

17.6.9 Pitfalls

Trap 1 — Forgetting F₁ = Y₁. If you set F₁ = 0 or some arbitrary value, the first few forecasts will be badly biased. The initialization matters. Trap 2 — Thinking α = 0.5 is always best. The optimal α depends on the data. Fast-changing series need high α; stable series need low α. There is no universal default. Trap 3 — Confusing α with confidence level. α here is a smoothing constant, not a significance level. Same symbol, completely different meaning. Trap 4 — Extrapolating too far. Exponential smoothing is for short-term forecasting (one period ahead). Forecasting 10 periods ahead with a flat model is essentially just repeating the last forecast — you have no trend to project.

17.6.10 Student Q&A

Q: Do we need to calculate forecast errors for all weeks? A: You can. Computing errors for all available weeks gives a stronger basis for comparison than just looking at week 7. The model with smaller errors overall is the better one. But in an exam, comparing the week 7 forecast against the given actual value is enough to draw a conclusion. The professor said the more thorough approach uses all errors, but the single-point comparison is acceptable for exam purposes.

17.6.11 Exam Guidance

Exam note: This is formula-based. Remember F₁ = Y₁. Remember (1−α) is the weight on the old forecast. Higher α = more responsive model. You may be asked to forecast one period ahead and compare two α values. Show all your work in a table — it keeps you organized.

17.6.12 Recap and Bridge

Simple exponential smoothing blends the latest observation with the previous forecast. Higher α trusts the latest data more. It works for flat series without trend or seasonality. Next: what if a time series has no pattern at all? That is white noise — and you should not try to forecast it.

17.6.13 Real-World & Domain Connection

Amazon's inventory management systems use exponential smoothing to forecast demand for millions of products. Fast-moving electronics (like a new iPhone case) get high α values — the model reacts quickly to demand spikes. Stable grocery items (like canned beans) get low α values — the model smooths out random fluctuations. Walmart and Target use similar systems. The choice of α is automated through optimization: the system tries many α values and picks the one that minimizes historical forecast error. This is the same logic you use when comparing α = 0.3 vs. α = 0.8 — just scaled to millions of products.

17.7 Autocorrelation and White Noise

17.7.1 Hook — Can You Predict This?

Here is a sequence: 3, 2, 1, 0, −1, −2, −1, 0, 1, 2, 3. Can you predict the next number? You probably can — it follows a pattern (it went down, then up). Now imagine TV static. Each pixel's brightness is random. No matter how long you stare, the next pixel is unpredictable. That second sequence is white noise. If your data is white noise, stop trying to forecast it — there is nothing to model.

17.7.2 Intuition — The TV Static Analogy

Tune your TV to a dead channel. The flickering "snow" on the screen is white noise. It has three properties. (1) The average brightness is neutral gray — mean zero if we center it. (2) The intensity of the flickering stays constant over time — constant variance. (3) Knowing the brightness of one pixel tells you nothing about the next pixel — no autocorrelation.

In time series terms, a white noise series is the residual left over after you have extracted all the signal — trend, seasonality, cycles. If your forecast errors look like white noise, you have captured everything. If they show a pattern, your model is missing something.

The analogy breaks when: real-world "white noise" is an idealization. Physical processes always have some tiny correlation at very short lags. But for statistical purposes, if the autocorrelation is within the ±2/√T confidence band, the series is effectively white noise.

17.7.3 Definition — Autocorrelation

Autocorrelation is correlation of a time series with a lagged version of itself:

ρ_{k} = Corr (X_{t}, X_{t + k})

Regular correlation measures the relationship between two different variables (X and Y). Autocorrelation measures the relationship between X at time t and X at time t+k — the same variable, separated by k time steps.

For lag k = 1: are adjacent values correlated? For lag k = 2: are values two steps apart correlated? And so on.

Autocovariance is the same idea applied to covariance. Autocorrelation = autocovariance divided by variance:

ρ_{k} = \frac{Cov ( X _{t} , X _{t + k} )}{Var ( X _{t} )}

17.7.4 Definition — White Noise

A time series is white noise if it satisfies three conditions:

Mean = 0. The series fluctuates around zero. If the mean is nonzero, subtract it first — the resulting series should be white noise.

Constant variance. The spread of the values does not grow, shrink, or oscillate over time. Formally, $Var (X_{t}) = σ^{2}$ for all t.

No autocorrelation. $ρ_{k} = 0$ for all lags k ≥ 1. Knowing X_t gives you zero information about X_{t+1}, X_{t+2}, or any future value.

If a time series has any detectable pattern, trend, or correlation across time, it is not white noise.

The professor summarized: "Time series without autocorrelation — we call it white noise. Mean zero, constant variance, and no pattern, then it is white noise."

17.7.5 Why White Noise Matters

If a time series is white noise, it cannot be forecast. There is no structure to model — the best forecast for all future periods is simply zero (or the series mean). Identifying white noise prevents you from wasting time and resources trying to predict the unpredictable. In practice, you check whether your model's residuals (forecast errors) are white noise. If they are, your model has extracted all available signal.

17.7.6 Worked Example — Is This White Noise?

Series: 3, 2, 1, 0, −1, −2, −1, 0, 1, 2, 3 Check 1 — Mean zero?

X ˉ = \frac{3 + 2 + 1 + 0 + ( - 1 ) + ( - 2 ) + ( - 1 ) + 0 + 1 + 2 + 3}{11} = \frac{8}{11} \approx 0.73

The mean is close to zero but not exactly zero. For practical purposes with a longer series, we could center it. But the pattern check will be decisive regardless.

Check 2 — Constant variance?

With the mean about zero, each deviation X_i − X̄ ≈ X_i. The squared values are 9, 4, 1, 0, 1, 4, 1, 0, 1, 4, 9. The variance is roughly constant — the series does not show increasing or decreasing spread over time. (The professor assessed this visually in class; a formal test like the Breusch-Pagan test would be used in practice.)

Check 3 — No pattern?

Look at the sequence: it decreases from 3 to −2, then increases back to 3. This is a clear V-shaped pattern. There is structure — you can predict the next value from the trend. ✗

Conclusion: This series has a pattern and is not white noise. Even though it passes the mean and variance checks about, the autocorrelation check fails decisively. You can see the pattern by eye. Sense-check: If this were white noise, the sequence 3,2,1,0,−1 would be just as likely as 3,−1,2,−2,0. But the first sequence shows clear decreasing order — that is autocorrelation.

17.7.7 Formal White Noise Test — The Confidence Band

A more formal approach uses the autocorrelation function (ACF). For a white noise series of length T, about 95% of the sample autocorrelations $ρ^_{k}$ should fall within:

\pm \frac{2}{T}

For T = 100, the band is ±0.2. For T = 25, the band is ±0.4.

If more than 5% of the autocorrelation spikes fall outside this band — or if any single spike is far outside — the series is likely not white noise. The professor described this as: "We calculate plus or minus 2 by root T where T is the length of the time series."

17.7.8 Assumptions and Scope

Scope: The white noise concept is a theoretical ideal. In practice:

The ±2/√T rule is approximate. It works for large T. For small T, the band is wider, and formal tests (Ljung-Box) are more reliable.
White noise ≠ independent. White noise requires zero correlation. It does not require independence — there could be nonlinear dependence (e.g., squared values are correlated) while linear correlation is zero. Financial returns often show this pattern.
Gaussian white noise adds the assumption of normality. This is the strictest form.

What breaks when assumptions fail: If residuals are not white noise, your model is misspecified. Add trend terms, seasonal terms, or switch to a different model class.

17.7.9 Visual Intuition — The ACF Plot

Picture a bar chart. The x-axis shows lag k = 1, 2, 3, …, 10. The y-axis shows autocorrelation from −1 to +1. Two horizontal dashed lines sit at +2/√T and −2/√T — the confidence band.

For a white noise series, most bars are short — they hover near zero, with maybe one or two poking slightly above the band by random chance. The plot looks like a city skyline after an earthquake — everything flattened.

For a non-white-noise series, the first few bars are tall. Lag 1 might be 0.8, lag 2 might be 0.6, lag 3 might be 0.4 — they decay gradually. This is the signature of autocorrelation: today's value is highly predictive of tomorrow's.

17.7.10 Pitfalls

Trap 1 — Judging white noise by eye alone. A series can "look random" but have subtle autocorrelation. Always check the ACF or the ±2/√T rule. Conversely, a series can look patterned (clusters of high values) but have zero autocorrelation if the clustering is nonlinear. Trap 2 — Confusing constant variance with constant values. Constant variance means the spread is stable, not that every value is the same. The series 1, −1, 1, −1, … has constant variance but is not white noise (it has perfect negative autocorrelation at lag 1). Trap 3 — Forgetting to center the series. White noise requires mean zero. If your series has mean 5, subtract 5 from every value before checking the other conditions. A student noted: "Since X bar is 0, subtracting the value gives the same number." This reasoning is correct when μ = 0, but only then. Trap 4 — Assuming "no autocorrelation" means "no relationship." Zero autocorrelation means zero linear relationship across time. There could still be nonlinear dependence (e.g., volatility clustering in stock returns, where large moves follow large moves but the direction is unpredictable).

17.7.11 Student Q&A

Q: How do we know the variance is constant in the example when X_i values change? A: When the mean is zero (or about zero), the deviation of every point from the mean is just the value itself. Since the squared deviations follow a symmetric pattern that does not grow, shrink, or oscillate over time, the variance stays constant. In the example, the squared values are symmetric (9, 4, 1, 0, 1, 4, 1, 0, 1, 4, 9) — there is no systematic increase or decrease. A student in class noted: "Since X bar is 0, subtracting the value gives the same number. The deviation remains the same." The professor confirmed this reasoning is correct for this specific example.

17.7.12 Exam Guidance

Exam note: White noise is a conceptual topic. You may be asked to determine whether a given series qualifies as white noise by checking the three conditions: mean zero, constant variance, no autocorrelation (no pattern). The confidence band formula ±2/√T may appear in a more detailed question.

17.7.13 Recap and Bridge

White noise = mean zero + constant variance + no autocorrelation. If your series is white noise, you cannot forecast it — there is no signal to extract. In practice, you test whether your model residuals are white noise to confirm you have captured all the structure. Next: a completely different topic — Gaussian Mixture Models, where we model data as coming from multiple overlapping normal distributions.

17.7.14 Real-World & Domain Connection

Bose and Sony noise-canceling headphones use white noise characterization at their core. The headphones have microphones that sample ambient sound. The noise is modeled as a stochastic process. If the ambient noise is about white (equal energy across all frequencies), the noise-canceling circuitry generates an inverted signal that is straightforward to compute. In finance, testing whether stock returns are white noise is a fundamental check — if returns were predictable (not white noise), arbitrage opportunities would exist. The efficient market hypothesis essentially claims that asset returns should be white noise (or close to it) after adjusting for risk. In signal processing, white noise is the null model: any deviation from white noise indicates a signal worth extracting.

17.8 Gaussian Mixture Models

17.8.1 Hook — When One Bell Curve Is Not Enough

You measure the heights of everyone in a shopping mall. Plot a histogram. You expect one bell curve — but you see two humps. One around 165 cm, another around 178 cm. Men and women are mixed in your data, and their heights follow different distributions. A single Gaussian cannot capture this. A Gaussian Mixture Model (GMM) can — it models your data as a weighted blend of two (or more) bell curves.

17.8.2 Intuition — The Cauvery River Analogy (Professor's Own)

Imagine the border region between Tamil Nadu and Karnataka — a hilly, forested area where the boundary is unclear. Each state is like a Gaussian cluster: its villages cluster around a center, with some spread into the periphery.

When an incident happens in the border hills, whose jurisdiction does it fall under? You cannot draw a hard line. Instead, you compute the probability that the incident belongs to Tamil Nadu versus Karnataka, based on how far it is from each state's center — adjusted for how spread out each state's villages are. You assign it to the higher probability.

This is exactly what GMM classification does. Each Gaussian is a "state," and new data points get a probabilistic assignment — not a hard yes/no.

The analogy breaks when: the clusters have very different sizes (mixing coefficients). A tiny cluster might still claim points near its center, but a large cluster has more "pull" through its higher prior probability.

17.8.3 Symbols

Symbol	Meaning	Type	Domain
π_k	mixing coefficient for component k	scalar	[0, 1], Σπ_k = 1
μ_k	mean of component k	scalar	real
σ_k²	variance of component k	scalar	positive real
𝒩(X∣μ,σ²)	Gaussian (normal) PDF evaluated at X	scalar	positive real
n_k	number of points in cluster k	integer	positive
N	total number of data points	integer	positive

17.8.4 Mathematical Formulation

The GMM probability density function (for K components):

P (X) = k = 1 \sum K π_{k} \cdot N (X ∣ μ_{k}, σ_{k}^{2})

where each component is a normal PDF:

N (X ∣ μ, σ^{2}) = \frac{1}{2 π σ ^{2}} exp (- \frac{( X - μ ) ^{2}}{2 σ ^{2}})

This is a weighted sum of bell curves. The mixing coefficients π_k tell you how much each bell curve contributes to the overall distribution. They must satisfy:

k = 1 \sum K π_{k} = 1, π_{k} \geq 0

Mixing coefficients — the prior probability:

π_{k} = \frac{n _{k}}{N}

If cluster 1 has 5 points and there are 10 total, then π₁ = 0.5. This is the prior probability that a randomly chosen point belongs to cluster k — before you see the point's value.

17.8.5 Worked Example — Two-Cluster GMM

Setup: 10 data points. The first 5 belong to Gaussian Component 1 (GC1). The last 5 belong to Gaussian Component 2 (GC2). GC1 values: 3.2, 3.8, 4.1, 4.5, 4.9 GC2 values: 7.8, 8.5, 9.0, 9.5, 10.2 Step 1 — Compute means and variances: GC1:

μ_{1} = \frac{3.2 + 3.8 + 4.1 + 4.5 + 4.9}{5} = \frac{20.5}{5} = 4.1

σ_{1}^{2} = \frac{( 3.2 - 4.1 ) ^{2} + ( 3.8 - 4.1 ) ^{2} + ( 4.1 - 4.1 ) ^{2} + ( 4.5 - 4.1 ) ^{2} + ( 4.9 - 4.1 ) ^{2}}{4}

= \frac{0.81 + 0.09 + 0 + 0.16 + 0.64}{4} = \frac{1.70}{4} = 0.425

GC2:

μ_{2} = \frac{7.8 + 8.5 + 9.0 + 9.5 + 10.2}{5} = \frac{45.0}{5} = 9.0

σ_{2}^{2} = \frac{( 7.8 - 9.0 ) ^{2} + ( 8.5 - 9.0 ) ^{2} + ( 9.0 - 9.0 ) ^{2} + ( 9.5 - 9.0 ) ^{2} + ( 10.2 - 9.0 ) ^{2}}{4}

= \frac{1.44 + 0.25 + 0 + 0.25 + 1.44}{4} = \frac{3.38}{4} = 0.845

Step 2 — Mixing coefficients:

π_{1} = \frac{5}{10} = 0.5, π_{2} = \frac{5}{10} = 0.5

Step 3 — Full GMM PDF:

P (X) = 0.5 \cdot \frac{1}{2 π \cdot 0.425} e^{- \frac{( X - 4.1 ) ^{2}}{2 \cdot 0.425}} + 0.5 \cdot \frac{1}{2 π \cdot 0.845} e^{- \frac{( X - 9.0 ) ^{2}}{2 \cdot 0.845}}

This is the complete model. For any X, plug it in and get the mixture density.

17.8.6 Classifying a New Point — Bayes Theorem

Problem: A new data point X = 6.5 arrives. Which component does it likely belong to? Method: Use Bayes theorem. The posterior probability that X belongs to GC1 is:

P (GC 1 ∣ X) = \frac{π _{1} \cdot N ( X ∣ μ _{1} , σ _{1}^{2} )}{P ( X )}

where P(X) is the GMM PDF from Step 3 (the total probability, serving as the normalizing denominator).

P (GC 2 ∣ X) = \frac{π _{2} \cdot N ( X ∣ μ _{2} , σ _{2}^{2} )}{P ( X )}

Compute the numerator for GC1 at X = 6.5:

π_{1} \cdot N (6.5 ∣ 4.1, 0.425) = 0.5 \cdot \frac{1}{2 π \cdot 0.425} e^{- \frac{( 6.5 - 4.1 ) ^{2}}{2 \cdot 0.425}}

= 0.5 \cdot \frac{1}{2.670} e^{- \frac{5.76}{0.85}} = 0.5 \cdot \frac{1}{1.634} \cdot e^{- 6.776} = 0.5 \cdot 0.612 \cdot 0.00114 \approx 0.00035

Compute the numerator for GC2 at X = 6.5:

π_{2} \cdot N (6.5 ∣ 9.0, 0.845) = 0.5 \cdot \frac{1}{2 π \cdot 0.845} e^{- \frac{( 6.5 - 9.0 ) ^{2}}{2 \cdot 0.845}}

= 0.5 \cdot \frac{1}{5.309} e^{- \frac{6.25}{1.69}} = 0.5 \cdot \frac{1}{2.304} \cdot e^{- 3.698} = 0.5 \cdot 0.434 \cdot 0.0247 \approx 0.00536

P (X) = 0.00035 + 0.00536 = 0.00571

P (GC 1 ∣ X = 6.5) = \frac{0.00035}{0.00571} \approx 0.061, P (GC 2 ∣ X = 6.5) = \frac{0.00536}{0.00571} \approx 0.939

Conclusion: X = 6.5 belongs to GC2 with 93.9% probability. Sense-check: X = 6.5 is 2.4 units from μ₁ = 4.1 (about 3.7 standard deviations away) but only 2.5 units from μ₂ = 9.0 (about 2.7 standard deviations away). GC2 is the more likely source — even though 6.5 is numerically closer to 4.1, the smaller variance of GC1 makes 6.5 an extreme outlier for that cluster.

17.8.7 Assumptions and Scope

Scope: GMM assumes:

Data comes from K Gaussian components. If the true clusters are non-Gaussian (e.g., moon-shaped, ring-shaped), GMM will fit ellipsoidal blobs that may not match the data.
K is known or chosen. In exam problems, K is given. In practice, you choose K using BIC, AIC, or cross-validation.
Soft clustering. Points have partial membership in all clusters. If you need hard assignments, you can take the cluster with the highest posterior probability.

What breaks when assumptions fail: Non-Gaussian clusters → use DBSCAN, spectral clustering, or other nonparametric methods. Unknown K → model selection criteria needed. Hard clustering required → K-means is simpler and faster (but less flexible).

17.8.8 Visual Intuition — Two Overlapping Bell Curves

Picture two bell curves on the same x-axis. The left curve (GC1) is tall and narrow — centered at 4.1 with a tight spread (σ₁² = 0.425). The right curve (GC2) is shorter and wider — centered at 9.0 with a broader spread (σ₂² = 0.845).

The curves overlap slightly around X = 6 to 7 — this is the "border region." A point at X = 6.5 sits in the right tail of GC1 and the left tail of GC2. Both curves have nonzero density there, but GC2's density is higher because GC1 drops off very sharply (small variance).

The mixing coefficients (both 0.5) mean the two curves are equally tall before scaling. If π₁ were 0.9 and π₂ were 0.1, the left curve would dominate the picture — most points come from GC1, and you would need strong evidence to assign a point to GC2.

17.8.9 Pitfalls

Trap 1 — Forgetting π_k = n_k/N. The mixing coefficient is just the proportion of points in cluster k. If you have 7 points in GC1 and 3 in GC2, π₁ = 0.7, π₂ = 0.3. Do not set both to 0.5 by default. Trap 2 — Confusing the GMM PDF with a single Gaussian. P(X) is the sum of weighted Gaussians, not a single Gaussian with some averaged μ and σ. The whole point is that the distribution can be multimodal. Trap 3 — Using hard assignment when the problem asks for soft. The posterior P(GC_k ∣ X) gives probabilities. Do not just say "X belongs to the nearest cluster center." The Bayes classification uses both distance (via the Gaussian PDF) and prior probability (via π_k). Trap 4 — Forgetting to normalize when classifying. The denominator P(X) is the sum of numerators across all clusters. P(GC1 ∣ X) + P(GC2 ∣ X) must equal 1. If your two probabilities do not sum to 1, you forgot the denominator.

17.8.10 Student Q&A

Q: Are the cluster assignments always given, or do we need to determine them? A: In exam problems, the clusters are explicitly stated. For example: "the first five values belong to GC1, the last five to GC2." In real implementation, you start with random initialization and iterate using the EM (Expectation-Maximization) algorithm. For exam purposes, the assignment is given. The professor explained: "Otherwise everybody will write their own answer, which is very difficult for us to evaluate."

Q: What if there is overlap between clusters? A: Overlapping clusters are exactly when GMM is most useful. If there were no overlap, you could just use hard clustering (like K-means). The soft assignment via Bayes theorem handles overlap naturally — a point in the overlap region gets partial membership in both clusters. The posterior probability tells you which cluster is more likely, even when both are plausible.

17.8.11 Exam Guidance

Exam note: GMM problems will give you the cluster assignments. You compute means, variances, mixing coefficients, write the PDF, and classify new points using Bayes theorem. The mixing coefficient is simply n_k/N. The PDF for the whole model is the weighted sum of the individual Gaussian PDFs.

17.8.12 Recap and Bridge

A GMM models data as a weighted sum of Gaussian components. π_k = n_k/N is the prior weight. Classify new points by computing P(GC_k ∣ X) using Bayes theorem — the cluster with the highest posterior probability wins. Next: we step back to basics — how do two variables move together? That is covariance and correlation.

17.8.13 Real-World & Domain Connection

Spotify uses GMMs to model song attributes for genre classification and recommendation. A song's audio features (tempo, energy, danceability, acousticness) are treated as coming from a mixture of Gaussian components — each component representing a genre or sub-genre. A new song gets a probabilistic assignment: "70% pop, 20% dance, 10% hip-hop." This soft classification feeds into recommendation algorithms. The same approach is used in speaker diarization — identifying who is speaking in a multi-person conversation. It is used in customer segmentation in marketing for overlapping behavior patterns. It is also used in anomaly detection in manufacturing — points with low probability under the GMM are flagged as potential defects. The Cauvery River analogy from the professor captures the essence: when boundaries blur, soft classification via Bayes theorem is the right tool.

17.9 Covariance and Correlation

17.9.1 Hook — Do Two Stocks Move Together?

You hold Apple and Microsoft stock. When Apple goes up, does Microsoft tend to go up too? By how much? And how tightly do they move together? Two numbers answer these questions: covariance gives the direction of the relationship; correlation gives both direction and strength on a standardized scale.

17.9.2 Intuition — The Two-Stock Analogy

Covariance is like knowing two dancers move in the same direction — when one steps forward, the other steps forward. But you do not know if they are doing a tight tango (every step perfectly synchronized) or a loose shuffle (they both drift forward but at different times and rates).

Correlation is the tango score. It takes the covariance and divides by how much each dancer waves their arms independently. A correlation of +1 means perfect lockstep. A correlation of 0 means their movements are unrelated. A correlation of −1 means they move in perfect opposition.

The analogy breaks when: the relationship is nonlinear. Two dancers doing a circular waltz have a strong relationship, but correlation (which measures linear association) might be near zero. Correlation only catches straight-line patterns.

17.9.3 Symbols

Symbol	Meaning	Type	Domain
Cov(X,Y)	covariance between X and Y	scalar	real
r or ρ	correlation coefficient	scalar	[−1, 1]
X̄	mean of X	scalar	real
Ȳ	mean of Y	scalar	real
σ_X	standard deviation of X	scalar	positive real
σ_Y	standard deviation of Y	scalar	positive real
n	number of data points	integer	n ≥ 2

17.9.4 Mathematical Formulation

Covariance — direction of linear relationship:

Cov (X, Y) = \frac{\sum _{i = 1}^{n} ( X _{i} - X ˉ ) ( Y _{i} - Y ˉ )}{n - 1}

For each data point, compute how far X is from its mean, and how far Y is from its mean. Multiply those deviations. If X and Y tend to be on the same side of their means together, the product is positive. If they tend to be on opposite sides, the product is negative. Sum and divide by n−1.

The professor described it simply: "X minus X bar into Y minus Y bar — it is very simple."

Correlation — standardized strength:

r = \frac{Cov ( X , Y )}{σ _{X} \cdot σ _{Y}}

Divide the covariance by the product of the two standard deviations. This strips away the units and scales the result to [−1, 1].

r = +1: perfect positive linear relationship
r = −1: perfect negative linear relationship
r = 0: no linear relationship
0 < |r| < 0.3: weak
0.3 ≤ |r| < 0.7: moderate
|r| ≥ 0.7: strong

17.9.5 Worked Example — Interpreting the Numbers

Given: For a dataset of stock returns, Cov(X,Y) = 1.254 and r = 0.84. Inference from covariance (+1.254):

The positive sign tells you the stocks move in the same direction. When X's return is above its average, Y's return tends to be above its average too. But the magnitude 1.254 does not tell you how strong the relationship is — it depends on the units of X and Y (e.g., dollars vs. percentages).

Inference from correlation (+0.84):

The stocks are strongly positively related. A correlation of 0.84 is close to +1. If X is one standard deviation above its mean, Y tends to be about 0.84 standard deviations above its mean.

The professor emphasized: "Covariance helps us understand they are positively related. But how much relation exists — we cannot say from covariance. Correlation tells us they are positively related and also strongly related."

17.9.6 The Key Distinction — Direction vs. Strength

Property	Covariance	Correlation
Sign (direction)	✓	✓
Magnitude (strength)	✗ (units-dependent)	✓ (unitless)
Range	(−∞, +∞)	[−1, +1]
Comparable across datasets	✗	✓

Covariance = direction only. Correlation = direction + strength.

17.9.7 Assumptions and Scope

Scope:

Linear relationship only. Both covariance and correlation measure linear association. A perfect U-shaped relationship can have r ≈ 0. Always plot your data first.
Sensitive to outliers. A single extreme point can dramatically inflate or deflate the correlation coefficient. Spearman's rank correlation is a strong alternative.
Correlation does not imply causation. r = 0.84 between ice cream sales and drowning deaths does not mean ice cream causes drowning. Both are driven by summer weather.

What breaks when assumptions fail: Nonlinear relationship → use Spearman's ρ or mutual information. Outliers → use strong correlation. Causal claims → need experimental design or causal inference methods.

17.9.8 Visual Intuition — The Scatter Plot Gallery

Picture four scatter plots, each with X on the horizontal axis and Y on the vertical:

Plot 1 (r ≈ +1): Points form a tight upward-sloping line. As X increases, Y increases almost perfectly. Like height vs. weight for adults. Plot 2 (r ≈ −1): Points form a tight downward-sloping line. As X increases, Y decreases almost perfectly. Like speed vs. travel time for a fixed distance. Plot 3 (r ≈ 0): Points form a shapeless cloud. No discernible upward or downward trend. Like shoe size vs. IQ. Plot 4 (r ≈ 0 but related): Points form a perfect U-shape. Y is high when X is very low or very high, and low in the middle. r ≈ 0 despite the perfect relationship — correlation misses nonlinear patterns.

17.9.9 Pitfalls

Trap 1 — Interpreting covariance magnitude directly. Cov(X,Y) = 100 does not mean "strong relationship." If X is measured in millimeters and Y in dollars, the covariance could be huge even for a weak relationship. Only correlation gives standardized strength. Trap 2 — Assuming correlation implies causation. This is the most famous statistical fallacy. r = 0.9 between two variables does not mean one causes the other. There could be a third variable driving both, or it could be pure coincidence. Trap 3 — Ignoring the scatter plot. Always plot the data. Anscombe's quartet is a famous set of four datasets with identical means, variances, correlations, and regression lines — but completely different patterns when plotted. Trap 4 — Using correlation for nonlinear relationships. r measures straight-line association. If the relationship curves, r will underestimate the true strength of association.

17.9.10 Student Q&A

Q: Does the sign of covariance alone give the direction? A: Yes. Positive covariance → positive relationship. Negative covariance → negative relationship. The sign alone tells you the direction. The magnitude without standardization does not tell you the strength — for that you need correlation.

Q: Is correlation always linear? A: Yes. Correlation measures linear relationship only. If the relationship is nonlinear (U-shaped, for example), correlation can be close to zero even though a strong relationship exists. For nonlinear relationships, correlation is the wrong tool — use Spearman's rank correlation or mutual information instead.

17.9.11 Exam Guidance

Exam note: Covariance and correlation are formula-substitution problems. The important part is the inference — explain what the numbers mean. "Positively related," "strongly related," "weakly related" are the key phrases. Know that covariance = direction, correlation = direction + strength.

17.9.12 Recap and Bridge

Cov(X,Y) tells you whether two variables move together or apart. r = Cov/(σ_X·σ_Y) standardizes this to [−1, +1], giving both direction and strength. Next: we take the final step — fitting a straight line through scattered data with linear regression.

17.9.13 Real-World & Domain Connection

Bloomberg Terminal's portfolio analytics are built on covariance and correlation matrices. A portfolio manager holding 50 stocks needs to know how they move together. The 50×50 correlation matrix captures every pairwise relationship. Modern Portfolio Theory (Markowitz, 1952) uses these matrices to construct portfolios that maximize expected return for a given level of risk. The core insight: diversification works because assets are not perfectly correlated. If all stocks had r = +1, diversification would be pointless. In finance, a correlation of 0.3 between two assets is considered excellent for diversification purposes. The same mathematics drives weather forecasting (correlation between atmospheric pressure at different locations), genomics (co-expression of genes), and recommendation systems (correlation between user preferences).

17.10 Linear Regression

17.10.1 Hook — What Is the Price of a House?

You know the square footage of a house. You want to predict its price. You have data on 10 recent sales: size and price for each. How do you draw the best straight line through those 10 points — and then use it to price an 11th house? Linear regression answers this.

17.10.2 Intuition — Drawing the Best Line

Imagine throwing 10 darts at a wall. They scatter in a rough upward-sloping pattern — bigger houses cost more. You take a ruler and draw a straight line through the cloud. Some darts are above the line, some below. You wiggle the ruler until the total squared vertical distance from darts to line is as small as possible.

That is linear regression. The line Ŷ = W₀ + W₁X is your ruler. W₀ is where it hits the Y-axis (the price when square footage is zero — the land value). W₁ is the slope (how much extra price you get per extra square foot).

The analogy breaks when: the true relationship is curved. If price increases faster for large houses than small ones, a straight line will systematically under-predict in some ranges and over-predict in others. That calls for polynomial regression or a transformation.

17.10.3 Symbols

Symbol	Meaning	Type	Domain
W₀	intercept	scalar	real
W₁	slope	scalar	real
X	independent (predictor) variable	scalar	real
Y	dependent (response) variable	scalar	real
Ŷ	predicted Y = W₀ + W₁X	scalar	real
n	number of data points	integer	n ≥ 2

17.10.4 Mathematical Formulation — The Normal Equations

The goal is to find W₀ and W₁ that minimize the sum of squared errors:

SSE = i = 1 \sum n (Y_{i} - Y^_{i})^{2} = i = 1 \sum n (Y_{i} - W_{0} - W_{1} X_{i})^{2}

Taking derivatives with respect to W₀ and W₁ and setting to zero gives the normal equations:

\sum Y = n W_{0} + W_{1} \sum X

\sum X Y = W_{0} \sum X + W_{1} \sum X^{2}

This is a 2×2 system of linear equations. Solve for W₀ and W₁.

Procedure:

Build a table: X, Y, X², XY
Compute ΣX, ΣY, ΣX², ΣXY
Plug into the two normal equations
Solve for W₀ and W₁
Write the fitted line: Ŷ = W₀ + W₁X
Use for prediction: plug in a new X

The professor described it as: "Taking summation over the first equation gives sum Y equals n W naught plus W1 sum X. Multiplying the first equation by X and summing gives sum XY equals W naught sum X plus W1 sum X squared."

17.10.5 Worked Example — Predicting Y from X

Data:

X	Y	X²	XY
1	2	1	2
2	4	4	8
3	5	9	15
4	4	16	16
5	5	25	25

Sums: ΣX = 15, ΣY = 20, ΣX² = 55, ΣXY = 66, n = 5. Normal equations:

20 = 5 W_{0} + 15 W_{1}

66 = 15 W_{0} + 55 W_{1}

Solve:

From equation (1): $W_{0} = \frac{20 - 15 W _{1}}{5} = 4 - 3 W_{1}$

Substitute into (2):

66 = 15 (4 - 3 W_{1}) + 55 W_{1}

66 = 60 - 45 W_{1} + 55 W_{1}

66 = 60 + 10 W_{1}

10 W_{1} = 6

W_{1} = 0.6

Then: $W_{0} = 4 - 3 (0.6) = 4 - 1.8 = 2.2$

Fitted line:

Y^= 2.2 + 0.6 X

Prediction for X = 6:

Y^= 2.2 + 0.6 (6) = 2.2 + 3.6 = 5.8

Sense-check: When X increases by 1, Y increases by 0.6 on average. The line starts at 2.2 when X = 0. For X = 3, prediction is 2.2 + 1.8 = 4.0 — the actual Y is 5, so the error is +1. The line goes through the middle of the data, not through every point.

17.10.6 Assumptions and Scope

Scope: Linear regression assumes:

Linear relationship. Y is about a linear function of X. Plot the data first.
Independent errors. The error for one observation does not depend on errors for others.
Homoscedasticity. The spread of errors is roughly constant across all values of X (no fanning out).
Normality of errors (for inference). For prediction alone, this is not required. For hypothesis tests on W₀ and W₁, errors should be about normal.

What breaks when assumptions fail: Nonlinear → polynomial regression or transformation. Heteroscedastic → weighted least squares. Dependent errors → time series models. Non-normal errors with small n → bootstrap.

17.10.7 Visual Intuition — The Scatter and the Line

Picture a scatter plot: X runs from 1 to 5 on the horizontal axis, Y runs from 2 to 5 on the vertical. Five blue dots trace a rough upward path: (1,2), (2,4), (3,5), (4,4), (5,5). They do not lie on a perfect line — there is scatter.

Now overlay a red line: Ŷ = 2.2 + 0.6X. At X = 1, the line is at 2.8 (above the dot at 2). At X = 2, the line is at 3.4 (below the dot at 4). At X = 3, the line is at 4.0 (below the dot at 5). At X = 4, the line is at 4.6 (above the dot at 4). At X = 5, the line is at 5.2 (above the dot at 5).

The vertical gaps between dots and line are the residuals. Some are positive, some negative. They sum to zero (a property of least squares). The line is the single best straight-line summary of the five points.

17.10.8 Pitfalls

Trap 1 — Extrapolating beyond the data range. The model fits X from 1 to 5. Predicting Y for X = 100 assumes the linear relationship continues forever — it probably does not. The intercept W₀ = 2.2 means "Y when X = 0," but if X = 0 is far outside the observed range, this number is meaningless. Trap 2 — Interpreting W₁ as causation. W₁ = 0.6 means "a one-unit increase in X is associated with a 0.6 increase in Y," not "X causes Y to increase by 0.6." Regression describes association, not causation. Trap 3 — Forgetting to compute X² and XY correctly. The sums in the normal equations are ΣX² (square first, then sum) not (ΣX)². A common calculator error. Trap 4 — Reversing X and Y. Regression of Y on X is not the same as regression of X on Y. The line that minimizes vertical errors (predicting Y from X) is different from the line that minimizes horizontal errors (predicting X from Y). Choose based on which variable you want to predict.

17.10.9 Student Q&A

Q: What if the X values are large — will the numbers be hard to compute? A: Exam problems generally use manageable numbers. The computation involves sums, squares, and solving two equations — all with small to moderate values. Make a clean table, compute carefully, and the algebra is straightforward. The table method is your best defense against arithmetic errors.

17.10.10 Exam Guidance

Exam note: Regression is formula-heavy but straightforward. The table method is recommended — it organizes the work and makes errors easy to spot. Any approach (normal equations, direct formulas for W₁ and W₀) is acceptable. The prediction step is important — you will likely be asked to forecast Y for a new X. Show all your work, especially the substitution step.

17.10.11 Recap and Bridge

Linear regression fits Ŷ = W₀ + W₁X by solving two normal equations. The table method (X, Y, X², XY) keeps you organized. The slope W₁ is the predicted change in Y per unit increase in X. Use the fitted line for prediction — but never extrapolate far beyond your data. This closes our tour of the lecture's core concepts.

17.10.12 Real-World & Domain Connection

Zillow's Zestimate is powered by regression models. The core model regresses sale price on square footage, number of bedrooms, number of bathrooms, lot size, location score, and year built. Each coefficient (W₁, W₂, …) represents the marginal value of that feature — for example, W_bedroom ≈ $15,000 means an extra bedroom adds about $15,000 to the predicted price, holding everything else constant. The model is continuously retrained on millions of recent sales. The normal equations you solve with n = 5 scale directly to n = 5,000,000 — the math is identical, just solved by a computer instead of by hand. In every domain where a numerical outcome needs to be predicted from features — house prices, crop yields, customer lifetime value, energy consumption — linear regression is the baseline model that more complex models must beat.

Exam Guidance Summary

Exam format: Open-book examination. The formula sheet (one-page cheat sheet) and statistical tables (Z, t, F, chi-square) will be provided.

Topic Weightage

Post-midterm topics form the bulk of the comprehensive exam.
Pre-midterm carryover: about 10–15%. The most likely pre-midterm topics to appear are joint distributions and interval estimation. Both discrete and continuous joint distributions are possible. The mid-semester exam had a continuous joint distribution problem, so discrete may appear this time.

Study Time Guide by Topic

Topic	Difficulty	Study Time	Key Focus
Model identification	Tricky (conceptual)	30 min	Walk the decision tree with every problem type
Hypothesis testing (Z, t, proportions)	Moderate	30 min	Formula review; one-tail vs. two-tail
ANOVA	Moderate (calculation-heavy)	20 min	Practice one full problem: SSTR, SSE, F, table
Chi-square	Easy (formula substitution)	10 min	Identification is the only challenge
Exponential smoothing	Easy (formula substitution)	10 min	F₁ = Y₁; compare α values
White noise	Easy (conceptual)	5 min	Three conditions: μ=0, constant σ², no pattern
GMM	Moderate	30 min	μ, σ², π_k, PDF, Bayes classification
Covariance & correlation	Easy (formula substitution)	10–15 min	Direction vs. strength inference
Regression	Easy (formula substitution)	10–15 min	Table method, solve 2 equations, predict
MLE	Moderate	30 min	Figuring out P(X) and differentiating
Double exponential smoothing	Moderate	—	Part of time series review

Study Strategy

Hypothesis testing is the largest and most confusing block. Spend the most time on model identification — which test to use when. If you pick the wrong test, everything else is wrong.
ANOVA and chi-square are new additions. Make sure you can identify and compute both.
Time series, GMM, covariance/regression are formula-driven. Go through the slides, substitute values, practice one problem each.
Total revision time for all post-midterm topics: about 2–3 hours.

What to Expect in the Exam

Tables (Z, t, F, chi-square) will be provided.
A formula cheat sheet was mentioned — confirm availability on the course portal.
Slides for ANOVA, chi-square, and GMM classification (newer content) will be uploaded separately.
Show all your work. In an open-book setting, writing the formula and showing substitution may be expected. In previous exams, evaluators focused more on the final answer, but this may change.

Key Formulas at Your Fingertips

Test	Formula
One-sample Z/t	$(X ˉ - μ_{0}) / (σ / n)$ or $(s / n)$
Two-sample Z/t	$(X ˉ_{1} - X ˉ_{2} - (μ_{1} - μ_{2})) / S E$
Paired t	$(D ˉ - μ_{D}) / (s_{D} / n), D = Y - X$
ANOVA F	$M S T R / M S E, d f_{1} = k - 1, d f_{2} = N - k$
Chi-square	$\sum (O - E)^{2} / E, df = (r - 1) (c - 1)$
Exp. smoothing	$F_{t + 1} = α Y_{t} + (1 - α) F_{t}$
Covariance	$\sum (x - x ˉ) (y - y ˉ) / (n - 1)$
Correlation	$C o v (X, Y) / (σ_{X} \cdot σ_{Y})$
Regression	$\sum Y = n W_{0} + W_{1} \sum X, \sum X Y = W_{0} \sum X + W_{1} \sum X^{2}$
GMM PDF	$\sum π_{k} \cdot N (X ∣ μ_{k}, σ_{k}^{2})$
GMM classification	$P (G C_{k} ∣ X) = π_{k} \cdot N (X ∣ μ_{k}, σ_{k}^{2}) / P (X)$

Key Industry Applications

The methods covered in this lecture power statistical decision-making across industries. Here is where each technique is used in production:

A/B testing (Netflix, Amazon, Google): The model identification decision tree is the backbone of experimentation platforms. Data scientists walk the same tree — means or proportions, one sample or two, large sample → Z-test — to determine whether a new recommendation algorithm, UI change, or pricing strategy improves results. Millions of experiments run on this logic daily.

Pharmaceutical clinical trials (Pfizer, Moderna): Paired t-tests measure within-subject changes — antibody levels before and after vaccination, blood pressure before and after medication. The paired design controls for individual variability and is the gold standard for demonstrating treatment efficacy to regulators like the FDA.

Agricultural field trials (Bayer/Monsanto): ANOVA compares crop yields across multiple fertilizer formulations, seed varieties, or irrigation methods. A single ANOVA replaces dozens of pairwise t-tests, preserving the 5% error rate. The results determine which products go to market.

CDC epidemiological studies: Chi-square tests of independence screen for associations between lifestyle factors and health outcomes — smoking status vs. hypertension, diet vs. diabetes, exercise frequency vs. heart disease. A significant chi-square triggers deeper investigation and potential public health interventions.

Amazon inventory forecasting: Exponential smoothing with tuned α values powers demand forecasting. Fast-moving electronics get α ≈ 0.8 (responsive to trends). Stable grocery items get α ≈ 0.3 (smooths out noise). The α values are optimized continuously from historical forecast errors.

Bose noise-canceling headphones: White noise characterization in signal processing enables effective active noise cancellation. The ambient sound is modeled as a stochastic process; identifying the white noise component allows the circuitry to generate precise anti-noise signals.

Spotify audio feature clustering: GMMs model song attributes (tempo, energy, danceability) as mixtures of Gaussian components. Each component represents a genre or sub-genre. New songs get soft genre assignments that feed into recommendation and playlist generation algorithms.

Bloomberg Terminal portfolio analytics: Covariance and correlation matrices are computed for thousands of assets in real time. Modern Portfolio Theory uses these matrices to construct diversified portfolios — the core insight being that assets with low correlation reduce overall portfolio risk without sacrificing expected return.

Zillow Zestimate: Multiple regression on square footage, bedrooms, bathrooms, location, and year built predicts property values for over 100 million homes. Each coefficient represents the marginal contribution of a feature to the predicted price — the same normal equations you solve by hand for n = 5 data points, scaled to n = 100,000,000.

ISM Lecture 17 notes · Hypothesis Testing Review, ANOVA, Chi-Square, Time Series, and GMM

Introduction to Statistical Methods· postgraduate· 2026-07-07

Sections Breakdown

1Model Identification Framework for Hypothesis Testing

Decision tree for choosing the correct hypothesis test based on problem language and keywords.

2One-Sample t-Test for a Small Sample

Testing a single population mean when n < 30 and population standard deviation is unknown.

3Paired t-Test

Comparing before-and-after measurements on the same subjects using difference scores D = Y − X.

4One-Way ANOVA

Comparing means across three or more groups using the F-statistic and between-vs-within variation decomposition.

5Chi-Square Test of Independence

Testing association between two categorical variables using observed and expected frequency counts.

6Simple Exponential Smoothing

Time series forecasting blending latest observation with previous forecast via smoothing constant α.

7Autocorrelation and White Noise

Identifying whether a time series has detectable structure or is pure unpredictable random noise.

8Gaussian Mixture Models

Modeling data as weighted sum of Gaussian distributions with soft classification via Bayes theorem.

9Covariance and Correlation

Measuring direction and strength of linear relationships between two variables.

10Linear Regression

Fitting a best-fit straight line using normal equations derived from minimizing squared errors.

Postgraduate students in statistics, data science, and machine learning

Exam Revision Notes

Below is the distilled, exam-ready core of this lecture. Every entry is built from the full textbook notes above. Use this section for rapid review — but if something doesn't make sense, go back to the full explanation in the main content.

Model Identification Framework

Must-know: Walk the decision tree: means vs proportions, then one/two/many, then large/small sample. Picking the wrong model invalidates everything that follows.

⚠️ Top pitfall: Running multiple t-tests instead of one ANOVA for three or more groups inflates Type I error to ~0.14 instead of 0.05.

Self-check: A problem says three fertilizers were tested on separate plots and crop yields were recorded. Which test do you use?

Connects to: One-Sample t-Test, Paired t-Test, ANOVA, Chi-Square Test

One-Sample t-Test

Must-know: Use when n < 30 and σ is unknown. t = (X̄ − μ₀) / (s / √n), df = n − 1. Look up the t-table critical value, not the Z-table.

$t = \frac{X ˉ - μ _{0}}{s / n}$

⚠️ Top pitfall: Using the Z-table (critical value 1.645) instead of the t-table (critical value ≈1.833 for df = 9, α = 0.05) gives false confidence for small samples.

Self-check: You test 10 cars. Sample mean = 18,686, claimed mean = 18,580, s = 167.74. What is the t-statistic?

Connects to: Paired t-Test, Model Identification Framework

Paired t-Test

Must-know: Same subjects measured twice. Compute D = Y − X for each pair, then run a one-sample t-test on the differences. H₀: μ_D = 0 (no effect).

$t = \frac{D ˉ - μ _{D}}{s _{D} / n}$

⚠️ Top pitfall: Using a two-sample t-test on paired data loses the within-person correlation and drastically reduces statistical power.

Self-check: 9 trainees had before/after scores measured. Is this a paired or two-sample problem?

Connects to: One-Sample t-Test, Model Identification Framework

One-Way ANOVA

Must-know: Compare 3+ group means with a single test. F = MSTR/MSE = (SSTR/(k−1)) / (SSE/(N−k)). Reject if F_calculated > F_table. Always right-tailed.

$F = \frac{M S T R}{M S E} = \frac{S S T R / ( k - 1 )}{S S E / ( N - k )}$

⚠️ Top pitfall: Using the wrong df in the F-table. Numerator df = k − 1, denominator df = N − k. Both are required for lookup.

Self-check: Three teaching methods, k = 3, N = 10. What are numerator and denominator degrees of freedom?

Connects to: Model Identification Framework, Chi-Square Test

Chi-Square Test of Independence

Must-know: Tests independence of two categorical variables. E_ij = (R_i × C_j) / N, χ² = Σ(O−E)²/E, df = (r−1)(c−1). H₀ is always independence.

$χ^{2} = \sum \frac{( O _{ij} - E _{ij} ) ^{2}}{E _{ij}}$

⚠️ Top pitfall: Forgetting df = (r−1)(c−1). A 3×2 table has df = 2, not df = 3×2 = 6.

Self-check: A 3×2 contingency table has N = 180. What is the degrees of freedom?

Connects to: ANOVA, Model Identification Framework

Simple Exponential Smoothing

Must-know: Forecast blends latest observation with previous forecast. F_{t+1} = α·Y_t + (1−α)·F_t. Higher α = more responsive to recent data. Initialize F₁ = Y₁.

$F_{t + 1} = α Y_{t} + (1 - α) F_{t}$

⚠️ Top pitfall: Forgetting F₁ = Y₁ for initialization. Setting the first forecast to zero or an arbitrary value biases all early forecasts.

Self-check: For α = 0.8, what weight does the latest observation get in the forecast? What about α = 0.3?

Connects to: White Noise, Linear Regression

White Noise and Autocorrelation

Must-know: White noise = mean zero + constant variance + no autocorrelation. If a series is white noise, it cannot be forecast — there is no signal to extract.

$\pm \frac{2}{T}$

⚠️ Top pitfall: Judging white noise by eye alone. A series can look random but have subtle autocorrelation. Use the ±2/√T confidence band for formal assessment.

Self-check: A time series 3, 2, 1, 0, −1, −2, −1, 0, 1, 2, 3 has mean near zero. Is it white noise?

Connects to: Simple Exponential Smoothing, Linear Regression

Gaussian Mixture Models

Must-know: Models data as weighted sum of K Gaussians. π_k = n_k / N. Classify new points via Bayes theorem: P(GC_k|X) = π_k · N(X | μ_k, σ_k²) / P(X).

$P (X) = \sum π_{k} \cdot N (X ∣ μ_{k}, σ_{k}^{2})$

⚠️ Top pitfall: Forgetting to normalize when classifying. P(GC₁ | X) + P(GC₂ | X) must equal 1 — divide by the total probability P(X).

Self-check: A GMM has two clusters with 5 points each. What are the mixing coefficients π₁ and π₂?

Connects to: Covariance and Correlation, Linear Regression

Covariance and Correlation

Must-know: Covariance gives the direction of linear relationship (sign only). Correlation = Cov/(σ_X · σ_Y) standardizes to [−1, +1], giving both direction and strength. |r| ≥ 0.7 is strong.

$r = \frac{Cov ( X , Y )}{σ _{X} \cdot σ _{Y}}$

⚠️ Top pitfall: Interpreting covariance magnitude as strength. Cov = 100 does not mean strong — only correlation is unitless and comparable across datasets.

Self-check: Cov(X,Y) = 1.254 and r = 0.84. What does each number tell you about the relationship?

Connects to: Linear Regression, Gaussian Mixture Models

Linear Regression

Must-know: Fit the best line via normal equations. Use the table method: compute ΣX, ΣY, ΣX², ΣXY, then solve the 2×2 system for W₀ (intercept) and W₁ (slope).

$\sum Y = n W_{0} + W_{1} \sum X, \sum X Y = W_{0} \sum X + W_{1} \sum X^{2}$

⚠️ Top pitfall: Extrapolating far beyond the data range. If X ranges from 1 to 5, predicting Y at X = 100 assumes the linear relationship continues — it probably does not.

Self-check: Given ΣX = 15, ΣY = 20, ΣX² = 55, ΣXY = 66, n = 5. Write and solve the two normal equations for W₀ and W₁.

Connects to: Covariance and Correlation, Simple Exponential Smoothing

Previous LectureLecture 16 · Gaussian Mixture Models, Maximum Likelihood Estimation, and Hypothesis Testing

Next LectureLecture 01 · Introduction to Machine Learning — Session 1

Was this lecture useful?

Loading comments…

🤖

BitsNotes AI Assistant

Subject Notes Assistant

Configure AI Chat

Choose how to access the chatbot

Have your own API key?

Switch to "Bring Your Own Key" tab above for unlimited access with any OpenAI-compatible provider.

AI Provider Preset

API Endpoint URL (OpenAI-Compatible)

API Key

Model Name

🔑 Enter API key above to fetch live models from provider, or enter model name manually.

Remember key for this session (until tab closes)

OpenAI-Compatible API Support

Choose any provider preset (Gemini, DeepSeek, Kimi, GLM, MiniMax, Qwen, OpenAI, Groq, Ollama, etc.) or enter a custom endpoint URL.

Security & Privacy First

Your API key is sent directly from your browser to your specified provider. BitsNotes servers never store or see your key.

Prerequisite Knowledge

Previously Covered in This Subject

Hypothesis Testing Review, ANOVA, Chi-Square, Time Series, and GMM

17.1 Model Identification Framework for Hypothesis Testing

17.1.1 The Core Problem — Why Model Identification Matters

17.1.2 Intuition — The Triage Nurse Analogy

17.1.3 The Decision Tree — Formal Rules

17.1.4 The Decision Tree as a Flowchart

17.1.5 Worked Example — Identifying Three Problems

17.1.6 Assumptions and Scope

17.1.7 Visual Intuition — The Decision Landscape

17.1.8 Pitfalls

17.1.9 Student Questions and Answers

17.1.10 Exam Guidance

17.1.11 Recap and Bridge

17.1.12 Real-World & Domain Connection

17.2 One-Sample t-Test for a Small Sample

17.2.1 Hook — When the Z-Table Is Not Enough

17.2.2 Intuition — The Food Inspector Analogy

17.2.3 Symbols

17.2.4 Mathematical Formulation

17.2.5 Worked Example — Sports Car Mileage

17.2.6 Assumptions and Scope

17.2.7 Visual Intuition — The t-Distribution vs. the Z

17.2.8 Pitfalls

17.2.9 Student Questions and Answers

17.2.10 Exam Guidance

17.2.11 Recap and Bridge

17.2.12 Real-World & Domain Connection

17.3 Paired t-Test

17.3.1 Hook — Same Person, Two Measurements

17.3.2 Intuition — The "Track Your Own Change" Analogy

17.3.3 Symbols

17.3.4 Mathematical Formulation

17.3.5 Worked Example — Training Program Effectiveness

17.3.6 Assumptions and Scope

17.3.7 Visual Intuition — The D Column Tells the Story

17.3.8 Pitfalls

17.3.9 Student Questions and Answers

17.3.10 Exam Guidance

17.3.11 Recap and Bridge

17.3.12 Real-World & Domain Connection

17.4 One-Way ANOVA

17.4.1 Hook — Why Not Just Run Multiple t-Tests?

17.4.2 Intuition — The Sports Team Height Analogy

17.4.3 Symbols

17.4.4 Mathematical Formulation

17.4.5 Two Computational Approaches

17.4.6 The ANOVA Table

17.4.7 Worked Example — Three-Group Comparison

17.4.8 Decision Rule and the F Distribution

17.4.9 Assumptions and Scope

17.4.10 Visual Intuition — Between vs. Within

17.4.11 Pitfalls

17.4.12 Student Q&A

17.4.13 Exam Guidance

17.4.14 Recap and Bridge

17.4.15 Real-World & Domain Connection

17.5 Chi-Square Test of Independence

17.5.1 Hook — Are Smoking and Hypertension Related?

17.5.2 Intuition — The Seating Preference Analogy

17.5.3 Symbols

17.5.4 Mathematical Formulation

17.5.5 Worked Example — Hypertension and Smoking

17.5.6 The Universal Decision Rule

17.5.7 Assumptions and Scope

17.5.8 Visual Intuition — The Observed vs. Expected Grid

17.5.9 Pitfalls

17.5.10 Student Q&A

17.5.11 Exam Guidance

17.5.12 Recap and Bridge

17.5.13 Real-World & Domain Connection

17.6 Simple Exponential Smoothing

17.6.1 Hook — Predicting Next Week's Sales

17.6.2 Intuition — The Weather App Analogy

17.6.3 Symbols

17.6.4 Mathematical Formulation

17.6.5 Worked Example — Weekly Forecasting

17.6.6 Model Selection — Beyond One Forecast

17.6.7 Assumptions and Scope