Artificial Computational Intelligence

Bayesian Networks

Published: 2026-07-26

Level: postgraduate

Audience: Postgraduate students in Artificial Computational Intelligence

Bayesian Networks

12.1 Review of Foundations: From Logic to Probability

12.1.1 Course Arc Recap

Hook: How do you reason about the world when you cannot be certain of anything? Logic gave us rules for deduction — but what happens when the premises themselves are uncertain?

The lecture opens by tracing the intellectual arc of the course so far — a progression from rigid deduction to flexible reasoning under uncertainty. Understanding where Bayesian networks sit in this arc is essential: they are not a standalone topic, but the natural next step after logic reaches its limits.

Stage 1 — Propositional Logic. The course began with propositional logic: representing knowledge as declarative statements (propositions) and inferring new knowledge using truth tables, theorem proving, resolution (unit resolution after converting to Conjunctive Normal Form, or CNF), and the DPLL algorithm. Propositional logic works well for domains where every fact is either true or false and the relationships between facts can be expressed as logical connectives (AND, OR, NOT, IMPLIES).

The key limitation of propositional logic is the absence of quantifiers. There is no built-in way to express "for all" or "for some." You can write "If it rains, the ground is wet," but you cannot write "For every person X, if X is human, then X is mortal" without inventing separate propositions for every person.

Stage 2 — First-Order Logic (Predicate Logic). First-order logic addressed this by introducing predicates (properties and relations), variables ( $x$ , $y$ ), quantifiers — universal ( $\forall$ , "for all") and existential ( $\exists$ , "there exists") — and instantiation techniques: universal instantiation (UI) and existential instantiation (EI). These allow general statements like "all humans are mortal" to be represented directly and reasoned about symbolically.

Stage 3 — Probability Theory. However, even first-order logic assumes a deterministic world: every statement is either true or false, and deduction yields certain conclusions. Real-world situations involve uncertainty — we know some facts, but we cannot always deduce outcomes with certainty. A patient has symptoms, but we cannot be 100% sure of the disease. A sensor gives a reading, but it might be noisy. This motivates the need for probability theory as a formal language for reasoning under uncertainty.

The prerequisite material includes basic conditional probability and Bayes' rule, which were reviewed in the previous lecture before being paused. Today's lecture picks up from there and introduces Bayesian networks as the primary tool for representing and reasoning with uncertain knowledge.

Intuition: Think of the course arc as building a toolkit for an increasingly messy world. Propositional logic is a Swiss Army knife for clean, binary problems. First-order logic adds a magnifying glass — you can now see patterns across many objects. Probability theory adds a weather forecast — you can now handle situations where you are not sure what is true. Bayesian networks are the organized toolbox that keeps all the probabilistic tools in order.

Q: The course teaches first-order logic, method by contradiction, etc. — what is the real use case when modern systems use embeddings?

A: Modern agentic systems use many advanced techniques, but they all trace back to these fundamentals. In closed organizations that do not want to use out-of-the-box LLMs (due to data privacy, regulatory constraints, or domain-specific requirements), rule-based AI systems built with first-order reasoning, logic-based inferencing, and custom small language models are still directly relevant. Dissertation students regularly build domain-specific models using reasoning techniques, fine-tuning, and knowledge representation. Models will keep changing, but the skill of building an AI system from first principles — deciding what to keep in memory, what to infer, and how to represent logic — is the core competency this course develops. The reasoning patterns you learn here (knowledge representation, inference algorithms, handling uncertainty) are the same patterns used inside every modern AI system, whether it is a Bayesian network, a neural language model, or a hybrid agent.

Recap: The course has moved from propositional logic (binary deduction) → first-order logic (quantified deduction) → probability theory (reasoning under uncertainty). Bayesian networks are the next step: a compact graphical language for encoding probabilistic relationships and performing inference. Every concept in this lecture builds on the conditional probability and Bayes' rule reviewed previously.

12.2 Bayesian Network Fundamentals

12.2.1 Definition and Purpose

Hook: A full joint probability table over 30 binary variables has $2^{30} \approx 1$ billion entries. Can we do better by exploiting the fact that most variables only directly depend on a few others?

A Bayesian network (also called a Bayes net) is a graphical notation for representing conditional independence assertions, enabling compact specification of full joint probability distributions over a set of random variables. Rather than listing every entry in a full joint probability table (which grows exponentially with the number of variables), a Bayesian network encodes only the local dependencies, making the representation tractable.

Intuition: Imagine a company org-chart. Each employee (variable) reports to a small number of managers (parents). You do not need to know the entire company structure to understand any one employee — you only need to know their direct managers and what those managers decided. A Bayesian network works the same way: each variable's behavior is determined by a small set of parent variables, not by the entire network.

Where the analogy breaks: In an org-chart, reporting relationships are administrative. In a Bayesian network, the edges represent probabilistic influence — knowing the parent's value changes the probability distribution of the child, but does not determine it with certainty.

A Bayesian network consists of two components:

A directed acyclic graph (DAG): Each node represents one random variable. Directed edges encode influence: an edge from node $X$ to node $Y$ means $X$ directly influences $Y$ . Variables with no incoming edges are independent (root variables); variables with incoming edges are dependent on their parent nodes. The graph must be acyclic — no directed path can loop back to its starting node. This ensures a well-defined probabilistic semantics.
Conditional probability tables (CPTs): Each node has an associated CPT that specifies the probability distribution over that variable for every combination of its parent values. For a node $X_{i}$ with parents $Parents (X_{i})$ , the CPT encodes $P (X_{i} ∣ Parents (X_{i}))$ . In the simplest case — a root node with no parents — the CPT is just the marginal probability of that variable.

Professor's plain-language description: "You will have a set of nodes, one per variable. Whoever is influencing will have a direction from an independent to a dependent variable, and a conditional distribution for each node given its parents."

Formal definition. Given $n$ random variables $X_{1}, X_{2}, \dots, X_{n}$ , a Bayesian network specifies the full joint distribution as a product of conditional probabilities:

P (x_{1}, x_{2}, \dots, x_{n}) = i = 1 \prod n P (x_{i} ∣ Parents (X_{i}))

where $Parents (X_{i})$ denotes the values of the parent nodes of $X_{i}$ that appear in the particular assignment $(x_{1}, \dots, x_{n})$ . This is Equation 13.2 in the textbook (Russell & Norvig). The key insight is that the entire joint distribution — which would normally require $2^{n}$ entries for $n$ binary variables — is encoded by a product of much smaller local tables.

Compactness example. Suppose we have $n = 30$ binary variables, each with at most $k = 5$ parents. Each CPT requires at most $2^{5} = 32$ entries (actually $2^{k}$ rows with probabilities that sum to 1, so $2^{k} - 1$ free parameters per node). The complete network requires roughly $2^{k} \cdot n = 32 \times 30 = 960$ numbers. The full joint distribution requires $2^{30} \approx 1, 073, 741, 824$ entries. This dramatic reduction — from over a billion to under a thousand — is what makes Bayesian networks practical.

12.2.2 Key Properties

CPT structure rules:

CPT rows must sum to 1. Each row in a CPT represents a conditional distribution over the variable for a specific parent configuration. Since the variable must take exactly one of its possible values, the probabilities in each row must sum to exactly 1.
CPT columns do NOT sum to 1. Different rows correspond to different conditioning contexts (different parent configurations). Adding column values is meaningless — it mixes probabilities conditioned on different evidence. Students should not add column values.
Independent (root) variables have small CPTs — just the marginal probability of the variable itself. For a binary variable: two entries $P (X)$ and $P (\neg X)$ , summing to 1. This is a single-row table.
Dependent variables have larger CPTs — one row for each combination of parent values. A variable with $n$ binary parents has $2^{n}$ rows, each containing the probabilities of the variable being true and false given that parent combination. For example, a node with 3 binary parents has $2^{3} = 8$ rows.
Conditional independence: If knowing the value of a variable's parents makes it independent of all non-descendants, the network encodes this as:

P (X_{i} ∣ Parents (X_{i}), NonDescendants (X_{i})) = P (X_{i} ∣ Parents (X_{i}))

This is what makes the compact representation possible. Each variable "screens off" the influence of its ancestors once its parents are known.

Scope: The conditional independence assumption is an assertion about the domain — it must be justified by domain knowledge, not just assumed for convenience. If a variable actually depends on a non-parent ancestor (e.g., a grandparent directly influences a grandchild), the network structure is wrong and the compact representation will produce incorrect inferences.

When it breaks: In domains with long-range dependencies (e.g., genetic inheritance across multiple generations, or cascading failures in a power grid), naive Bayesian network structures may miss important non-parent influences. Domain expertise or structure-learning algorithms are needed to capture these.

12.2.3 Construction Procedure

Step-by-step procedure for constructing a Bayesian network from a problem description:

Identify all random variables in the domain. List every quantity that is uncertain and relevant to the problem.
Determine independence and dependence among the variables — which are root (independent) variables, and which depend on others. Root variables have no incoming edges.
Find conditional independencies — for each dependent variable, identify which other variables directly influence it (its parents). The key question is: "Given the values of which other variables is $X_{i}$ independent of all remaining variables?" Those are the parents.
Draw nodes and directed edges — place each variable as a node; draw a directed edge from each parent to each child. Ensure the graph is acyclic (no directed cycles).
Construct CPTs for each node — the CPT encodes $P (X_{i} ∣ Parents (X_{i}))$ for every combination of parent values. Root nodes get a single-row marginal probability table.

Ordering matters. The textbook (Russell & Norvig, §13.2) emphasizes that node ordering affects network compactness. Choosing a causal order — causes before effects — typically yields the most compact network with the fewest edges and smallest CPTs. Choosing a diagnostic order (effects before causes) can produce a much more complex network with redundant edges. For example, in the burglary domain, ordering B, E, A, J, M (causes first) gives 10 parameters; ordering M, J, A, B, E (effects first) gives 13; and a pathological ordering M, J, E, B, A gives 31 — the same as the full joint distribution.

Professor's emphasis — a common terminology trap: "ML doesn't mean machine learning here — ML means marginalization." Marginalization is a probability concept for summing out (eliminating) variables from a joint distribution. The abbreviation "ML" will appear in inference contexts throughout this lecture. Students from an ML (machine learning) background must not misread it.

Recap: A Bayesian network = DAG (structure) + CPTs (parameters). The DAG encodes conditional independence; the CPTs encode local probability distributions. The full joint distribution is the product of all CPT entries. This representation is exponentially more compact than a full joint table when each variable has few parents. Next: a concrete example of constructing a Bayesian network from a problem description.

12.3 Construction: Dental Health Example

12.3.1 Problem Setup

Hook: Can a toothache tell you whether you have a cavity? Not with certainty — but a Bayesian network can tell you exactly how likely it is, given what the dentist finds.

Consider a simple world with four random variables:

Weather ( $W$ ): whether the day's weather is good or not.
Cavity ( $V$ ): whether a person has a dental cavity. (The professor uses $V$ — likely short for "void" or the Latin vitium — rather than the more common $C$ , which is reserved for Catch.)
Toothache ( $T$ ): whether the person has a toothache.
Catch ( $C$ ): whether the dentist is able to catch (detect) the cavity during examination.

Intuition: Think of a doctor's diagnostic process. The patient reports a symptom (toothache). The doctor runs a test (the dentist probes the tooth and tries to "catch" the cavity). The actual condition (cavity) is the hidden cause that connects the symptom and the test result. Weather is a completely unrelated factor — like the color of the patient's shirt. It has no bearing on the diagnosis.

The relationships given in the problem:

Weather is completely independent of all other variables. Having a toothache, cavity, or the dentist catching the cavity has nothing to do with the day's weather.
Cavity is a direct cause of both toothache and catch. If a person has a cavity, they will likely have a toothache, and the dentist should be able to detect it.
There is no direct relation between toothache and catch. A person might have a toothache without a cavity (for other dental reasons), and catch (detection) is purely about the dentist's ability to find an existing cavity.
Toothache and catch are conditionally independent given cavity. Formally:

P (T, C ∣ V) = P (T ∣ V) \cdot P (C ∣ V)

This means: once we know whether the patient has a cavity, knowing whether the dentist caught it tells us nothing additional about whether the patient has a toothache, and vice versa. The cavity "screens off" the two variables from each other.

Professor's plain-language description: "You having toothache is different, and it is not dependent on we catching that particular tooth with germs."

What conditional independence means here. Without knowing cavity status, toothache and catch are marginally dependent — if the dentist catches something, it is more likely the patient also has a toothache (because both are symptoms of a cavity). But once cavity is known, this dependence vanishes. This is the "explaining away" phenomenon: the cavity explains both the toothache and the catch, so knowing one gives no additional information about the other.

12.3.2 Drawing the Network

Assigning short names: $W$ for weather, $V$ for cavity, $T$ for toothache, $C$ for catch.

Step 1 — Draw four nodes: $W$ , $V$ , $T$ , $C$ .

Step 2 — Identify root variables: Weather ( $W$ ) has no incoming edges — it is an independent (root) variable. Cavity ( $V$ ) also has no incoming edges — it is independently a root variable. (Weather does not cause cavities in this model.)

Step 3 — Draw directed edges:

$V \to T$ — cavity causes toothache.
$V \to C$ — cavity enables catch (detection).

Step 4 — Verify no missing edges: No edge between $T$ and $C$ — they are conditionally independent given $V$ . No edge from $W$ to anything — weather is isolated.

The resulting graph:

$W$ and $V$ are root nodes (no parents). $T$ and $C$ are children of $V$ . Weather stands alone.

12.3.3 Conditional Probability Tables

The CPT data was provided by the professor as follows:

Weather ( $W$ ) — root node, marginal probability:

$W$	$P (W)$
T	0.5
F	0.5

This is a single-row table (no parents). It says good and bad weather are equally likely.

Cavity ( $V$ ) — root node, marginal probability:

$V$	$P (V)$
T	0.4
F	0.6

40% of the population has a cavity. Note: $P (V) + P (\neg V) = 0.4 + 0.6 = 1$ . ✓

Toothache ( $T$ ) given Cavity ( $V$ ) — dependent node, 2 rows (one per parent value):

$V$	$T$	$P (T ∣ V)$
T	T	0.3
T	F	0.7
F	T	0.2
F	F	0.8

Reading the table: $P (T = F ∣ V = T) = 0.7$ means "the probability of not having a toothache given that you do have a cavity is 0.7." This is surprising — even with a cavity, most people do not have a toothache. The professor confirmed this interpretation with the class.

Row sums: Row 1: $0.3 + 0.7 = 1$ ✓. Row 2: $0.2 + 0.8 = 1$ ✓.

Catch ( $C$ ) given Cavity ( $V$ ) — dependent node, 2 rows:

$V$	$C$	$P (C ∣ V)$
T	T	0.1
T	F	0.9
F	T	0.4
F	F	0.6

Reading the table: $P (C = T ∣ V = F) = 0.4$ means "the probability of the dentist catching something even when there is no cavity is 0.4." This is a false positive — the dentist detects a cavity that is not there. Each row sums to 1.

Worked sense-check — verifying the CPTs are self-consistent.

Let us verify that the CPT entries are valid probabilities and consistent with the conditional independence assumption.

All entries are in $[0, 1]$ . ✓
All row sums equal 1. ✓
Weather is independent of everything — its CPT has no parent columns. ✓
Toothache depends only on cavity — its CPT has exactly one parent column ( $V$ ). ✓
Catch depends only on cavity — its CPT has exactly one parent column ( $V$ ). ✓
There is no CPT for toothache that mentions catch, and no CPT for catch that mentions toothache. This confirms conditional independence given cavity. ✓

Q: Can someone tell me what this particular cell denotes — the one with $V = F$ , $C = F$ ?

A: In plain English: "What is the probability that the dentist does NOT catch anything given that there is NO cavity?" Reading the Catch CPT: when $V = F$ and $C = F$ , the value is 0.6. This makes intuitive sense — if there is no cavity, the dentist is more likely (60%) to correctly find nothing than to falsely detect something (40%).

Q: Is $P (\neg T)$ ever greater than one?

A: No. Probability can never be greater than 1. Even when summing multiple cells, the result must stay within $[0, 1]$ . This is an axiom of probability theory. If your calculation produces a value greater than 1, you have made an arithmetic error or a structural error (e.g., applying the chain rule in the wrong order — see Section 12.4).

Q: Should we add the telephone as a variable when the problem mentions John confusing telephone rings with the alarm?

A: No. The mention of the telephone is simply explaining why John sometimes calls when there is no alarm — it is a false positive scenario. The telephone itself is not a variable in the Bayesian network. No new node is needed. The textbook (Russell & Norvig, §13.1) makes the same point: "The network does not have nodes corresponding to Mary's currently listening to loud music or to the telephone ringing and confusing John. These factors are summarized in the uncertainty associated with the links from Alarm to JohnCalls and MaryCalls." This is an example of context-specific independence — the telephone only matters when the alarm does not ring, and its effect is already captured in the CPT entry $P (J = T ∣ A = F) = 0.05$ .

Pitfall 1 — Confusing the variable name $V$ with "variable." The professor uses $V$ for cavity (likely from Latin vitium). Do not read it as a generic variable stand-in. On the exam, use whatever names the problem gives.

Pitfall 2 — Reading CPT rows across different parent configurations. $P (T = T ∣ V = T) = 0.3$ and $P (T = T ∣ V = F) = 0.2$ are different conditional probabilities. Do not add them or compare them as if they were the same distribution.

Pitfall 3 — Assuming conditional independence means marginal independence. Toothache and catch are conditionally independent given cavity, but they are marginally dependent. If you do not know cavity status and you learn the dentist caught something, your probability of toothache goes up. Only after learning cavity status does the dependence vanish.

Recap: The dental health network has 4 variables, 2 root nodes ( $W$ , $V$ ), 2 edges ( $V \to T$ , $V \to C$ ), and 4 CPTs. Toothache and catch are conditionally independent given cavity — knowing one does not help predict the other once cavity is known. This network will be used to practice joint probability queries in the next section.

12.4 Linearization and Chain Rule

12.4.1 Query Classification: Joint vs. Conditional

Hook: Before you can compute anything on a Bayesian network, you must first ask: "Am I computing the probability of everything together, or the probability of something given something else?" The answer determines the entire solution strategy.

Before solving any query on a Bayesian network, the first step is to classify it:

Joint probability query: All variables are listed inside a single probability expression with commas. Example: $P (W, \neg T, \neg V, C)$ — "What is the probability of good weather, no toothache, no cavity, and catching?" There is no vertical bar — no variable is conditioned on any other.
Conditional probability query: There is a vertical bar separating query from evidence. Example: $P (B ∣ J, M)$ — "What is the probability of burglary given that John and Mary called?" The left side of the bar is the query; the right side is the evidence (what is already known).

How to tell from plain English: If the problem says "What is the probability of X, Y, and Z?" (nothing is stated as given), it is a joint probability query. If it says "What is the probability of X given Y?" or "X happened, given that Y already happened, what is the probability of Z?" — the word "given" signals a conditional query. The word "given" always maps to the vertical bar $∣$ in the mathematical notation.

12.4.2 Linearization

Linearization is the process of rewriting a probability query so that all child (dependent) variables appear before their parent variables. Independent variables can appear anywhere (beginning, middle, or end), but for ease of calculation, it is best to place them at the start or end.

Why linearization is necessary: When you apply the chain rule (see below), the order of variables determines how conditional probabilities are decomposed. If a parent appears before its child in the chain rule expansion, the decomposition will not correctly account for the dependency relationship, leading to incorrect results — you might even get probabilities greater than 1.

The rule: In the linearized order $X_{1}, X_{2}, \dots, X_{n}$ , for every edge $X_{i} \to X_{j}$ in the DAG, $X_{j}$ (child) must appear before $X_{i}$ (parent). This is a topological ordering of the DAG in reverse.

Professor's emphasis: "If you don't perform linearization, then you would go wrong. If you apply chain rule on a non-linearized form, you will not get appropriate answers."

Worked example — linearizing the dental query.

The original query: $P (W, \neg T, \neg V, C)$

Step 1 — Identify parent-child relationships in the DAG:

$V \to T$ (cavity is parent of toothache)
$V \to C$ (cavity is parent of catch)
$W$ has no parents and is not a parent of anything.

Step 2 — Check the current order: In the expression $P (W, \neg T, \neg V, C)$ :

$\neg T$ (child of $V$ ) appears before $\neg V$ (parent). ✓
But $\neg V$ (parent of $C$ ) appears before $C$ (child). ✗ — This violates the rule.

Step 3 — Rewrite: Move $C$ before $\neg V$ :

P (W, \neg T, C, \neg V)

Now children ( $\neg T$ , $C$ ) appear before their parent ( $\neg V$ ). ✓

Valid alternative linearizations (all correct):

$P (\neg T, C, \neg V, W)$
$P (C, \neg T, \neg V, W)$
$P (\neg T, C, W, \neg V)$

The key rule is that children precede parents. The position of the independent variable $W$ does not affect correctness, but placing it at the start or end keeps calculations simpler.

Q: We are rearranging terms — does this change the question itself?

A: No. $P (W, \neg T, \neg V, C)$ and $P (W, \neg T, C, \neg V)$ represent exactly the same joint probability. The order of variables in a joint probability expression does not matter — they are all inside the same probability bracket with commas, not conditioned on anything. Linearization is a computational trick to make the chain rule work correctly, not a change to the query.

Q: When the graph is big, will linearization also impact?

A: Yes. As the graph grows and more dependency layers exist, following the correct linearization order becomes increasingly important to avoid errors. With many variables, getting the order wrong can produce nonsensical results (probabilities greater than 1, or values that do not sum to 1). On the exam, always check your linearization before applying the chain rule.

12.4.3 Chain Rule

The chain rule (also called the product rule of probability) decomposes a joint probability into a product of conditional probabilities. It is a fundamental identity in probability theory — it holds for any set of random variables, regardless of independence.

Chain rule formula. Given a linearized sequence of variables $X_{1}, X_{2}, \dots, X_{n}$ :

P (X_{1}, X_{2}, \dots, X_{n}) = P (X_{1} ∣ X_{2}, \dots, X_{n}) \cdot P (X_{2} ∣ X_{3}, \dots, X_{n}) \dots P (X_{n - 1} ∣ X_{n}) \cdot P (X_{n})

Pattern: Take the first variable conditioned on all remaining ones, then the second variable conditioned on all remaining ones after removing the first, and so on. The last variable stands alone (no conditioning).

Why this works: This is just repeated application of the definition of conditional probability: $P (A, B) = P (A ∣ B) \cdot P (B)$ . Apply it once to split $X_{1}$ from the rest, then again to split $X_{2}$ from the rest, and so on.

The textbook (Russell & Norvig, §13.2) derives this formally:

P (x_{1}, \dots, x_{n}) = P (x_{n} ∣ x_{n - 1}, \dots, x_{1}) \cdot P (x_{n - 1}, \dots, x_{1}) = \dots = i = 1 \prod n P (x_{i} ∣ x_{i - 1}, \dots, x_{1})

Comparing with the Bayesian network formula $P (x_{1}, \dots, x_{n}) = \prod P (x_{i} ∣ Parents (X_{i}))$ , the network is a correct representation if and only if each variable is conditionally independent of its other predecessors, given its parents — that is, $P (X_{i} ∣ X_{i - 1}, \dots, X_{1}) = P (X_{i} ∣ Parents (X_{i}))$ .

Worked example — applying chain rule to the dental query.

Starting from the linearized query $P (W, \neg T, C, \neg V)$ (4 variables):

P (W, \neg T, C, \neg V) = P (W ∣ \neg T, C, \neg V) \cdot P (\neg T ∣ C, \neg V) \cdot P (C ∣ \neg V) \cdot P (\neg V)

Step-by-step:

Take $W$ and condition on everything else: $P (W ∣ \neg T, C, \neg V)$ . Three remaining variables: $\neg T, C, \neg V$ .
Take $\neg T$ and condition on the rest: $P (\neg T ∣ C, \neg V)$ . Two remaining: $C, \neg V$ .
Take $C$ and condition on the rest: $P (C ∣ \neg V)$ . One remaining: $\neg V$ .
Last variable stands alone: $P (\neg V)$ .

The product of all four terms equals the original joint probability. ✓

12.4.4 Canceling Irrelevant Evidence

After applying the chain rule, examine each conditional probability term and cancel any variables in the "given" part that do not actually influence the variable being conditioned. This step uses the conditional independence properties encoded in the Bayesian network.

Cancellation rule: For each term $P (X_{i} ∣ X_{i + 1}, \dots, X_{n})$ , remove from the "given" part any variable $Z$ that is not a parent (or ancestor through an unblocked path) of $X_{i}$ . Formally, if $X_{i}$ is conditionally independent of $Z$ given the remaining variables in the "given" part, then $Z$ can be dropped.

Applying cancellation to the dental query:

First term: $P (W ∣ \neg T, C, \neg V)$ . Weather ( $W$ ) is a completely independent root variable — it does not depend on toothache, catch, or cavity. So all the given variables are irrelevant:

P (W ∣ \neg T, C, \neg V) = P (W)

Second term: $P (\neg T ∣ C, \neg V)$ . Toothache depends only on cavity (its parent). Catch ( $C$ ) is not a parent of toothache — toothache and catch are conditionally independent given cavity. So:

P (\neg T ∣ C, \neg V) = P (\neg T ∣ \neg V)

Third term: $P (C ∣ \neg V)$ . Catch depends on cavity (its parent). There is nothing to cancel — only the parent is given. ✓

Fourth term: $P (\neg V)$ . Only one variable, nothing to cancel. ✓

Professor's intuition: "Sometimes in the given part, there will be a lot of evidence which will be unrelated to me. Me having toothache or not depends on cavity. But catch is also given here and it doesn't really matter to me."

Simplified expression after cancellation:

P (W, \neg T, C, \neg V) = P (W) \cdot P (\neg T ∣ \neg V) \cdot P (C ∣ \neg V) \cdot P (\neg V)

Each term now references only the variable and its parents — exactly what the CPTs provide. The cancellation step has reduced the chain rule expression to a product of local conditional probabilities, which is the Bayesian network formula in action.

12.4.5 Numerical Computation — Dental Example

Full worked computation.

After cancellation, the simplified expression is:

P (W, \neg T, C, \neg V) = P (W) \cdot P (\neg T ∣ \neg V) \cdot P (C ∣ \neg V) \cdot P (\neg V)

Step 1 — Look up each term from the CPTs:

Term	CPT Source	Row	Value
$P (W)$	Weather table, $W = T$	$W = T$	0.5
$P (\neg T ∣ \neg V)$	Toothache table	$V = F, T = F$	0.8
$P (C ∣ \neg V)$	Catch table	$V = F, C = T$	0.4
$P (\neg V)$	Cavity table	$V = F$	0.6

Step 2 — Multiply:

P (W, \neg T, C, \neg V) = 0.5 \times 0.8 \times 0.4 \times 0.6

Let us compute step by step:

$0.5 \times 0.8 = 0.4$
$0.4 \times 0.4 = 0.16$
$0.16 \times 0.6 = 0.096$

Final answer: $P (W, \neg T, C, \neg V) = 0.096$

Sense-check: This is a valid probability — it is between 0 and 1. It is reasonably small, which makes sense: we are asking for a specific combination of 4 binary variables, so the probability should be roughly $1/16 = 0.0625$ if all were uniform. Our value of 0.096 is in that ballpark. ✓

Professor's arithmetic warning: "Not 0.0096 — it is 0.096." This is a common arithmetic error students make when multiplying decimals. When you multiply $0.5 \times 0.8 \times 0.4 \times 0.6$ , the result is 0.096, not 0.0096. Count the decimal places carefully: each factor has 1 decimal place, so the product has $1 + 1 + 1 + 1 = 4$ decimal places in the raw multiplication, but the leading digits produce a value close to 0.1, not 0.01. On the exam, double-check your arithmetic by estimating the order of magnitude before computing.

Pitfall 1 — Skipping linearization. If you apply the chain rule without linearizing first, you may get a parent before its child, which breaks the conditional independence cancellation step and can produce probabilities > 1.

Pitfall 2 — Forgetting to cancel. After the chain rule, every term has extra variables in the "given" part. If you do not cancel irrelevant evidence, you cannot look up the values in the CPTs — the CPTs only give probabilities conditioned on parents, not on arbitrary other variables.

Pitfall 3 — Multiplying wrong CPT rows. When looking up $P (\neg T ∣ \neg V)$ , you need the row where $V = F$ AND $T = F$ . Students often grab the wrong row by misreading the negation. Always write out which specific values you need before looking up the table.

Recap — the complete joint query procedure:

Write the probability expression.
Classify: joint or conditional.
Linearize: children before parents.
Apply chain rule: first variable conditioned on the rest, and so on.
Cancel irrelevant evidence using conditional independence.
Substitute values from CPTs and compute.

This procedure is the foundation for all Bayesian network inference. The next section applies it to a larger, more realistic network.

12.5 Joint Probability Query: Burglary Alarm Network

12.5.1 Problem Description

Hook: You have a burglar alarm at home. It sometimes goes off during earthquakes. Two neighbors promised to call you when they hear it — but one nearly always calls (even when the phone rings and confuses him), while the other often misses it because of loud music. If both neighbors call, what is the probability there was actually a burglary?

This is the canonical Bayesian network example, due to Judea Pearl (a resident of earthquake-prone Los Angeles). It is more complex than the dental example because the alarm has two parent causes (burglary and earthquake) and two child effects (John calls and Mary calls), creating a richer dependency structure.

Key statements from the problem:

John nearly always calls when he hears the alarm. However, sometimes he confuses the telephone ring with the alarm and calls even when the alarm did not ring. This describes a false positive — $P (J = T ∣ A = F) > 0$ — and does not require adding a telephone variable (see the Q&A in Section 12.3).
Mary likes loud music and often misses the alarm altogether. This describes a false negative — $P (M = T ∣ A = T) < 1$ — meaning even when the alarm rings, Mary might not call.

Intuition: The alarm is like a smoke detector with two possible triggers: a real fire (burglary) or burnt toast (earthquake). John is the neighbor who panics at every beep. Mary is the neighbor who sleeps through everything. Both are unreliable witnesses — but combining their reports still gives you useful information about whether there was a real fire.

12.5.2 Network Construction

Variables:

$B$ — Burglary (independent, root node)
$E$ — Earthquake (independent, root node)
$A$ — Alarm (dependent: caused by $B$ or $E$ )
$J$ — John calls (dependent: influenced by $A$ )
$M$ — Mary calls (dependent: influenced by $A$ )

Dependencies (following the causal order — causes before effects):

$B$ and $E$ are independent root variables. Burglary and earthquakes happen independently.
$A$ depends on both $B$ and $E$ (edges: $B \to A$ , $E \to A$ ). Either event can trigger the alarm.
$J$ depends on $A$ (edge: $A \to J$ ). John's calling is influenced by the alarm, not directly by burglary or earthquake.
$M$ depends on $A$ (edge: $A \to M$ ). Mary's calling is also influenced only by the alarm.

The network structure:

This is a "v-structure" at $A$ (two parents converging on one child) and a "fork" at $A$ (one parent splitting to two children). These structures have important implications for conditional independence (see Section 12.8).

12.5.3 Conditional Probability Tables

The CPT values used for this example are the standard textbook values (Russell & Norvig). The professor provided $P (B) = 0.001$ , $P (E) = 0.002$ , and partial alarm table values in the lecture.

Burglary ( $B$ ) — root node:

$B$	$P (B)$
T	0.001
F	0.999

Burglaries are rare — 1 in 1000 days.

Earthquake ( $E$ ) — root node:

$E$	$P (E)$
T	0.002
F	0.998

Earthquakes are also rare — 2 in 1000 days.

Alarm ( $A$ ) given Burglary ( $B$ ) and Earthquake ( $E$ ) — 4 rows (2 parents, each binary):

$B$	$E$	$P (A ∣ B, E)$	$P (\neg A ∣ B, E)$
T	T	0.95	0.05
T	F	0.94	0.06
F	T	0.29	0.71
F	F	0.001	0.999

Reading the CPT: The professor explained:

"First row: burglary happened, earthquake also happened, and 0.95 is the probability of alarm ringing."
"Third row: burglary did not happen, earthquake happened — 0.29 alarm rings. That is why it was told fairly reliable." (The alarm responds to earthquakes, but less reliably than to burglaries.)
"Fourth row: burglary also did not happen, earthquake also did not happen, but still the alarm is ringing — 0.001." (False alarm rate.)

For the negation values: "If $P (A ∣ B, E) = 0.95$ , then $P (\neg A ∣ B, E) = 0.05$ . In plain English: burglary happened, earthquake also happened, but alarm did not ring."

Row sums: Each row sums to 1. ✓

John calls ( $J$ ) given Alarm ( $A$ ):

$A$	$P (J ∣ A)$	$P (\neg J ∣ A)$
T	0.9	0.1
F	0.05	0.95

Professor's explanation: "Alarm did not ring, but still he confused with telephone and he called. That is 0.05."

Mary calls ( $M$ ) given Alarm ( $A$ ):

$A$	$P (M ∣ A)$	$P (\neg M ∣ A)$
T	0.7	0.3
F	0.01	0.99

Mary is less reliable than John: even when the alarm rings, she only calls 70% of the time (vs. John's 90%). And when there is no alarm, she almost never calls (1% vs. John's 5%).

Scope — simplifying assumptions in this network:

Burglary and earthquake are assumed independent. In reality, an earthquake might increase burglary risk (looting), but the model ignores this.
John and Mary are assumed conditionally independent given the alarm. They do not confer before calling. In reality, they might discuss it first.
The telephone confusion and Mary's loud music are not modeled as separate variables — their effects are absorbed into the CPT entries.
All variables are binary (true/false). In reality, alarm severity, earthquake magnitude, and time-of-day would matter.

These simplifications are intentional — they keep the network tractable while capturing the essential reasoning structure.

12.5.4 Worked Example: Joint Probability Query

Query (in plain English): "Calculate the probability that the alarm has sounded, but neither burglary nor earthquake happened, but both John and Mary called."

Step 1 — Write the probability statement:

P (A, \neg B, \neg E, J, M)

Step 2 — Classify the query: This is a joint probability query. All five variables are asked together — none is given as evidence. There is no vertical bar. ✓

Step 3 — Linearize:

The original order $A, \neg B, \neg E, J, M$ has a problem: $A$ (a parent of $J$ and $M$ ) appears before $J$ and $M$ (its children). This violates the rule — children must come before parents.

Rewrite with children first:

P (J, M, A, \neg B, \neg E)

Here $J$ and $M$ (children of $A$ ) come before $A$ , and $A$ (child of $B$ and $E$ ) comes before $\neg B$ and $\neg E$ (its parents). The independent variables $B$ and $E$ are at the end. ✓

Step 4 — Apply chain rule:

P (J, M, A, \neg B, \neg E) = P (J ∣ M, A, \neg B, \neg E) \cdot P (M ∣ A, \neg B, \neg E) \cdot P (A ∣ \neg B, \neg E) \cdot P (\neg B ∣ \neg E) \cdot P (\neg E)

Step 5 — Cancel irrelevant evidence:

Term 1: $P (J ∣ M, A, \neg B, \neg E)$ . John calls depends only on the alarm state. Mary, burglary, and earthquake do not directly influence John. Cancel $M$ , $\neg B$ , $\neg E$ : $P (J ∣ M, A, \neg B, \neg E) = P (J ∣ A)$
Term 2: $P (M ∣ A, \neg B, \neg E)$ . Mary calls depends only on the alarm state. Cancel $\neg B$ and $\neg E$ : $P (M ∣ A, \neg B, \neg E) = P (M ∣ A)$
Term 3: $P (A ∣ \neg B, \neg E)$ . Alarm depends on both burglary and earthquake. Both are relevant — nothing to cancel. ✓
Term 4: $P (\neg B ∣ \neg E)$ . Burglary is independent of earthquake. Cancel $\neg E$ : $P (\neg B ∣ \neg E) = P (\neg B)$
Term 5: $P (\neg E)$ . Standalone — nothing to cancel. ✓

Step 6 — Simplified expression:

P (J, M, A, \neg B, \neg E) = P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ \neg B, \neg E) \cdot P (\neg B) \cdot P (\neg E)

This is exactly the Bayesian network formula: each term is a local conditional probability from a CPT. ✓

Step 7 — Substitute values from CPTs:

Term	CPT Source	Row	Value
$P (J ∣ A)$	John's CPT	$A = T, J = T$	0.9
$P (M ∣ A)$	Mary's CPT	$A = T, M = T$	0.7
$P (A ∣ \neg B, \neg E)$	Alarm CPT	$B = F, E = F, A = T$	0.001
$P (\neg B)$	Burglary CPT	$B = F$	0.999
$P (\neg E)$	Earthquake CPT	$E = F$	0.998

Step 8 — Multiply:

= 0.9 \times 0.7 \times 0.001 \times 0.999 \times 0.998

Step by step:

$0.9 \times 0.7 = 0.63$
$0.63 \times 0.001 = 0.00063$
$0.00063 \times 0.999 = 0.00062937$
$0.00062937 \times 0.998 = 0.000628112 \approx 0.000628$

Final answer: $P (A, \neg B, \neg E, J, M) \approx 0.000628$

Sense-check: This is a very small number, which makes sense. We are asking for a very specific scenario: alarm rings (rare when no burglary or earthquake), both John and Mary call (common if alarm rings), no burglary (common), no earthquake (common). The dominant bottleneck is the alarm ringing without any real cause ( $P = 0.001$ ), which makes the whole scenario unlikely. ✓

Recap: The burglary alarm network has 5 variables, 2 root nodes ( $B$ , $E$ ), and a v-structure at $A$ . The joint query procedure — linearize, chain rule, cancel, substitute — produces the answer directly from the CPTs. The result (0.000628) confirms that the alarm ringing without any real cause is unlikely, even though both neighbors called. Next: what if we want to infer burglary given that John and Mary called?

12.6 Conditional Probability Query with Marginalization

12.6.1 Problem Setup

Hook: Burglaries are rare (1 in 1000 days). But if both your neighbors call you at work saying the alarm went off — how worried should you be? Bayesian inference gives you the exact number.

Query (in plain English): "What is the probability that burglary happened, given that both John and Mary called?"

P (B ∣ J, M)

This is a conditional probability query — $B$ is the query, and $J$ , $M$ are the evidence. The query is at the top of the network (root) with evidence at the bottom (descendants), making it a diagnostic query (reasoning from effects back to causes).

Key challenge: Unlike the joint probability queries in Sections 12.4–12.5, this query involves only $B$ , $J$ , and $M$ . The variables $A$ (alarm) and $E$ (earthquake) are not directly in the query, yet they influence the relationship between $B$ and $J, M$ . These are called hidden variables (also called latent variables or nuisance variables) — they exist in the network but are not part of the query itself. We must account for them through marginalization — summing over all their possible values.

Professor's explanation: "In this example, for BJM, the hidden variables A and E don't come in my query. So I should perform marginalization."

Intuition: Imagine you hear that both neighbors called. You want to know if there was a burglary. But you do not know whether the alarm actually rang (maybe both neighbors were confused) or whether there was an earthquake (which could also trigger the alarm). These unknowns — alarm state and earthquake — are hidden variables. To get the right answer, you must consider every possible combination of these unknowns and weigh them by how likely each combination is.

12.6.2 Converting Conditional to Joint Probability

The key insight is to convert the conditional query into a form that involves only joint probability terms — which we already know how to compute using linearization and chain rule.

Step 1 — Apply the basic probability rule:

P (B ∣ J, M) + P (\neg B ∣ J, M) = 1

This says: given that John and Mary called, either burglary happened or it did not. The two possibilities are exhaustive and mutually exclusive.

Step 2 — Apply the conditional probability definition to each term. Recall: $P (X ∣ Y) = \frac{P ( X , Y )}{P ( Y )}$ :

\frac{P ( B , J , M )}{P ( J , M )} + \frac{P ( \neg B , J , M )}{P ( J , M )} = 1

Step 3 — Factor out $\frac{1}{P ( J , M )}$ :

\frac{1}{P ( J , M )} [P (B, J, M) + P (\neg B, J, M)] = 1

Step 4 — Define $α$ (alpha) as the normalization constant:

α = \frac{1}{P ( J , M )} = \frac{1}{P ( B , J , M ) + P ( \neg B , J , M )}

Therefore:

α [P (B, J, M) + P (\neg B, J, M)] = 1

Step 5 — Express the original query using $α$ :

P (B ∣ J, M) = \frac{P ( B , J , M )}{P ( J , M )} = α \cdot P (B, J, M)

Key insight: The right-hand side contains only joint probability terms — no conditional probabilities remain. We already know how to solve joint probability queries using linearization and chain rule (Sections 12.4–12.5). The conditional query has been converted to a form we can handle.

Professor's emphasis: "Whenever there is a Bayesian network given, and if you are directly asked the conditional probability, how we solve is: first step — apply basic probability rule, then apply conditional probability theorem, take the common factor out, keep that as alpha, and substitute."

The $α$ trick works for any number of query values. If the query variable has $k$ possible values, you compute $k$ joint probability terms and normalize them so they sum to 1. For a binary query variable like burglary ( $B$ or $\neg B$ ), you compute two terms: $P (B, J, M)$ and $P (\neg B, J, M)$ . The normalization constant $α$ ensures the final probabilities sum to 1.

12.6.3 Marginalization Over Hidden Variables

Now we need to compute $P (B, J, M)$ and $P (\neg B, J, M)$ . Both are joint probability queries, but they involve only three of the five network variables. The hidden variables $A$ and $E$ are not in the query, yet they influence $J$ and $M$ through the network. We must marginalize — sum over all possible values of the hidden variables.

Marginalization formula. To compute a joint probability over a subset of variables, sum the full joint probability over all possible values of the hidden variables:

P (B, J, M) = A \sum E \sum P (B, J, M, A, E)

where each summation $\sum_{X}$ means "add up the expression once with $X = true$ and once with $X = false .$ "

Why this works: The full joint distribution $P (B, J, M, A, E)$ covers every possible world. To get $P (B, J, M)$ , we "sum out" the variables we do not care about ( $A$ and $E$ ), accumulating probability across all their possible values. This is the sum rule (or marginalization rule) of probability.

Professor's plain-language: "Earthquake could have happened or not happened. When earthquake happened, alarm could have rung or not rung. Similarly, when earthquake did not happen also, alarm could have rung or not rung. So in all those four cases, you should compute."

What marginalization means conceptually: Earthquake can be true or false. When earthquake happens, alarm can ring or not ring. When earthquake does not happen, alarm can still ring or not ring. This gives $2^{2} = 4$ branches (two hidden variables, each binary). For each of these four combinations, we must compute the probability of our query $P (B, J, M)$ .

12.6.4 Enumeration: Expanding the Marginalization

Full worked computation of $P (B, J, M)$ .

Step 1 — Write the summation with chain rule applied:

P (B, J, M) = A \sum E \sum P (J, M, A, B, E)

Apply chain rule to the joint inside the summation (linearized order: $J, M, A, B, E$ — children before parents):

= A \sum E \sum P (J ∣ M, A, B, E) \cdot P (M ∣ A, B, E) \cdot P (A ∣ B, E) \cdot P (B ∣ E) \cdot P (E)

Step 2 — Cancel irrelevant evidence:

$P (J ∣ M, A, B, E) = P (J ∣ A)$ — John depends only on alarm.
$P (M ∣ A, B, E) = P (M ∣ A)$ — Mary depends only on alarm.
$P (A ∣ B, E)$ — alarm depends on both; keep as is.
$P (B ∣ E) = P (B)$ — burglary is independent of earthquake.
$P (E)$ — standalone.

P (B, J, M) = A \sum E \sum P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ B, E) \cdot P (B) \cdot P (E)

Step 3 — Eliminate the summation over $E$ :

To remove $\sum_{E}$ , enumerate: keep one copy with $E$ (true) and one copy with $\neg E$ (false):

= A \sum [P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ B, E) \cdot P (B) \cdot P (E)] + A \sum [P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ B, \neg E) \cdot P (B) \cdot P (\neg E)]

Step 4 — Eliminate the summation over $A$ :

Apply the same technique to each of the two lines — enumerate $A = T$ and $A = F$ :

Line 1 (with $E$ ) splits into:

$P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ B, E) \cdot P (B) \cdot P (E)$
$P (J ∣ \neg A) \cdot P (M ∣ \neg A) \cdot P (\neg A ∣ B, E) \cdot P (B) \cdot P (E)$

Line 2 (with $\neg E$ ) splits into:

$P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ B, \neg E) \cdot P (B) \cdot P (\neg E)$
$P (J ∣ \neg A) \cdot P (M ∣ \neg A) \cdot P (\neg A ∣ B, \neg E) \cdot P (B) \cdot P (\neg E)$

Professor's verbal description: "Wherever E is there, you put true, you will get the first line. Wherever E was there, you put negation E, it gave you another line. Then for A also put true false."

Result — four fully expanded terms (no summations remain):

P (B, J, M) = Term 1: A = T, E = T P (J ∣ A) P (M ∣ A) P (A ∣ B, E) P (B) P (E) + Term 2: A = F, E = T P (J ∣\neg A) P (M ∣\neg A) P (\neg A ∣ B, E) P (B) P (E) + Term 3: A = T, E = F P (J ∣ A) P (M ∣ A) P (A ∣ B, \neg E) P (B) P (\neg E) + Term 4: A = F, E = F P (J ∣\neg A) P (M ∣\neg A) P (\neg A ∣ B, \neg E) P (B) P (\neg E)

Step 5 — Substitute numerical values from CPTs:

Using the CPT values from Section 12.5.3:

Term 1: $A = T, E = T$ :

0.9 \times 0.7 \times 0.95 \times 0.001 \times 0.002 = 0.000001197

Term 2: $A = F, E = T$ :

0.05 \times 0.01 \times 0.05 \times 0.001 \times 0.002 = 0.00000000005

Term 3: $A = T, E = F$ :

0.9 \times 0.7 \times 0.94 \times 0.001 \times 0.998 = 0.000591277

Term 4: $A = F, E = F$ :

0.05 \times 0.01 \times 0.06 \times 0.001 \times 0.998 = 0.00000003

Sum:

P (B, J, M) \approx 0.000001197 + 0.00000000005 + 0.000591277 + 0.00000003 \approx 0.000592504

Observation: Term 3 dominates the sum. This makes sense: the most likely scenario for burglary + both calling is that there was no earthquake (probability 0.998) but the alarm still went off due to the burglary (probability 0.94), and both neighbors called. The other scenarios involve earthquakes (rare) or the alarm not ringing (very unlikely given a burglary).

12.6.5 Computing $P (\neg B, J, M)$

Full worked computation of $P (\neg B, J, M)$ .

The same procedure is applied with $\neg B$ instead of $B$ . The only differences:

Use $P (\neg B) = 0.999$ instead of $P (B) = 0.001$ .
Use the $\neg B$ rows of the alarm CPT: $P (A ∣ \neg B, E)$ and $P (\neg A ∣ \neg B, E)$ .

P (\neg B, J, M) = A \sum E \sum P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣ \neg B, E) \cdot P (\neg B) \cdot P (E)

Term 1: $A = T, E = T$ :

0.9 \times 0.7 \times 0.29 \times 0.999 \times 0.002 = 0.000364634

Term 2: $A = F, E = T$ :

0.05 \times 0.01 \times 0.71 \times 0.999 \times 0.002 = 0.000000709

Term 3: $A = T, E = F$ :

0.9 \times 0.7 \times 0.001 \times 0.999 \times 0.998 = 0.000627513

Term 4: $A = F, E = F$ :

0.05 \times 0.01 \times 0.999 \times 0.999 \times 0.998 = 0.000497502

Sum:

P (\neg B, J, M) \approx 0.000364634 + 0.000000709 + 0.000627513 + 0.000497502 \approx 0.001490358

Observation: Here, Term 3 and Term 4 are both significant. Term 3 corresponds to "no burglary, no earthquake, but alarm rang anyway (false alarm) and both called." Term 4 corresponds to "no burglary, no earthquake, alarm did NOT ring, but both neighbors called anyway (both confused)." The false-alarm scenario (Term 3) is slightly more likely than the double-confusion scenario (Term 4), but both contribute.

12.6.6 Final Answer

Computing $P (B ∣ J, M)$ :

P (B ∣ J, M) = \frac{P ( B , J , M )}{P ( B , J , M ) + P ( \neg B , J , M )} = \frac{0.000592504}{0.000592504 + 0.001490358}

Step 1 — Add the denominator:

0.000592504 + 0.001490358 = 0.002082862

Step 2 — Divide:

P (B ∣ J, M) = \frac{0.000592504}{0.002082862} \approx 0.2844

Final answer: $P (B ∣ J, M) \approx 0.2844$ (about 28.4%)

Sense-check: The prior probability of burglary was only 0.001 (0.1%). After both neighbors call, the posterior probability jumps to 0.2844 (28.4%) — a nearly 285-fold increase. This is the power of Bayesian updating: strong evidence (two independent witnesses calling) can dramatically shift beliefs away from the prior. The probability is not higher than 28.4% because there is still a significant chance that both neighbors called for other reasons (false alarm, earthquake, or both confused). ✓

Critical pitfall — the negation trap:

Professor's warning: "Do not do 1 minus $P (B J M)$ to get $P (\neg B J M)$ ."

The expression $P (\neg (B, J, M))$ means "the probability that NOT all three occur together" — this is $1 - P (B, J, M)$ . But $P (\neg B, J, M)$ means "the probability that $B$ is false WHILE $J$ and $M$ are true" — this is a different quantity entirely.

P (\neg (B, J, M)) = 1 - P (B, J, M) = 1 - 0.000592504 = 0.999407496

P (\neg B, J, M) = 0.001490358 (computed separately above)

These are completely different numbers! $P (\neg (B, J, M))$ is close to 1 because it is very unlikely that all three specific values occur together. $P (\neg B, J, M)$ is small because it is unlikely that both neighbors call at all.

Key rule: You must compute $P (\neg B, J, M)$ separately using the same marginalization procedure — not by subtracting from 1.

Pitfall 2 — Forgetting to marginalize. If you try to compute $P (B, J, M)$ directly without summing over $A$ and $E$ , you will be missing terms. The hidden variables must be accounted for.

Pitfall 3 — Wrong CPT rows for the $\neg B$ case. When computing $P (\neg B, J, M)$ , you need the alarm CPT rows where $B = F$ , not $B = T$ . This is a common source of arithmetic errors.

Pitfall 4 — Not normalizing. After computing $P (B, J, M)$ and $P (\neg B, J, M)$ , you must divide by their sum to get the final conditional probability. The two joint terms do not sum to 1 by themselves — they sum to $P (J, M)$ , which is typically a small number.

Recap — the complete conditional query procedure:

Write the conditional probability expression (e.g., $P (B ∣ J, M)$ ).
Apply the basic probability rule: $P (B ∣ J, M) + P (\neg B ∣ J, M) = 1$ .
Convert to joint form using $α$ : $P (B ∣ J, M) = α \cdot P (B, J, M)$ .
Compute each joint term by marginalizing over hidden variables: enumerate all combinations of hidden variable values, apply chain rule, cancel, substitute, sum.
Normalize: divide each joint term by their sum.

This is the standard method for exact inference in Bayesian networks by enumeration. The next section classifies the types of queries you might encounter.

12.7 Inference Query Types

12.7.1 Diagnostic, Causal, Intercausal, and Mixed Queries

Hook: Not all queries on a Bayesian network are created equal. The direction of evidence flow relative to the query determines the query type — and each type has different computational characteristics.

Based on the relative position of the query node and evidence nodes in the Bayesian network, queries are classified into four types. Understanding these types helps you choose the right inference strategy and anticipate the difficulty of the computation.

Diagnostic Query — evidence flows upward (from effects to causes).

Pattern: $P (cause ∣ effect)$ — "Given the symptom, what is the disease?"
Example: $P (B ∣ J)$ — "What is the probability of burglary given that John called?"
Burglary ( $B$ ) is at the top of the network, John calling ( $J$ ) is at the bottom. The evidence flows upward through the network to inform the query.
Diagnostic reasoning is what doctors do: given symptoms, infer the disease. It is called "diagnostic" because it reasons backward from diagnosis to cause.

Causal Query — evidence flows downward (from causes to effects).

Pattern: $P (effect ∣ cause)$ — "Given the disease, what is the probability of the symptom?"
Example: $P (J ∣ B, \neg A)$ — "What is the probability of John calling given burglary happened and alarm did not ring?"
Burglary ( $B$ ) is at the top (parent evidence), alarm ( $A$ ) is a parent of John, and John ( $J$ ) is at the bottom (query). The evidence is upstream.
Causal reasoning is what engineers do: given the cause, predict the effect. It is called "causal" because the evidence flows in the natural causal direction.

Intercausal Query — evidence comes from different branches converging on a shared effect, with the query in the middle.

Pattern: $P (cause_{1} ∣ cause_{2}, effect)$ — "Given cause 2 and the effect, what is cause 1?"
Example: $P (E ∣ B, M)$ — "What is the probability of earthquake given burglary happened and Mary called?"
Burglary ( $B$ ) is a parent evidence (one cause), Mary calling ( $M$ ) is a descendant evidence (shared effect), and earthquake ( $E$ ) is the query in the middle.
The pattern is E-Q-E (evidence, query, evidence) — the query node sits between two evidence nodes from different branches.
Intercausal reasoning is "explaining away": if we know burglary happened, that explains the alarm, making earthquake less likely. Evidence about one cause reduces belief in the other cause.

Mixed Query — evidence comes from both above and below the query.

Pattern: $P (middle ∣ parent, child)$ — one evidence node is a parent, another is a child.
Example: $P (A ∣ E, J)$ — "What is the probability of alarm ringing given earthquake happened and John called?"
Earthquake ( $E$ ) is a parent of alarm, and John ( $J$ ) is a child of alarm. The query ( $A$ ) sits between them.
Professor's note: "Mixed are usually complex." They combine both diagnostic and causal reasoning in a single query.

Classification exercise — identify the query type for each:

Query	Type	Reasoning
$P (B ∣ J)$	Diagnostic	Query (B) is above evidence (J)
$P (J ∣ B)$	Causal	Evidence (B) is above query (J)
$P (E ∣ B, M)$	Intercausal	Evidence from two branches: B (parent) and M (descendant)
$P (A ∣ E, J)$	Mixed	Evidence from both parent (E) and child (J) of A
$P (B ∣ J, M)$	Diagnostic	Query (B) is above both evidence nodes (J, M)

Scope — why query type matters:

Diagnostic queries (effects → causes) are the most common in practice (medical diagnosis, fault detection, spam filtering).
Causal queries (causes → effects) are the easiest to compute because evidence flows in the same direction as the edges.
Intercausal queries involve "explaining away" — a subtle phenomenon where learning about one cause reduces belief in another. This is unique to networks with multiple causes converging on a shared effect.
Mixed queries are the most complex and often require the most computation.

Recap: Query types are determined by the position of query and evidence in the DAG: diagnostic (upward), causal (downward), intercausal (across branches), mixed (both directions). On the exam, you will be given a network and a query and asked to classify the type. The key is to identify whether the evidence is above, below, or on both sides of the query node.

12.8 Conditional Independence and Markov Blanket

12.8.1 General Independence Rule

Hook: How much of the network do you actually need to look at to answer a query? Surprisingly, only a small "neighborhood" around the query node matters — everything else can be ignored.

A node in a Bayesian network is conditionally independent of its non-descendants, given its parents. This is the fundamental independence assumption that makes compact representation possible. It is the formal statement of what the DAG structure encodes.

Formal statement: For every variable $X_{i}$ in a Bayesian network:

P (X_{i} ∣ Parents (X_{i}), NonDescendants (X_{i})) = P (X_{i} ∣ Parents (X_{i}))

In plain English: once you know the values of a variable's parents, knowing anything about its non-descendants (ancestors, siblings, etc.) does not change the probability of $X_{i}$ .

Example: In the burglary network, JohnCalls is conditionally independent of Burglary, Earthquake, and MaryCalls given the value of Alarm. Once you know whether the alarm rang, knowing whether there was a burglary does not change the probability of John calling.

Intuition: Think of a water pipe system. Each junction (variable) receives water only from its direct upstream pipes (parents). Once you know the flow rate at each upstream pipe, the flow rate at the junction is determined — it does not matter what is happening further upstream or in other branches of the system. The parents "screen off" all upstream influence.

The textbook (Russell & Norvig, §13.2.1) illustrates this with Figure 13.4(a): a node $X$ is conditionally independent of all nodes in the "non-descendant" zone given its parents in the "parent" zone.

12.8.2 Markov Blanket

Markov Blanket definition. A node is conditionally independent of all other nodes in the network, given its parents, children, and children's parents (co-parents). This set — parents + children + co-parents — is called the Markov blanket of the node.

P (X_{i} ∣ MarkovBlanket (X_{i}), AllOtherNodes) = P (X_{i} ∣ MarkovBlanket (X_{i}))

The Markov blanket is the minimal set of nodes that completely shields $X_{i}$ from the rest of the network. No smaller set will do.

Professor's explanation: "A Markov blanket is the one where you are querying something given that query's parent and children. Everything else is not dependent. You are only concerned about it."

What is included in the Markov blanket:

Parents — direct causes of the node.
Children — direct effects of the node.
Co-parents (children's other parents) — these matter because they can "explain away" the node's influence on its children. If a child has two parents and you learn that one parent is true, that changes the probability of the child, which in turn affects your belief about the other parent.

Markov blanket example — burglary network.

For the Alarm node ( $A$ ) in the burglary network:

Parents: Burglary ( $B$ ), Earthquake ( $E$ )
Children: JohnCalls ( $J$ ), MaryCalls ( $M$ )
Co-parents: None (JohnCalls and MaryCalls each have only one parent — Alarm)

So the Markov blanket of Alarm is ${B, E, J, M}$ — every other node in the network. Given the values of all four, the state of Alarm is fully determined (in a probabilistic sense).

For the Burglary node ( $B$ ):

Parents: None (root node)
Children: Alarm ( $A$ )
Co-parents: Earthquake ( $E$ ) — because $E$ is also a parent of Alarm (Burglary's child)

So the Markov blanket of Burglary is ${A, E}$ . Given the values of Alarm and Earthquake, Burglary is independent of JohnCalls and MaryCalls. This makes intuitive sense: if you know the alarm state and whether there was an earthquake, knowing what John and Mary did does not add information about burglary.

The textbook (Russell & Norvig, §13.2.1) illustrates this with Figure 13.4(b): the Markov blanket of a node $X$ forms a "shield" (shown as a lavender area) that separates $X$ from all other nodes.

Q: How do we determine which hidden variables need to be summed out during exact inference? Do we marginalize over all unobserved variables or only those that influence?

A: Only those that influence. In other words, all variables in the Markov blanket — all parents, grandparents, children, and co-parents that influence the situation. Completely isolated, unobserved variables that do not influence or get influenced by the query can be ignored. For example, in the burglary network, if you are computing $P (B ∣ J, M)$ , the hidden variables are $A$ and $E$ (both in Burglary's Markov blanket). If there were an isolated "weather" variable elsewhere in the network with no connections to any of these nodes, it would not need to be marginalized.

Pitfall 1 — Confusing Markov blanket with neighborhood. The Markov blanket includes co-parents (children's other parents), which are not directly connected to the node. In the burglary network, Earthquake is in Burglary's Markov blanket even though there is no direct edge between them — they are connected only through Alarm.

Pitfall 2 — Thinking the Markov blanket blocks all influence. The Markov blanket blocks influence in a Bayesian network with the given structure. If the network structure is wrong (missing an edge), the Markov blanket will be incomplete and inference will be incorrect.

Recap: A node is conditionally independent of its non-descendants given its parents. The Markov blanket (parents + children + co-parents) is the minimal shield that separates a node from the rest of the network. For inference, only variables in the Markov blanket need to be considered — isolated variables can be ignored. This principle is what makes exact inference tractable in practice.

12.9 Exact Inference: Enumeration

12.9.1 The Enumeration Method

Hook: You have seen the full procedure in Sections 12.4–12.6. Now we give it a name and formalize it as a general-purpose algorithm.

Enumeration is the exact inference technique demonstrated in Sections 12.4–12.6. The name comes from the core operation: enumerating every possible value of each hidden variable and summing the results. It is the simplest exact inference method for Bayesian networks, and understanding it thoroughly is essential before moving to more efficient methods like variable elimination.

The Enumeration Algorithm (step by step):

Convert conditional queries to joint form using the $α$ normalization technique (Section 12.6.2).
Identify hidden variables — network variables not in the query that influence the query through the Markov blanket (Section 12.8). Isolated variables that do not influence or get influenced by the query can be ignored.
Marginalize by writing summations $\sum_{X}$ over each hidden variable.
Apply chain rule inside the summation to decompose the joint probability into a product of conditional probabilities.
Cancel irrelevant evidence using the conditional independence properties encoded in the network structure.
Enumerate each hidden variable: for each summation $\sum_{X}$ , expand the expression into two copies — one with $X = true$ and one with $X = false$ . This eliminates the summation symbol. Each binary hidden variable doubles the number of terms.
Substitute numerical values from the CPTs and compute the final answer.

Worked trace — burglary example (recap).

For $P (B ∣ J, M)$ :

Hidden variables: $A$ (alarm) and $E$ (earthquake) — 2 binary variables.
After enumeration: $2^{2} = 4$ terms.
Each term is a product of 5 CPT lookups.
Total CPT lookups: $4 \times 5 = 20$ (plus another 20 for the $\neg B$ case = 40 total).
Final answer: $P (B ∣ J, M) \approx 0.2844$ .

Complexity. For $k$ hidden binary variables, the number of terms after full enumeration is $2^{k}$ . Each term requires looking up and multiplying $n$ CPT entries (where $n$ is the number of variables in the network). So the total work is $O (n \cdot 2^{k})$ .

In the burglary example: $k = 2$ hidden variables, $n = 5$ total variables → $O (5 \cdot 4) = O (20)$ operations per joint term. Very manageable.
But if $k = 30$ hidden variables: $2^{30} \approx 1$ billion terms. Completely intractable.

This exponential blowup is the fundamental limitation of enumeration. It works well for small networks but becomes impractical for large ones.

Professor's insight on the number of hidden variables: "If the missing variable is separately isolated somewhere, like ice is falling, it doesn't really matter. It is not getting influenced, nor influencing. It is a fully isolated independent variable. You can leave it alone."

Pitfall 1 — Counting hidden variables incorrectly. Only variables in the Markov blanket of the query (and evidence) that are not themselves in the query or evidence are hidden. Isolated variables do not count.

Pitfall 2 — Exponential blowup. Each hidden binary variable doubles the work. Three variables give 8 terms; ten give 1024; thirty give over a billion. On the exam, problems will have 2–3 hidden variables to keep the computation manageable.

Pitfall 3 — Forgetting to enumerate both values. When expanding $\sum_{X}$ , you must include both $X = T$ and $X = F$ . Missing one value gives an incomplete (incorrect) sum.

Recap: Enumeration is the brute-force exact inference method: convert conditional to joint, marginalize by summing over hidden variables, chain rule, cancel, enumerate, compute. It is conceptually simple but exponentially expensive in the number of hidden variables. The next section introduces variable elimination, a smarter approach that avoids the exponential blowup by eliminating variables one at a time.

12.10 Exact Inference: Variable Elimination

12.10.1 Concept and Preview

Hook: Enumeration works, but it doubles the work for every hidden variable. What if we could eliminate hidden variables one at a time, shrinking the problem as we go, instead of expanding it exponentially?

Variable elimination is a more efficient exact inference technique that avoids the exponential blowup of full enumeration. Instead of expanding all hidden variables simultaneously into $2^{k}$ terms, it eliminates them one at a time, absorbing each variable's effect into its dependent nodes' CPTs. This process progressively shrinks the Bayesian network until only the query variable remains.

Intuition: Imagine you are packing for a trip and you have a pile of clothes (variables) to fit into a suitcase (the final answer). Enumeration is like laying everything out at once on the floor — the pile gets huge. Variable elimination is like folding each item and putting it in the suitcase one at a time — the suitcase stays manageable. You are doing the same total work, but in a structured way that never creates a huge intermediate pile.

How variable elimination differs from enumeration:

Enumeration: Expand all hidden variables at once → $2^{k}$ terms → multiply and sum.
Variable elimination: Process one hidden variable at a time → multiply its CPT into its child's CPT → the hidden variable disappears → repeat with the next hidden variable.

The key operation is pointwise multiplication of factors (CPTs) followed by summing out one variable. Each elimination step produces a new, smaller factor that replaces the old ones.

12.10.2 Example: Fever Network

The professor introduced a fever network with four variables to illustrate variable elimination:

Viral infection ( $V$ ) — independent, root node
Food poisoning ( $F P$ ) — independent, root node
Flu ( $F$ ) — dependent on $V$
Fever ( $F e$ ) — dependent on $F$ and $F P$

The network structure:

Query: $P (F e)$ — "What is the probability of someone having fever?"

This is a total probability query — no evidence is given, no conditioning. It is neither a joint nor a conditional query as defined in earlier sections. It asks for the marginal probability of fever, accounting for all possible combinations of viral infection, food poisoning, and flu.

Variable elimination procedure (preview):

Step 1 — Eliminate $V$ (viral infection):

Merge the effect of $V$ into $F$ 's table. $V$ is a parent of $F$ , so the CPT $P (F ∣ V)$ already encodes how viral infection affects flu. To eliminate $V$ , we compute the marginal distribution of $F$ :

P (F) = V \sum P (F ∣ V) \cdot P (V)

After this step, $V$ disappears from the network. $F$ becomes a root variable with updated probabilities that already account for the viral infection influence. The network is now: $F$ , $F P$ , $F e$ (three tables).

Step 2 — Eliminate $F$ and $F P$ :

Now both $F$ and $F P$ are parents of $F e$ . Merge their effects into $F e$ 's table by computing:

P (F e) = F \sum F P \sum P (F e ∣ F, F P) \cdot P (F) \cdot P (F P)

After this step, both $F$ and $F P$ disappear. The network is: $F e$ only (one table with the final probabilities).

Step 3 — Read the answer:

The final table for $F e$ gives:

$P (F e = T)$ — probability of having fever
$P (F e = F)$ — probability of not having fever

Professor's result: $P (F e) = 0.46$ .

Critical pitfall — do not just sum the fever CPT rows:

Professor's warning: "If you just go to the fever table and add all the cases — all four rows — and tell this is the probability of fever, that is wrong. If you add all four, what will be the answer? Will it be less than one? Greater than one? Then the whole point is wrong."

Simply summing $P (F e ∣ F, F P)$ values across all four parent configurations is incorrect because each row corresponds to a different parent configuration with different probabilities. You must weight each row by the probability of that parent configuration:

P (F e) = F \sum F P \sum P (F e ∣ F, F P) \cdot P (F) \cdot P (F P)

The weights $P (F)$ and $P (F P)$ are not uniform — some parent configurations are much more likely than others. Variable elimination properly accounts for these weights.

Professor's hint on elimination order: "Who is dependent? I want fever. Which means I have to merge their parents. If they should be alone, then its parent has to be merged. If there are other random variables which are not related, I will not include them."

The optimal elimination order follows the dependency chain from leaves toward the query variable. Start with variables that are farthest from the query and work inward.

Why variable elimination is more efficient than enumeration:

In the fever network:

Enumeration would expand $V$ , $F$ , and $F P$ simultaneously: $2^{3} = 8$ terms (if all binary).
Variable elimination processes them one at a time: eliminate $V$ first (producing a small table for $F$ ), then eliminate $F$ and $F P$ together (producing the final table for $F e$ ). The intermediate tables never grow as large as the fully enumerated expression.

In general, the complexity of variable elimination depends on the treewidth of the network — a measure of how "tree-like" the graph is. For tree-structured networks, variable elimination runs in time linear in the number of nodes. For dense networks, it can still be exponential, but typically much less so than full enumeration.

Pitfall 1 — Wrong elimination order. Eliminating a variable before its parents have been merged into it can produce incorrect intermediate tables. Always eliminate from leaves toward the query.

Pitfall 2 — Forgetting to multiply by parent probabilities. When marginalizing out a variable, you must weight each CPT row by the probability of that parent configuration. Simply summing CPT rows ignores the likelihood of each configuration.

Pitfall 3 — Confusing marginalization with averaging. Marginalization is a weighted sum, not an unweighted average. The weights come from the parent probability distributions.

Recap: Variable elimination avoids the exponential blowup of enumeration by eliminating hidden variables one at a time, absorbing each into its child's CPT. The procedure shrinks the network until only the query variable remains. The fever example gives $P (F e) = 0.46$ . The next lecture will cover variable elimination in full detail (using the fever example with actual CPT values), approximate inference techniques, and Hidden Markov Models.

12.11 Exam Guidance Summary

12.11.1 Expected Question Types and Exam Tips

Exam note: The professor emphasized multiple times that Bayesian network problems will definitely appear in the exam. This is one of the most heavily tested topics in the course.

Question types to expect:

Constructing a Bayesian network from an English problem description. You will be given a scenario (like the dental or burglary example) and asked to identify variables, draw the DAG, and fill in CPTs.
Joint probability queries. Given a Bayesian network with CPTs, compute a specific joint probability. Full procedure: linearize → chain rule → cancel irrelevant evidence → substitute → compute. Show all intermediate steps.
Conditional probability queries. Given a Bayesian network with CPTs, compute a conditional probability like $P (B ∣ J, M)$ . Full procedure: apply basic probability rule → convert to joint via $α$ → marginalize over hidden variables → enumerate → compute. This will be a lengthy problem.
Query type classification. The network diagram and a query will be given. Students must classify the query as diagnostic, causal, intercausal, or mixed.

What will be given in the exam:

The Bayesian network diagram may or may not be given. If not given, you must construct it from the text description.
CPT data will be given as text statements (e.g., "the probability of having cavity is 40%"), not always as formatted tables. You must deduce which CPT cell each statement corresponds to. This is a common source of confusion — practice mapping English descriptions to CPT rows.
Sometimes the full Bayesian network with CPTs will be given directly.
A calculator is allowed in the exam.

What is expected of you:

Show all intermediate steps — linearization, chain rule expansion, cancellation, and substitution. Partial credit is available for correct steps even if the final arithmetic is wrong.
Write all assumptions explicitly — state which variables are independent, which are conditionally independent, and why you can cancel certain evidence.
Present work in a table format where helpful — it is easier to grade and helps you avoid arithmetic errors.

Difficulty warning from the professor: "This will be a very lengthy problem from exam point of view. There are shortcuts to do this problem. I don't want to tell that shortcut. We did the right way, and I want you also to do the right way. If you solve one or two problems, you will understand what the trick is."

Exam note — study advice: Practice the burglary alarm conditional probability problem (Section 12.6) at home. The professor noted that while the explanation seems simple in class, it becomes tricky during exam preparation if not practiced. Do at least two full problems end-to-end: one joint query and one conditional query with marginalization.

Upcoming topics: The next lecture will cover variable elimination in detail (using the fever example with actual CPT values), approximate inference techniques, and Hidden Markov Models (temporal probability — how past observations influence current predictions). These will complete the current module.

12.12 Key Industry Applications

12.12.1 Real-World Connections

Hook: Bayesian networks are not just a textbook exercise — they are actively used in industries where transparency, auditability, and reasoning under uncertainty matter more than raw predictive power.

Rule-based AI systems in closed organizations. Organizations that cannot use external LLMs (due to data privacy, regulatory constraints, or domain-specific requirements) build their own AI systems using logic-based reasoning, first-order logic, and Bayesian networks. These are not obsolete — they are actively used in finance (fraud detection with explainable models), healthcare (clinical decision support where regulators require reasoning traces), and defense (systems that must operate without cloud connectivity).
Domain-specific model building in dissertations and research. Students and researchers build custom models for specific domains using reasoning techniques, knowledge representation, and fine-tuning of small language models, rather than calling cloud APIs. Bayesian networks are a foundational tool for such work — they provide a formal framework for encoding domain knowledge and performing principled inference.
Burglary alarm and sensor fusion systems. The canonical example illustrates real-world sensor fusion — combining multiple unreliable signals (alarm sensors, human observers, camera feeds) to infer unobservable states (burglary, earthquake, fire). This pattern appears in intrusion detection, autonomous vehicle perception (combining lidar, radar, and camera), industrial fault diagnosis, and any system that combines noisy sensors.
Medical diagnosis. The dental example (cavity, toothache, catch) models real diagnostic reasoning — inferring the presence of a condition (cavity) from observable symptoms (toothache) and test results (catch). Bayesian networks are widely used in clinical decision support systems (e.g., the QMR-DT network for internal medicine, the PATHFINDER network for lymph node pathology). The key advantage over black-box models is explainability: a doctor can trace exactly which evidence contributed to the diagnosis.
Temporal reasoning and prediction. Hidden Markov Models (mentioned as the next topic) are used in speech recognition (decoding phonemes from audio signals), activity recognition (inferring human actions from sensor data), financial time series (modeling regime changes in markets), and weather prediction (tracking storm systems). These are Bayesian networks extended to sequential data, where past states influence current observations.

ACI Lecture 12 notes · Bayesian Networks

Artificial Computational Intelligence· postgraduate· 2026-07-26

Sections Breakdown

1Review of Foundations: From Logic to Probability

Course arc recap from propositional logic through first-order logic to probability theory, motivating Bayesian networks.

2Bayesian Network Fundamentals

Definition of Bayesian networks as DAGs with CPTs, compact representation of joint distributions, construction procedure.

3Construction: Dental Health Example

4-variable dental network (Weather, Cavity, Toothache, Catch) with CPTs and conditional independence demonstration.

4Linearization and Chain Rule

Joint vs conditional query classification, linearization procedure, chain rule decomposition, cancellation of irrelevant evidence.

5Joint Probability Query: Burglary Alarm Network

5-variable burglary alarm network construction and full worked joint probability query yielding 0.000628.

6Conditional Probability Query with Marginalization

Converting conditional to joint via alpha normalization, marginalizing over hidden variables, computing P(B|J,M) = 0.2844.

7Inference Query Types

Classification of queries as diagnostic, causal, intercausal, or mixed based on evidence flow direction.

8Conditional Independence and Markov Blanket

General independence rule, Markov blanket definition (parents + children + co-parents), minimal shielding property.

9Exact Inference: Enumeration

7-step enumeration algorithm with O(n * 2^k) complexity analysis and exponential blowup limitation.

10Exact Inference: Variable Elimination

More efficient exact inference eliminating variables one at a time, fever network example, P(Fe) = 0.46.

11Exam Guidance Summary

Expected exam question types, study advice, and difficulty warnings from the professor.

12Key Industry Applications

Real-world applications in sensor fusion, medical diagnosis, closed-organization AI, and temporal reasoning.

Postgraduate students in Artificial Computational Intelligence

Exam Revision Notes

Below is the distilled, exam-ready core. Every entry comes from the full explanation above. Use this section for rapid review; return to the main notes when a point needs more context.

Review of Foundations: From Logic to Probability

Must-know: The course arc moves from deterministic logic to probabilistic reasoning because real-world problems involve uncertainty. Bayesian networks are the primary tool for representing and reasoning with uncertain knowledge.

⚠️ Top pitfall: Confusing the role of each reasoning paradigm — propositional logic handles binary facts, first-order logic adds quantifiers, probability handles uncertainty. Do not try to solve uncertain problems with pure logic.

Self-check: Why can propositional logic not handle statements like 'all humans are mortal'?

Connects to: 12.2, 12.3

Bayesian Network Fundamentals

Must-know: A Bayesian network = DAG + CPTs. The full joint distribution is the product of all conditional probabilities: P(x1,...,xn) = prod P(xi | Parents(Xi)). CPT rows sum to 1; columns do not.

$P (x_{1}, \dots, x_{n}) = i = 1 \prod n P (x_{i} ∣ Parents (X_{i}))$

⚠️ Top pitfall: Confusing 'ML' as machine learning when it means marginalization in this context. Also: adding CPT column values — columns do NOT sum to 1.

Self-check: How many rows does the CPT for a binary variable with 3 binary parents have?

Connects to: 12.3, 12.8

Construction: Dental Health Example

Must-know: Identify root vs dependent variables from problem description. Draw edges from causes to effects. CPT rows sum to 1. Toothache and Catch are conditionally independent given Cavity — knowing one does not help once you know the other through Cavity.

$P (T, C ∣ V) = P (T ∣ V) \cdot P (C ∣ V)$

⚠️ Top pitfall: Adding a telephone node when the problem mentions John confusing telephone with alarm — this is a false positive captured in the CPT, not a new variable. Also: confusing conditional independence with marginal independence.

Self-check: In the dental network, what are the parents of Toothache? What is the CPT size?

Connects to: 12.2, 12.4, 12.8

Linearization and Chain Rule

Must-know: Always linearize before applying chain rule. Children must appear before parents. After chain rule, cancel irrelevant evidence. Common error: 0.5 x 0.8 x 0.4 x 0.6 = 0.096, not 0.0096.

$P (X_{1}, \dots, X_{n}) = P (X_{1} ∣ X_{2}, \dots, X_{n}) \cdot P (X_{2} ∣ X_{3}, \dots, X_{n}) \dots P (X_{n})$

⚠️ Top pitfall: Skipping linearization produces wrong answers (possibly > 1). Arithmetic errors in decimal multiplication. Forgetting to cancel irrelevant evidence before looking up CPT values.

Self-check: What is the linearized form of P(W, not-T, not-V, C) in the dental network?

Connects to: 12.3, 12.5, 12.6

Joint Probability Query: Burglary Alarm Network

Must-know: The burglary alarm network has a v-structure at A (two parents B,E). Apply the full joint query procedure: linearize, chain rule, cancel, substitute. The answer 0.000628 reflects the low prior probability of the alarm ringing without cause.

$P (J, M, A, \neg B, \neg E) = P (J ∣ A) \cdot P (M ∣ A) \cdot P (A ∣\neg B, \neg E) \cdot P (\neg B) \cdot P (\neg E)$

⚠️ Top pitfall: Forgetting to linearize (A before J,M is wrong). Using the wrong row of the 4-row alarm CPT. Confusing P(A|not-B,E) with P(A|B,not-E).

Self-check: In the burglary network, what are the parents of the Alarm variable? How many rows does its CPT have?

Connects to: 12.4, 12.6, 12.7

Conditional Probability Query with Marginalization

Must-know: Convert conditional to joint via alpha: P(B|JM) = alpha × P(B,J,M). Marginalize by summing over hidden variables. Do NOT compute P(not-B,J,M) as 1 − P(B,J,M) — compute it separately. Final answer: 0.2844.

$P (B ∣ J, M) = α \cdot P (B, J, M) = \frac{P ( B , J , M )}{P ( B , J , M ) + P ( \neg B , J , M )}$

⚠️ Top pitfall: Using 1 - P(BJM) to get P(not-B,J,M) — these are different quantities. Forgetting to normalize by alpha. Using wrong CPT rows for the not-B case.

Self-check: If P(B,J,M) = 0.000592 and P(not-B,J,M) = 0.001490, what is P(B|J,M)?

Connects to: 12.5, 12.7, 12.9

Inference Query Types

Must-know: Four query types: diagnostic (P(cause|effect)), causal (P(effect|cause)), intercausal (P(cause1|cause2,effect) — explaining away), mixed (evidence from both parents and children). Classify by whether evidence is above, below, or on both sides of the query.

⚠️ Top pitfall: Confusing intercausal with mixed queries. Intercausal has evidence from two different branches at the same level; mixed has evidence from both parents and children of the query.

Self-check: What type of query is P(E|B,M) in the burglary network? Why?

Connects to: 12.5, 12.6, 12.8

Conditional Independence and Markov Blanket

Must-know: Markov blanket = parents + children + co-parents. A node is independent of everything outside its Markov blanket given the blanket nodes. For inference, only Markov blanket variables need to be considered.

$P (X_{i} ∣ Parents (X_{i}), NonDescendants (X_{i})) = P (X_{i} ∣ Parents (X_{i}))$

⚠️ Top pitfall: Forgetting co-parents in the Markov blanket. Co-parents matter because of explaining away — if a child has two parents, learning one parent changes belief about the other.

Self-check: What is the Markov blanket of the Burglary node in the alarm network?

Connects to: 12.2, 12.7, 12.9

Exact Inference: Enumeration

Must-know: Enumeration: 7-step procedure. For k hidden binary variables, 2^k terms after expansion. Works for small networks (k < ~20), intractable for large ones.

⚠️ Top pitfall: Counting hidden variables incorrectly (only Markov blanket variables count). Forgetting to enumerate both true and false values. Exponential blowup with many hidden variables.

Self-check: How many terms result from enumerating 3 hidden binary variables?

Connects to: 12.6, 12.10

Exact Inference: Variable Elimination

Must-know: Variable elimination eliminates variables one at a time by merging their CPTs into child CPTs, then summing out. Do NOT just sum all fever CPT rows — must weight by parent probabilities. Elimination order: leaves toward query.

$P (F e) = F \sum F P \sum P (F e ∣ F, F P) \cdot P (F) \cdot P (F P)$

⚠️ Top pitfall: Summing CPT rows without weighting by parent configuration probabilities. Wrong elimination order. Confusing marginalization with averaging.

Self-check: Why is it wrong to add all four rows of the fever CPT to get P(fever)?

Connects to: 12.9, 12.6

Exam Guidance Summary

Must-know: BN problems are guaranteed on exam. CPT data given as text statements, not tables. Show all intermediate steps. Practice the burglary alarm conditional query problem end-to-end.

⚠️ Top pitfall: Not showing intermediate steps (loss of partial credit). Misreading CPT values from text descriptions. Not writing assumptions explicitly.

Self-check: List the four question types expected in the exam for Bayesian networks.

Connects to: 12.3, 12.4, 12.6, 12.7

Key Industry Applications

Must-know: Bayesian networks provide explainable reasoning — you can trace which evidence contributed to a conclusion. This makes them preferred over black-box models in healthcare, finance, and defense.

⚠️ Top pitfall: Assuming Bayesian networks are obsolete because of deep learning. They remain the tool of choice when explainability, auditability, or small data is required.

Self-check: Name two industries where Bayesian networks are preferred over black-box models and explain why.

Connects to: 12.1, 12.2

Previous LectureLecture 11 · First Order Logic, Forward Chaining, and Backward Chaining

Next LectureLecture 13 · Bayesian Networks: D-Separation, Variable Elimination, and Approximate Inferencing

Was this lecture useful?

Loading comments…

🤖

BitsNotes AI Assistant

Subject Notes Assistant

Configure AI Chat

Choose how to access the chatbot

Have your own API key?

Switch to "Bring Your Own Key" tab above for unlimited access with any OpenAI-compatible provider.

AI Provider Preset

API Endpoint URL (OpenAI-Compatible)

API Key

Model Name

🔑 Enter API key above to fetch live models from provider, or enter model name manually.

Remember key for this session (until tab closes)

OpenAI-Compatible API Support

Choose any provider preset (Gemini, DeepSeek, Kimi, GLM, MiniMax, Qwen, OpenAI, Groq, Ollama, etc.) or enter a custom endpoint URL.

Security & Privacy First

Your API key is sent directly from your browser to your specified provider. BitsNotes servers never store or see your key.

Bayesian Networks

12.1 Review of Foundations: From Logic to Probability

12.1.1 Course Arc Recap

12.2 Bayesian Network Fundamentals

12.2.1 Definition and Purpose

12.2.2 Key Properties

12.2.3 Construction Procedure

12.3 Construction: Dental Health Example

12.3.1 Problem Setup

12.3.2 Drawing the Network

12.3.3 Conditional Probability Tables

12.4 Linearization and Chain Rule

12.4.1 Query Classification: Joint vs. Conditional

12.4.2 Linearization

12.4.3 Chain Rule

12.4.4 Canceling Irrelevant Evidence

12.4.5 Numerical Computation — Dental Example

12.5 Joint Probability Query: Burglary Alarm Network

12.5.1 Problem Description

12.5.2 Network Construction

12.5.3 Conditional Probability Tables

12.5.4 Worked Example: Joint Probability Query

12.6 Conditional Probability Query with Marginalization

12.6.1 Problem Setup

12.6.2 Converting Conditional to Joint Probability

12.6.3 Marginalization Over Hidden Variables

12.6.4 Enumeration: Expanding the Marginalization

12.6.5 Computing P(¬B,J,M)

12.6.6 Final Answer

12.7 Inference Query Types

12.7.1 Diagnostic, Causal, Intercausal, and Mixed Queries

12.8 Conditional Independence and Markov Blanket

12.8.1 General Independence Rule

12.8.2 Markov Blanket

12.9 Exact Inference: Enumeration

12.9.1 The Enumeration Method

12.10 Exact Inference: Variable Elimination

12.10.1 Concept and Preview

12.10.2 Example: Fever Network

12.11 Exam Guidance Summary

12.11.1 Expected Question Types and Exam Tips

12.12 Key Industry Applications

12.12.1 Real-World Connections

ACI Lecture 12 notes · Bayesian Networks

Sections Breakdown

Exam Revision Notes

Review of Foundations: From Logic to Probability

Bayesian Network Fundamentals

Construction: Dental Health Example

Linearization and Chain Rule

Joint Probability Query: Burglary Alarm Network

Conditional Probability Query with Marginalization

Inference Query Types

Conditional Independence and Markov Blanket

Exact Inference: Enumeration

Exact Inference: Variable Elimination

Exam Guidance Summary

Key Industry Applications

Was this lecture useful?

BitsNotes AI Assistant

Configure AI Chat

Sign in to use AI Chat

12.6.5 Computing $P (\neg B, J, M)$