—

Advanced Statistical Methods

Eigenvectors, Cluster Analysis, and Non-Parametric Procedures

Published: 2026-08-11

Level: postgraduate

Audience: Postgraduate students in Advanced Statistical Methods

Prerequisite Knowledge

This lecture builds on the following concepts from earlier lectures. If any feel unfamiliar, review the linked notes before proceeding.

Previously Covered in This Subject

Eigenvalues and eigenvectors — covered in Lecture 13 (principal component analysis: the characteristic equation and principal directions) and Lecture 14 (eigenvalues and the share of variation in the LDA worked data)
Cluster analysis — covered in Lecture 14 (hierarchical cluster analysis: the distance matrix, the agglomeration schedule, and the dendrogram)
Hypothesis testing and the normal tables — covered in Lectures 2 to 4 (Z-scores, the Z test, and the p-value approach)

This session closes out the two review topics left over from the earlier discussions — eigenvalues and eigenvectors, and cluster analysis. It then opens a new family of tools: non-parametric procedures, with the sign test worked end to end and a preview of the run test. Two sessions remain in the course. Today's completes the leftover topics. The final session is planned as a full review from the beginning of the course, with guidance on how the paper is set up and how to prepare for the comprehensive examination.

15.1 Eigenvalues and Eigenvectors

15.1.1 Vector Representation: Two Equivalent Notations

Hook — why we are here. In the earlier sessions on principal component analysis and Fisher's linear discriminant analysis, two mathematical objects kept appearing at the center of everything: eigenvalues and eigenvectors. Both problems leaned on them heavily, and the reason becomes plain in a moment: an eigenvector is a direction that a matrix does not bend. If you know which directions a matrix leaves alone, you know how it is built — and that is why the same two objects appear in principal component analysis, in discriminant analysis, and again in cluster analysis below.

A vector can be represented in two equivalent notations. The first uses the unit vectors $i$ and $j$ — unit vectors meaning vectors of length 1 pointing along the $x$ -axis and the $y$ -axis respectively. In that notation a simple vector is written as

$r = 1 i + 3 j$

The second, and the one preferred in most mathematical treatments, is the column vector:

$r = [13]$

Both forms describe the same object. Geometrically, in a two-dimensional plane with $x$ along the horizontal axis and $y$ along the vertical axis, the vector $r$ lands at the point $(1, 3)$ : 1 step along the $x$ -axis and 3 steps along the $y$ -axis. To draw the vector itself we always draw the direction passing through the origin — the arrow from the origin to the point $(1, 3)$ . Nothing about the arrow's length is fixed by this picture; the arrow just marks a direction and a distance from the origin.

Intuition + analogy — an arrow is a move on a map. Think of a hiker standing at the origin of a map grid. The vector $1 i + 3 j$ tells the hiker to walk 1 pace east and 3 paces north; the column vector $[13]$ says exactly the same thing. The two notations are just two ways of writing one instruction. Where the analogy breaks: a hiker walks one pace after the other, while the vector form treats both steps as a single arrow — the order does not matter, only the end point $(1, 3)$ .

Worked example — reading both notations from the same arrow. Take the vector with tip at $(1, 3)$ . In unit-vector form: $r = 1 i + 3 j$ , because the $x$ -coordinate is 1 and the $y$ -coordinate is 3. In column form:

$r = [13]$

Draw the arrow from the origin $(0, 0)$ to the tip $(1, 3)$ . Now double the coordinates to $(2, 6)$ . The arrow points in the same direction but is twice as long — this direction-versus-length distinction is the seed of the eigenvalue idea: the arrow's direction is its identity, and scaling the coordinates changes only its length. Sense-check: both notations name the same point $(1, 3)$ , and no information is lost in moving between them; the two forms are interchangeable by convention, with the column form the default for matrix work.

15.1.2 Matrix–Vector Multiplication and Direction Change

Now multiply this vector by a matrix. Take

$A = [1221], x = [13]$

and compute $A x$ :

$A x = [1221] [13] = [1 (1) + 2 (3) 2 (1) + 1 (3)] = [75]$

The first row of the product: 1 times 1 plus 2 times 3 gives 7. The second row: 2 times 1 plus 1 times 3 gives 5. So the matrix takes the old vector $(1, 3)$ and produces a completely new vector $(7, 5)$ . Plot both in the same diagram: the arrow for $(1, 3)$ points one way, and the arrow for $(7, 5)$ points another way. The key observation is simple: multiplying a vector by a matrix generally changes the vector's direction. That is the typical behavior of a matrix–vector product.

Why bother with such a matrix? Because matrices like this are exactly the kind we meet as covariance matrices later, and this direction-changing behavior is what makes eigenvalue analysis interesting. A matrix-vector product gives a new vector in some new predominant direction — unless the vector happens to be special, as the next example shows.

Worked example — the multiplication, step by step. The product rule: the $i$ -th entry of $A x$ is the dot product of the $i$ -th row of $A$ with $x$ . Row 1 is $(1, 2)$ , so

$1 (1) + 2 (3) = 1 + 6 = 7$

Row 2 is $(2, 1)$ , so

$2 (1) + 1 (3) = 2 + 3 = 5$

So $A x = (7, 5)$ . Sense-check: each entry of the result is a weighted sum of the input coordinates, so the result is a genuine mix of $x$ and $y$ — a mix that is exactly why the direction can change.

For a visual reading: draw two axes crossing at the origin. Arrow 1 goes from $(0, 0)$ to $(1, 3)$ — steep, close to the vertical axis. Arrow 2 goes from $(0, 0)$ to $(7, 5)$ — shallower, closer to the horizontal axis. The two arrows point in visibly different directions, which is the whole point: the matrix rotated the vector off its original line. The takeaway of the picture: for a generic vector, a matrix acts like a device that turns and rescales; only for special vectors does it fail to turn at all.

Assumptions & scope — what this example assumes. The multiplication rule above assumes compatible shapes: a $2 \times 2$ matrix acting on a $2 \times 1$ vector. The number of columns of $A$ must equal the number of rows of $x$ ; otherwise the product is not defined. The direction-changing behavior is the typical case for non-identity matrices; the identity matrix $I$ is the exception that leaves every vector untouched. This scope note matters because the eigenvalue equation below hunts exactly for the vectors where the "typical" turning behavior fails.

15.1.3 Eigenvectors: Direction Fixed, Magnitude Scaled

Take the same matrix $A = [1221]$ but choose the vector $x = [11]$ . Then

$A x = [1221] [11] = [33] = 3 [11]$

Here something surprising happens: there is no change in the direction at all — the new vector $(3, 3)$ points exactly along the old vector $(1, 1)$ . Only the magnitude changes: the result is 3 times the old vector. A vector with this behavior — a vector whose direction the matrix leaves alone, changing only its magnitude — is called an eigenvector (from the German "eigen", meaning own or characteristic: the vector the matrix can call its own). Such vectors are the reason for this whole discussion: for a given matrix, most vectors get twisted into new directions, but some special vectors keep their direction and merely stretch or shrink.

A second example makes the pattern unmistakable. This time take

$A = [73 3 - 1], x = [31]$

Multiplying: $7 (3) + 3 (1) = 21 + 3 = 24$ in the first row, and $3 (3) + (- 1) (1) = 9 - 1 = 8$ in the second row, so

$A x = [73 3 - 1] [31] = [248] = 8 [31]$

Again the new vector $(24, 8)$ is a scalar multiple of the old one $(3, 1)$ : same direction, 8 times the magnitude. We can express this general pattern as

$A x = λ x$

where $λ$ (lambda) is the eigenvalue — the number the vector is scaled by — and $x$ is the eigenvector. In the first example $λ = 3$ with eigenvector $(1, 1)$ ; in the second, $λ = 8$ with eigenvector $(3, 1)$ . So we have found vectors such that $A x$ can be written as $λ x$ : the matrix behaves on these directions as if it were just a number. Contrast the two cases: for a generic vector, multiplying by a matrix produces a change in direction; for an eigenvector, there is no change in direction, only a change in magnitude. This special, "beautiful" behavior is exactly what eigenvalues and eigenvectors are about.

Intuition + analogy — the stretching machine. Picture a machine that holds a sheet of graph paper and can stretch or turn it. Feed it a grid drawn on the sheet. Most lines on the grid come out rotated to new directions — the matrix behavior of the previous section. But one or two special lines come out lying exactly along their old directions: the machine only stretched them, pulling every point along that line to a fixed multiple of its old distance from the origin. Those lines are the eigenvectors; the multiple is the eigenvalue. The analogy maps cleanly: the sheet is the plane, the machine is $A$ , a line through the origin that keeps its direction is an eigenvector, and the stretch factor (3 in the first example, 8 in the second) is $λ$ . Where the analogy breaks: a physical sheet can bend in many exotic ways, while a matrix only produces straight-line, linear effects — but for understanding "stretched, never turned", the machine is enough.

Pitfalls — traps around the definition.

The zero vector is excluded. $x = 0$ satisfies $A 0 = λ 0$ for every $λ$ , so it would be a "trivial eigenvector" for any number. Eigenvectors are required to be nonzero; otherwise the concept carries no information.
The eigenvalue is tied to the matrix, not to the vector alone. An eigenvector of $A$ is not an eigenvector of some other matrix $B$ in general. The pair $(A, λ, x)$ travels together.
Direction is preserved, not the vector. $(3, 3) = 3 (1, 1)$ points along $(1, 1)$ , but it is a different vector in magnitude. Students sometimes think "same vector" instead of "same direction, scaled".
Scalar multiples are the same eigenvector. If $(1, 1)$ is an eigenvector, then $(2, 2)$ , $(5, 5)$ , and any nonzero multiple are all eigenvectors for the same $λ$ . The direction is the identity; the length is arbitrary.

15.1.4 Finding Eigenvalues and Eigenvectors

Given a matrix $A$ , how do we actually find these special vectors and their scaling numbers? The recipe is: solve the characteristic equation

$∣ A - λ I ∣ = 0$

where $I$ is the identity matrix and $∣ \cdot ∣$ denotes the determinant. Expanding $∣ A - λ I ∣$ gives a polynomial in $λ$ (for a $2 \times 2$ matrix, a quadratic); the roots of that polynomial are the eigenvalues — the values of lambda. For each eigenvalue $λ$ , substitute it back into the homogeneous system

$(A - λ I) x = 0$

and solve for $x$ ; the nonzero solutions are the eigenvectors belonging to that eigenvalue. The eigenvalues come first, then the eigenvectors, one system per eigenvalue. In the worked example above, the matrix $[1221]$ yields the eigenvalue $λ = 3$ with eigenvector $(1, 1)$ , and the matrix $[73 3 - 1]$ yields $λ = 8$ with eigenvector $(3, 1)$ .

Formalize — the recipe, symbol by symbol.

$A$ is the given square matrix (here $2 \times 2$ ; the recipe extends to $n \times n$ ).
$λ$ (lambda) is the unknown scalar we are solving for — the eigenvalue.
$I$ is the identity matrix, $[1001]$ in two dimensions, which leaves every vector unchanged.
$∣ \cdot ∣$ denotes the determinant — a single number computed from the matrix entries.

Why does this work? We want nonzero solutions of $(A - λ I) x = 0$ . A homogeneous system has a nonzero solution exactly when the coefficient matrix $A - λ I$ is singular — when its determinant is zero. So the condition $∣ A - λ I ∣ = 0$ is precisely the condition that a nonzero eigenvector exists. For a $2 \times 2$ matrix the determinant expands to a quadratic in $λ$ , which has at most two roots — so at most two eigenvalues, matching the two examples above.

Worked example — finding both eigenvalues of $A = [1221]$ . Step 1: form $A - λ I$ :

$A - λ I = [1 - λ 2 2 1 - λ]$

Step 2: set the determinant to zero. For a $2 \times 2$ matrix $[a c b d]$ , the determinant is $a d - b c$ :

$∣ A - λ I ∣ = (1 - λ) (1 - λ) - (2) (2) = (1 - λ)^{2} - 4 = 0$

Step 3: solve:

$(1 - λ)^{2} = 4 \Rightarrow 1 - λ = \pm 2 \Rightarrow λ = - 1 or λ = 3$

So the eigenvalues are $λ_{1} = 3$ and $λ_{2} = - 1$ . Step 4: for $λ = 3$ , solve $(A - 3 I) x = 0$ :

$A - 3 I = [- 2 2 2 - 2], [- 2 2 2 - 2] [x_{1} x_{2}] = [00]$

Both rows give $- 2 x_{1} + 2 x_{2} = 0$ , so $x_{1} = x_{2}$ . A solution is $x = (1, 1)$ , matching the lecture's example. Step 5: for $λ = - 1$ , solve $(A + I) x = 0$ :

$A + I = [2222], 2 x_{1} + 2 x_{2} = 0 \Rightarrow x_{1} = - x_{2}$

A solution is $x = (1, - 1)$ . Check: $A (1, - 1) = (1 - 2, 2 - 1) = (- 1, 1) = - 1 (1, - 1)$ . Sense-check: the two eigenvalues $3$ and $- 1$ sum to the trace of $A$ ( $1 + 1 = 2 = 3 + (- 1)$ ) and multiply to the determinant ( $1 (1) - 2 (2) = - 3 = 3 \times (- 1)$ ), two standard consistency identities.

Worked example — finding both eigenvalues of $A = [73 3 - 1]$ . Step 1: form

$A - λ I = [7 - λ 3 3 - 1 - λ]$

Step 2: the determinant:

$∣ A - λ I ∣ = (7 - λ) (- 1 - λ) - (3) (3) = 0$

Step 3: expand and solve:

$(7 - λ) (- 1 - λ) = - 7 - 7 λ + λ + λ^{2} = λ^{2} - 6 λ - 7$

$λ^{2} - 6 λ - 7 - 9 = λ^{2} - 6 λ - 16 = 0 \Rightarrow (λ - 8) (λ + 2) = 0 \Rightarrow λ = 8 or λ = - 2$

Step 4: for $λ = 8$ , $(A - 8 I) x = 0$ gives

$[- 1 3 3 - 9] [x_{1} x_{2}] = [00] \Rightarrow - x_{1} + 3 x_{2} = 0 \Rightarrow x_{1} = 3 x_{2}$

So $x = (3, 1)$ works, matching the lecture. Step 5: for $λ = - 2$ , $(A + 2 I) x = 0$ :

$[9331] [x_{1} x_{2}] = [00] \Rightarrow 3 x_{1} + x_{2} = 0 \Rightarrow x_{2} = - 3 x_{1}$

So $x = (1, - 3)$ works. Sense-check: trace $7 + (- 1) = 6 = 8 + (- 2)$ and determinant $7 (- 1) - 9 = - 16 = 8 \times (- 2)$ ; the checks confirm both roots.

Assumptions & scope — when the recipe applies.

The characteristic equation assumes a square matrix; eigenvectors are not defined for rectangular matrices.
For a $2 \times 2$ matrix the polynomial is quadratic, so there are exactly two roots counting multiplicity — one of which is the maximum eigenvalue that drives the LDA projection below.
For symmetric matrices (and covariance and scatter matrices are symmetric), all eigenvalues are real. Eigenvalues can be negative in general — as $- 1$ and $- 2$ above show — but eigenvalues of covariance/scatter matrices are never negative, since variances cannot be negative.
If $∣ A - λ I ∣ = 0$ is satisfied at a value $λ$ but $(A - λ I) x = 0$ has only the zero solution, then the matrix is defective; for the symmetric matrices used in this course every eigenvalue yields a genuine eigenvector.

Recap + bridge. To find eigenvectors and eigenvalues: solve $∣ A - λ I ∣ = 0$ for the eigenvalues, then solve $(A - λ I) x = 0$ for each eigenvector. The lecture's matrices give $(λ = 3, x = (1, 1))$ and $(λ = 8, x = (3, 1))$ , and a full solution finds a second eigenvalue-and-eigenvector pair for each. This two-step recipe is the engine behind the projection directions of Fisher's linear discriminant analysis, to which the lecture now returns.

15.1.5 Eigenvectors in Principal Component Analysis and Fisher's LDA

This is where the concept earns its keep in the earlier topics. In the earlier discussion of Fisher's linear discriminant analysis (LDA), we had two different classes and we wanted to classify them. The plan was to identify a vector, draw it in the two-dimensional plane, and project all the points onto that line, so that the samples from the different classes are well separated. In other words: finding the projection to a line such that the samples from different classes are as separated as possible after projection.

To avoid the mismatch that plain projection along an arbitrary direction would produce, we computed the within-class scatter for the two classes — the scatter matrices $S_{1}$ and $S_{2}$ , each playing the role of a matrix $A$ — and then the total within-class scatter. From that scatter matrix we sought a vector that does not change its direction under the matrix: exactly the eigenvalue problem. The eigenvector corresponding to the maximum eigenvalue is the direction we want: if we find the eigenvector belonging to the largest eigenvalue and project all the sample observations onto that line, it gives the best possible separation between the classes.

From the earlier session, the eigenvector that came out of that LDA exercise was about $(0.22, 0.24)$ in base form, and any scalar multiple represents the same direction: multiply by 10 and you get $(- 2.2, 2.4)$ ; multiply by 100 and you get $(- 22, 24)$ . All of these point along one line: the x-component negative (towards the left), the y-component positive (upwards). This is the projected line onto which all data points should be dropped for the best separation. That is what the eigenvalue–eigenvector machinery was doing inside principal component analysis and Fisher's linear discriminant analysis: the eigenvector of the largest eigenvalue supplies the projection direction. Along it, the matrix only stretches the data instead of scrambling its direction.

Reconciling the sign of the eigenvector. The spoken values mix a positive base $(0.22, 0.24)$ with scaled versions $(- 2.2, 2.4)$ and $(- 22, 24)$ that have a negative x-component. The consistent reading: the base eigenvector is $(- 0.22, 0.24)$ — x-component about $- 0.22$ , y-component about $0.24$ — and its scalar multiples $(- 2.2, 2.4)$ and $(- 22, 24)$ all point along the same line. The sign itself is a free convention: if $x$ is an eigenvector, then $- x$ is the same eigenvector direction (a line has no intrinsic arrow). The direction the professor describes — x negative, y positive, that is, pointing left and up — is what the analysis uses, and any nonzero scalar multiple is equally valid as long as it points along that line.

In practice this is how the scatter matrix earns its role as $A$ : compute $S_{1}$ and $S_{2}$ from the two classes, add them into the total within-class scatter matrix, then solve the eigenvalue problem $∣ S - λ I ∣ = 0$ for that matrix. The eigenvector belonging to the largest eigenvalue is the best-separation direction, and the eigenvalue itself measures how much the scatter is concentrated along it. The reference text confirms the same logic from the software side: the eigenvalue of the discriminant function is the ratio of between-group to within-group variance, and "a higher eigenvalue is always desirable".

Recap + bridge — what to carry forward. An eigenvector is a direction the matrix only stretches; its eigenvalue is the stretch factor. LDA and principal component analysis both boil down to one eigenvalue problem: pick the eigenvector of the largest eigenvalue and project the data onto it. The same idea of "directions that summarize the data" returns in the next section, where respondents are grouped by how close their opinion vectors are.

15.1.6 Student Questions and Answers

Q: Do we have to draw vectors or geometric representations on the exam?

A: No. There is no point in drawing vectors and so on — you do not need to worry about it. The geometric picture is only for the sake of understanding the concept; drawing the representation is not required from the exam point of view. What matters on the exam is the concept itself: what an eigenvector is, what the eigenvalue equation states, and how the machinery feeds into principal component analysis and linear discriminant analysis.

Q: Do we have to do the software part of the analysis ourselves?

A: No. The software part is not expected from your side. The entire output will be given to you, and you need to address it as a recommendation — your role is only to understand the problem and interpret the results. No heavy calculation is required from you: a few exam questions you will need to work out, and a few you will simply look at the output, observe it, and answer. The same pattern appeared in the midterm: some questions asked you to compute, some asked you to read the given output and interpret it. So the exam tests interpretation of results and the ability to write a recommendation, not the running of software.

Q: In the previous session a point came up about the representation of a vector — is the unit-vector form the same as the column form?

A: Yes, they are two ways of writing the same vector. The unit-vector notation writes $r = 1 i + 3 j$ using the unit vectors $i$ and $j$ along the x- and y-axes; the column form writes the same vector as $[13]$ . Geometrically both describe the arrow from the origin to the point $(1, 3)$ , so the representation is one vector drawn in two notations, and that was the point being clarified. Use whichever form the problem is written in; convert freely, since the two carry identical information.

15.2 Cluster Analysis

15.2.1 What Cluster Analysis Is

Hook — one market, many customers. A car maker is about to launch a small car to a market of millions. Should the company send one identical advertisement to everyone? Almost certainly not. But how does it know which messages different people want to hear? Cluster analysis answers that: split the market into a few groups of people who think alike, label each group, and then target each group with its own message.

The second leftover topic is cluster analysis, the technique that divides a set of objects into similar groups. The idea: the entire set of individuals is heterogeneous, and cluster analysis splits it into homogeneous groups — similar groups of objects that we define as clusters. Making similar groups is only half the job. Once the groups are formed, you also need to give them meaningful labels, so that each cluster carries useful information about the people in it. In the reference text this is exactly how the technique is introduced: a classification technique whose emerging groups are homogeneous in their composition and heterogeneous compared with the other groups, also known as numerical taxonomy or Q analysis because the same grouping logic is used across psychology, sociology, engineering, and management.

The motivating example from the earlier discussion was market segmentation in the car market. The interest was in finding out the purchase intentions of individuals about a small four-wheeler — the Tata Nano — around its launch. By capturing people's attitudes and opinions, we club together individuals with similar interest in one group and individuals with a different set of similar interests in another group. If we can find those groups, then targeting their needs — tailoring the offering to their demands — becomes much easier from the managerial point of view. Real-world: this is exactly how marketing teams segment a customer base before a product launch, so that one campaign can speak to each segment's priorities instead of shouting at everyone the same way. Commercial segmentation systems such as ACORN (which classifies residential neighborhoods on about 40 variables) and PRIZM (which creates dozens of lifestyle categories from education, affluence, family life cycle, and similar variables) are built on exactly this logic.

Recap. Cluster analysis = split a heterogeneous set of objects into homogeneous groups, then label each group so the grouping carries meaning. The rest of this section is the machinery that turns "similar" into a computable number: distance, then merging, then choosing how many groups.

15.2.2 The Double Goal: Minimum Within, Maximum Between

Before any algorithm, the logic of good clustering must be stated: intra-cluster distances are to be minimized, and inter-cluster distances are to be maximized. Objects inside the same cluster should be as close as possible to each other, while objects in different clusters should be as far apart as possible. The hard part is that both must be handled together: at one point we minimize, at another we maximize. The technique we are about to use takes care of both.

The workhorse that measures closeness is a simple distance matrix. We compute the pairwise distances between every pair of individuals, and that matrix is what we use to decide similarity. Two individuals with a very small distance between them have similar opinions; a large distance means their opinions differ.

Formalize — what "minimize within, maximize between" means in practice.

Intra-cluster distance (within): the gap between two members of the same cluster. Good clustering keeps these small — members of one group have nearly identical opinion profiles.
Inter-cluster distance (between): the gap between members of different clusters. Good clustering keeps these large — different groups genuinely differ.
The distance matrix feeds both goals: its smallest entries reveal which pairs belong together (minimum within), and the levels at which clusters merge reveal how far apart the groups are (maximum between).

A geometric reading: draw the respondents as points in a space with one axis per attitude. Good clusters look like three tight, well-separated blobs — points packed inside each blob, and a visible gap between blobs. Bad clustering looks like one undifferentiated cloud: no internal packing and no gaps. The rest of the procedure is a systematic way to find the blobs.

15.2.3 Distance Matrix and Similar Pairs

To find out which pairs of respondents are similar — for example, among respondents A, B and C, is AB similar, or AC, or BC — we use the simple Euclidean distance formula between the two response vectors. For two respondents with response vectors $x = (x_{1}, x_{2}, \dots, x_{9})$ and $y = (y_{1}, y_{2}, \dots, y_{9})$ , the distance is

$d (x, y) = j = 1 \sum 9 (x_{j} - y_{j})^{2}$

This is the standard Euclidean distance: square the gap between the two respondents on each attribute, add the nine squares, then take the square root. The formula matches the reference treatment, where the distance between two objects $i$ and $j$ across $k$ variables is written as $d_{ij} = \sum_{k} (X_{ik} - X_{j k})^{2}$ — the same sum of squared deviations with a square root. The book adds one practical note: for actual clustering, software often works with the squared Euclidean distance (the same sum without the square root), which is faster to compute and preserves the ordering of similarities, since the square root is a monotone function.

Applied to all pairs of respondents, this gives the distance matrix — a table whose entries are the distances between every pair of individuals. The smallest entry identifies the most similar pair: the pair whose opinions are closest. That much we can read off the matrix directly; the merging machinery that follows uses exactly these distances, one minimum after another.

Worked example — Euclidean distance with two variables. Take the breakfast-attitude example from the reference text. Two variables matter: nutrition and ease of preparation, each rated on a 10-point scale. Person A chose (nutrition = 1, ease = 2) and person B chose (5, 2). Their distance:

$d (A, B) = (5 - 1)^{2} + (2 - 2)^{2} = 16 + 0 = 4.0$

Add a third person C with (6, 2). Then

$d (A, C) = (6 - 1)^{2} + (2 - 2)^{2} = 25 + 0 = 5.0$

$d (B, C) = (6 - 5)^{2} + (2 - 2)^{2} = 1 + 0 = 1.0$

So the smallest distance is $d (B, C) = 1.0$ : B and C are the most similar pair — the shorter the distance, the greater the similarity. Sense-check: B and C differ by 1 point on nutrition and agree exactly on ease of preparation, while A and B differ by 4 points on nutrition, so the ordering $B - C < A - B < A - C$ matches the raw numbers.

Assumptions & scope — when Euclidean distance misleads.

Scale matters. Euclidean distance is not scale invariant: a variable measured in larger units silently dominates the sum of squares. The reference text recommends standardizing the data when the variables are measured on very different scales, while keeping the questionnaire scales roughly comparable in the first place.
All variables count equally. The formula gives every attribute the same weight; if two respondents disagree wildly on one attribute they are far apart even if they agree on everything else.
Ordinal caveat. The nine attitudes here are Likert ratings (1–5), which are ordinal in spirit; the course follows the text in treating them as metric for the distance computation, as standard practice in marketing research.

15.2.4 Hierarchical Agglomeration: Schedule and Dendrogram

The procedure adopted for this problem is the hierarchical method: a stepwise process that starts with the most similar pair and progressively merges clusters, building the entire grouping in a tree-like structure. The same process can be viewed in two complementary outputs. One is the agglomeration schedule — a table that records, stage by stage, which similar pairs (or clusters) join together and the distance at which they join. The other is the dendrogram — the full pictorial representation of the merging history, a tree where each merge is a fork that brings two leaves or branches together. From the dendrogram we can see, at a glance, which respondents were close to each other and in what order they clubbed together.

Formalize — the stepwise logic.

Start with every respondent as its own cluster (25 clusters for 25 respondents).
Look up the smallest entry in the distance matrix — the most similar pair.
Merge that pair into one cluster; record the stage, the two members, and the merge distance.
Recompute the distances between the new cluster and every remaining cluster.
Repeat steps 2–4 until everything sits in one cluster. With $n$ objects there are $n - 1$ merging stages.

The agglomeration schedule is the numeric log of this loop — one row per stage. The dendrogram is the same log drawn as a tree: each respondent is a leaf at the left, each merge joins two branches into a fork, and the horizontal position of a fork shows the distance at which the merge happened.

A convention worth reading carefully appears in the schedule: a negative sign in front of a number indicates that we are merging a leaf. Specifically, "minus k indicates the kth leaf; otherwise the number refers to the cluster formed at the kth stage." So an entry like $- 3$ and $- 8$ means the two original leaves 3 and 8 are being joined, while an entry like 2 means "the cluster that was formed at stage 2." This sign convention is how the schedule tells a leaf from a previously formed cluster.

Worked example — reading the sign convention. Suppose the schedule shows a row with Cluster 1 = $- 3$ , Cluster 2 = $- 8$ . Both numbers carry a minus sign, so both are original leaves: respondent 3 joins respondent 8. In the next row, Cluster 1 = $- 21$ and Cluster 2 = 2: the minus says leaf 21 is involved, and the positive 2 says the other partner is the cluster formed back at stage 2 — not an original leaf, but a group that already contains several respondents. Sense-check: the number 2 alone would be ambiguous without the convention; the sign is what tells you whether the partner is a single respondent or an assembled group.

15.2.5 The Nano Purchase-Intentions Study: Data and Variables

The example used throughout comes from the book Research Methodology: Concepts and Cases by Deepak Chawla and Neena Sondhi, where the problem appears in Chapter 18 with the data on page 628. The study was conducted on four-wheeler owners about their purchase intentions for a small car, the Tata Nano, around the time of its launch. Management wanted to survey potential buyers and identify their purchase intentions, so they prepared a questionnaire and captured responses. The questionnaire has two parts — attitudes and demographic profiles — but the analysis interest here is only in the attitudinal variables, of which there are nine, labeled 1A through 1I:

1A — Indian technology of high order
1B — buy made in India
1C — value for money
1D — convenience over style
1E — don't do wasteful expenditure
1F — no compromise on safety
1G — save or not spend it
1H — try new things
1I — be a part of change in the world

These questions were posed to 25 respondents, and each rated every statement on a 5-point Likert scale. On that scale, 1 means strongly disagree, 2 means disagree, 3 means neutral, 4 means agree, and 5 means strongly agree. A respondent might answer, for example, 4, 5, 2, 3, 1, 5, and so on across the nine statements. The data matrix is 25 by 9: 25 rows (respondents) and 9 columns (attributes, 1A to 1I), i.e., 25 measurements on 9 features. As an illustration, the first respondent gave the responses 5, 5, 3, 2, 3, 3, 4, 1, 1 across the nine attitudinal variables. The task: club all 25 customers according to their interests. Which two customers give the same opinion on an attribute? How should the customers be grouped, and how many groups make sense?

Intuition — the 25-by-9 data grid. Imagine a spreadsheet with 25 rows of customers and 9 columns of attitude scores. Two customers are "similar" when their rows of nine numbers are nearly identical. Respondent 1's row $(5, 5, 3, 2, 3, 3, 4, 1, 1)$ is a single point in a 9-dimensional space; the distance formula measures how far any two such points sit apart. The clustering algorithm then finds which points clump together.

15.2.6 Worked Walkthrough: Reading the Merges

The analysis was run on the Nano data sheet, computing the distance matrix first and then the hierarchical clustering. In the distance matrix, look at respondents 3 and 8: their distance is 0. That indicates they are in exactly the same opinion profile — in mathematical terms their distance is minimum, namely zero. So at height 0 these two are clubbed together. In the same way, respondents 6 and 11 clubbed together, and 7 and 13 clubbed together — each pair is a minimum-distance merge.

Reading the agglomeration schedule stage by stage:

Stage 1: merge leaves 3 and 8 at height 0. The remaining 23 respondents stay as they are, so there are 24 clusters.
Stage 2: merge leaves 6 and 11 at height 0 — with 3 and 8 already merged, the count falls to 23 clusters.
Stage 3: the entry is $- 21$ and 2: leaf 21 merges with the cluster formed at stage 2, which is the ${6, 11}$ cluster. So 6, 11 and 21 end up together, and the count falls to 22 clusters.

Resolving the tangled stage readings. The spoken description of which pair merges at stage 2 is tangled ("three, eight and six, eleven is merged together"). The reading above — 3 and 8 at stage 1, 6 and 11 at stage 2, leaf 21 joining the stage-2 cluster at stage 3 — is the one consistent with the stated cluster counts 24 → 23 → 22 and with the final memberships, in which 3 and 8 sit together and 6, 11 and 21 sit together. The reference text's own schedule for the Nano data lists the same height-zero pairs — 18 and 25, 11 and 21, 7 and 13, 6 and 11, 3 and 8 — merged across the first stages at coefficient 0.000, with the ordering between software runs differing slightly but the resulting memberships identical. What matters for the exam is the reading rule, not the exact row order: minus k means leaf k, a positive k means the cluster formed at stage k, and every merge joins the closest existing pair at the height recorded.

The same logic continues: at every stage, the closest pair of existing clusters — measured by the height or distance of the merge — joins together. The process ends at the last stage with a single cluster containing all 25 respondents. The point of interest is that neither extreme is meaningful: too many clusters makes the grouping meaningless, and too few clusters is also not so good — the right number sits in between, and deciding it is a step of its own.

Intuition — the two extremes. With 25 clusters (before any merge), every customer is a "group" of one: the grouping carries no information at all. With 1 cluster (after the final merge), everyone is one undifferentiated market: the grouping carries no information either. The useful solutions sit in between, where each group is big enough to matter and distinct enough to describe. The next subsection is the formal way to find that middle ground.

15.2.7 Worked Walkthrough: Deciding the Number of Clusters

Two approaches were offered for choosing the number of clusters. The first is simply inspecting the dendrogram and letting intuition pick the level that splits the respondents into a sensible number of groups. The second is the difference method, a small numerical justification: take the agglomeration coefficients (the heights at which merges happen) and compute successive differences.

Start from the last coefficient, when all objects are grouped into a single cluster (stage 24).
Subtract the coefficient for two clusters (stage 23). The difference came out as $34.70 - 21.27 = 13.43$ .
Next, subtract the coefficient for three clusters from the two-cluster coefficient. The difference here showed a big jump: about 15.3.
Going on down, the remaining successive differences are small: 0.58, 1.19, 0.8, and so on.

Resolving the transcribed coefficients. The spoken numbers "34 minus 7 to 21 minus 27" are read as the coefficients 34.70 and 21.27, giving the stated difference 13.43, and the "15.3" jump between the two- and three-cluster coefficients is kept as spoken. These come from the software output shown in the session. The reference text runs the same difference method on its own schedule for the same data with coefficients 59.222 (one cluster) and 40.667 (two clusters), giving $59.222 - 40.667 = 18.55$ , then $40.667 - 11.800 = 28.867$ between two and three clusters, then $11.800 - 8.50 = 3.5$ between three and four. The numbers differ between runs because the coefficient scale depends on the linkage method and software, but the conclusion is identical in both: the largest variation happens when moving from two clusters to three, so a three-cluster solution is chosen.

The insight: most differences between consecutive coefficients are small, but there is a sudden, large change at one particular point. The maximum variation happens when we move from two clusters to three clusters. That large jump is the signal. The instructor is careful not to guarantee the answer: this number-of-clusters decision is debatable, and you could try to convince yourself that two clusters are enough by looking at the problem. But with this numerical justification, we can say that a three-cluster solution is adequate for further analysis of the respondents. Three clusters suffice to divide the entire set of 25 respondents according to their purchase intentions. The same conclusion can be reached just by looking at the dendrogram — the column differences are one way, the eye is another; it is up to us which to use. So the two things to know are how the clusters merge (from the distance matrix via the similar pairs) and how the number of clusters is decided (by dendrogram inspection or by the height differences).

Pitfalls — choosing the number of clusters.

Reading the jump at the wrong end. The difference method looks for the largest successive difference; here that is the two-to-three step (about 15.3 in the lecture output, 28.867 in the reference run). Students sometimes take the first large difference from the top and stop — but the convention is to inspect where the biggest change occurs, then choose the smaller of the two cluster counts on either side of it.
Forgetting that the choice is judgment-based. The instructor explicitly says the answer is debatable: the difference method is a numerical justification, not a proof. A two-cluster solution can be defended by eye; the method says three is adequate, not unique.
Treating the dendrogram and the schedule as different answers. Both outputs describe the same merges; the dendrogram is the picture of the schedule. If the eye and the differences disagree, re-check which coefficient belongs to which stage.

15.2.8 Worked Walkthrough: Centroids and Cluster Labels

With three clusters agreed on, the 25 respondents split as:

Cluster 1 (C1): 1, 4, 9, 23, 24, 12, 14, 20, 16, 19 — 10 respondents.
Cluster 2 (C2): 15, 17, 22, 2, 5, 10 — 6 respondents.
Cluster 3 (C3): 3, 8, 6, 11, 7, 13, 21, 18, 25 — 9 respondents.

These memberships match the three-cluster column of the reference text's membership table for the Nano survey. To interpret the clusters, compute the cluster centroid — the average of the respondents' opinions within each cluster, attribute by attribute. For attribute 1A (Indian technology of high order), take the members of C1 and look up their opinions in the data set: 5, 5, 4, 5, 5, 4, 4, 4, 4, 4. Adding these gives 44, and dividing by the 10 members gives

$\frac{44}{10} = 4.4$

For C2, the six members' opinions on 1A are 3, 2, 1, 3, 2, 2, which add to 13:

$\frac{13}{6} = 2.17$

For C3, the opinions on 1A add to 18:

$\frac{18}{9} = 2.0$

Resolving the garbled spoken count. The spoken phrase "turned out to be 18 for 19 respondents" is garbled; C3 has 9 members by the membership list above (respondents 3, 8, 6, 11, 7, 13, 21, 18, 25), so the centroid is $18/9 = 2.0$ . The reference text's centroid table for the Nano survey confirms all three values on statement 1A: 4.40 for the patriotic cluster (10 members), 2.17 for the cautious cluster (6 members), and 2.00 for the innovative cluster (9 members) — exactly 4.4, 2.17 and 2.0.

The same exercise repeats for 1B, 1C, 1D, 1E, 1F and so on, producing a centroid profile for each cluster across all nine attributes.

The labels then come from the highest values — the attributes scoring above 4 — for each cluster. Cluster 1 is high on 1A (Indian technology of high order) and 1B (buy made in India), which points to the patriotic Indian: customers whose purchase intention is driven by Indian technology and made-in-India products. Cluster 2 is high on value for money, convenience over style, no wasteful expenditure, no compromise on safety, and save rather than spend — the cautious customer, who weighs every purchase carefully instead of buying on patriotic sentiment. Cluster 3 is high on try new things and be a part of change in the world — the innovative customer, who wants novelty and to be part of something new. The titles given here are the ones from the book (copied from the source), but any other decent title that captures the profile is acceptable. The underlying truth is simple: all 25 will not behave in a similar fashion. Some individuals care about one set of things, others about a different perspective on their intentions, and the centroid profiles make those differences visible.

Worked example — the full centroid profile of one cluster. For C1, the ten respondents' average opinions define the profile: 4.4 on 1A (Indian technology of high order) and similarly high on 1B (buy made in India). The reference table for the Nano survey shows the patriotic cluster at 4.40 on 1A and 4.70 on 1B, versus the cautious cluster at 2.17 and 1.67 and the innovative cluster at 2.00 and 2.22. So the label comes from the attribute on which this cluster scores highest — here the two made-in-India statements clear the 4-mark. Sense-check: a centroid of 4.4 means the typical member of C1 "agrees" with statement 1A; a centroid of 2.0–2.2 means the typical members of C2 and C3 "disagree" — the three clusters genuinely differ on the patriotic dimension, which is why the label is trustworthy.

Pitfalls — labeling the clusters.

Label from the high-scoring attributes, not the low ones. The book labels clusters by the statements scoring above 4. A cluster is not defined by what it rejects.
Do not invent a label that the profile does not support. The book's titles (patriotic, cautious, innovative) are examples — but any replacement title must be backed by the centroid values.
Cluster numbering is not fixed across software. The same three groups may be printed as different cluster numbers by different packages; always identify a cluster by its membership or its profile, not by its printed number. The reference text itself labels the same three groups with different numbers in its hierarchical and K-means runs.

Real-world: after labeling, a company knows which cluster to target with which message — customers identified as cautious can be addressed with value-and-safety-oriented models, and so on, which is how cluster analysis feeds directly into market targeting. The same pattern plays out in customer relationship management: segments identified by clustering determine which offer, which channel, and which timing each customer group receives, rather than one blanket campaign for everyone.

Recap + bridge. Cluster analysis: measure similarity with Euclidean distances, merge the closest pairs hierarchically into an agglomeration schedule and dendrogram, choose the number of clusters where the coefficient jump is largest (three here), then label each cluster from its centroid profile. The clusters of the Nano study became the patriotic, cautious, and innovative customers. With that, the two leftover topics from the earlier sessions are complete — the session now turns to a new family of tools, the non-parametric procedures.

15.2.9 Student Questions and Answers

Q: How do we determine that, for example, k = 4 — that the clusters are far from each other, and what about the inertia within clusters?

A: The distance (similarity) from the distance matrix is what takes care of how close the points are within each cluster, and the height (the difference) takes care of how far the clusters are from each other. Think in terms of the agglomeration schedule: the similarity within clusters comes from the distance matrix. The separation between clusters comes from the height — the level at which clusters merge. If anything further comes up on this point, it will be addressed later. In short: within-cluster closeness is a distance-matrix story, between-cluster separation is a height story, and the number of clusters is chosen where the height differences jump the most.

Q: Will the data sets for the assignment case studies be shared for practice?

A: Yes. The data sets used for the assignments have been extracted and shared — including a zip file with around 30 data sets covering many chapters. Data sets for the remaining chapters are planned to be shared by Monday or Tuesday. If you bought the textbook, you may already have access to its data sets, and those are useful for practicing the case studies. The Nano data set itself is in Chapter 18 of the Deepak Chawla book, at page 628.

15.3 Non-Parametric Procedures: The Sign Test

15.3.1 Parametric versus Non-Parametric Procedures

Hook — what do you do when nothing is normal? Every test seen so far — the Z test, the T test — quietly assumes the data came from a normal distribution. But what if the data are not normal, or you only have "above or below" information instead of exact numbers? The tools in this section were built for exactly those situations.

Everything discussed so far — the Z distribution, the T distribution — falls under parametric procedures, where parameters are involved: the mean $μ$ , the variance $σ^{2}$ , or proportions $p$ . These parameters are tested through Z tests and T tests. Underneath every parametric procedure sits an assumption about the population distribution, the shape of the distribution, which we know in advance: essentially that the data come from a normal distribution.

Non-parametric procedures are the counterpart for the situations where those assumptions do not hold. There is no involvement of $μ$ , $σ^{2}$ or $p$ . If we do not know anything about the population distribution or the population parameters, the methods are distribution-free: we do not know the parent distribution and we do not need to assume one. In the same way that all parametric methods carry the basic assumption of a normal distribution, non-parametric methods are distinguished by having no such assumption about the shape of the data. They work better than parametric procedures when the data are not distributed normally, or when the measurements are ordinal — things assessed in some qualitative way. And when a problem can legitimately be solved both ways, priority goes to the parametric procedure; non-parametric is the alternative we reach for when the original parametric route is not available.

Formalize — the contrast, dimension by dimension.

Dimension	Parametric procedures	Non-parametric procedures
Target of inference	Parameters: mean $μ$ , variance $σ^{2}$ , proportion $p$	No parameters; often the median
Distribution assumption	Normal population required	None — distribution-free
Data level	Metric (interval/ratio) data	Works with ordinal data too
Typical tools	Z test, T test, F test	Sign test, run test, and similar
When to prefer	When the normality assumption holds	When normality fails or data are ordinal

The one-line rule: use the parametric procedure whenever it legitimately applies, and switch to the non-parametric counterpart when the parametric route is not available — this mirrors the reference treatment, which lists the normality assumption as the defining difference between the two families.

Of the many non-parametric approaches that exist, the discussion focuses only on two: the sign test and the run test. The sign test is the natural non-parametric counterpart of the one-sample Z test for a median. The run test, previewed below, checks whether a sample was drawn by chance. No other non-parametric methods are required at this stage.

Pitfalls — the two directions of misuse.

Using non-parametric methods when the parametric ones apply. If the data are normal and metric, the Z/T tests are more powerful; the professor's rule is explicit: priority goes to the parametric procedure.
Treating "no assumption" as "no information". Non-parametric tests are not assumption-free in the sense of testing nothing; they replace the normality assumption with a weaker one — here, that the median splits the population at 0.5. The trade-off is less power when normality truly holds.

15.3.2 Why the Median: The Non-Parametric Target

When we talk about parametric procedures, the location parameter we discuss is the mean. For non-parametric procedures, the location we discuss is the median — the reason being that the median is a measure that involves no parameter assumption. The median divides the entire distribution into two halves: it is a positional average, sitting in the middle of the data as the second quartile $Q_{2}$ , the 50th percentile. Because the median splits the data at 0.5, testing a claim about the median is the same as testing a claim about the probability $P$ :

$H_{0} : median = m_{0} ⟺ H_{0} : P = 0.5$

against $H_{A} : median \neq = m_{0}$ , equivalently $H_{A} : P \neq = 0.5$ . Either way of writing the hypothesis is acceptable.

The sign test itself is a versatile method for hypothesis testing, and the name gives away its machinery: the entire problem is reduced to positive and negative signs. There is no numerical quantification beyond that: the data values become plus and minus signs. The test statistic is a count of those signs.

Intuition + analogy — flipping a fair coin. If the true median is $m_{0}$ , then half the population sits above it and half below. So each observation is like a coin flip: plus (above) with probability 0.5, minus (below) with probability 0.5. A claim like "the median is 450" becomes the claim "the coin is fair" — $P = 0.5$ . If you see far more pluses than a fair coin would produce, the median claim is in doubt. Where the analogy breaks: a coin flip is exactly 50/50 by design, while the sign test hypothesizes the 50/50 split and then checks the data against it.

15.3.3 The Sign Test Procedure

The procedure is simple:

State the null and alternative hypotheses about the median (or equivalently about $P = 0.5$ ).
Compare every observation with the hypothesized median $m_{0}$ under $H_{0}$ .
Use a plus sign (+) if the observation is above the hypothesized value; use a minus sign (−) if it is below.
Discard any observation exactly equal to the hypothesized value — such a value can get neither sign.

For example, with a hypothesized median of 450, an observation of 562 gets a plus sign, 415 gets a minus sign, and 860 gets a plus sign. If one of the sample values equals 450 exactly, it is discarded and the effective sample size drops: with 10 observations and one tie, $n$ becomes 9. This discarding operation is part of the procedure — plus if more, minus if less, discard if matched.

After the sign assignment, the numerical values are gone; the data are now a string of pluses and minuses. The test statistic for the sign test is the number of plus signs, $X$ .

Formalize — the sign assignment, step by step. Let $m_{0}$ be the hypothesized median. For each observation $x_{i}$ :

$sign_{i} = ⎩ ⎨ ⎧ +, -, discarded, x_{i} > m_{0} x_{i} < m_{0} x_{i} = m_{0}$

The effective sample size is $n =$ (original sample size) − (number of ties), and the test statistic is $X =$ number of plus signs among the $n$ retained observations. Note the reference texts phrase the rule identically: observations greater than the hypothesized value become plus signs, observations less become minus signs, and observations exactly equal are omitted, reducing the sample size accordingly.

15.3.4 The Binomial Connection and Sample-Size Convention

Why does the sign test work out mathematically? After conversion, each of the $n$ observations is classified into exactly two possibilities — plus or minus. A quantity that can take only two outcomes (positive or negative, yes or no) is governed by the binomial distribution. The two outcomes are possible, the trials are independent, and under $H_{0}$ the probability of a plus is $p = 0.5$ . So the number of plus signs follows

$X \sim Binomial (n, 0.5)$

under the null hypothesis. These are exactly the assumptions of the binomial distribution — two outcomes, independent trials, constant success probability — and they are what allow us to compute probabilities for the test.

The sample-size convention matters next. For parametric procedures, $n > 30$ is a large sample and $n < 30$ a small sample. For non-parametric procedures the convention is: $n \leq 20$ is the small-sample case and $n > 20$ is the large-sample case. The instructor presents this as a blind rule — no theoretical discussion, just a convention to follow. For a small sample, we go back to the binomial distribution itself and read the binomial tables. For a large sample, the binomial is approximated by the normal distribution, and the normal tables suffice. That is the case the course focuses on, since the normal tables are already heavily used in the hypothesis-testing parts of the syllabus.

Formalize — why binomial, and when to approximate. A binomial experiment needs exactly three things: two possible outcomes per trial, independent trials, and a constant success probability. The sign test supplies all three: plus/minus outcomes, independent observations, and $p = 0.5$ under the null. Small samples use the binomial tables directly because the exact probabilities are computable; large samples use the normal approximation $X \approx N (n p, n pq)$ , since the central limit theorem makes the binomial mound-shaped when $n$ is large. The convention boundary for this course: $n \leq 20$ small, $n > 20$ large.

Assumptions & scope — where the sign test works and where it breaks.

Independence of observations is required: one respondent's sign must not change the next respondent's sign. Violate this (e.g., panel data measured over time on the same stores) and the binomial model no longer holds.
Ties must be discarded, not assigned. An observation exactly at $m_{0}$ gets no sign; assigning it either sign would bias the count.
The binomial null is symmetric by design ( $p = 0.5$ ). The test checks the median claim; it is not a test about the mean.
Ordinal data are fine — that is a strength of the method. The magnitudes are deliberately thrown away; the cost is that a big plus and a tiny plus count identically.

15.3.5 Worked Example: Potato Chip Sales (Small Sample)

To illustrate the sign test, consider the weekly sales of potato chips by a grocery store chain. The management decided to carry the new potato chips product based on the manufacturer's estimate that the median weekly sale would be 450 dollars. (Notice: for this non-parametric setting the claim is about the median, not the mean.) After carrying the product for three months, management requested a hypothesis test about the population median weekly sales. The hypotheses are $H_{0}$ : median $= 450$ dollars against $H_{A}$ : median $\neq = 450$ dollars. Equivalently, $H_{0} : P = 0.5$ against $H_{A} : P \neq = 0.5$ .

A sample of 10 store managers was taken, and each observation was compared with 450: observations above get a plus sign, observations below get a minus sign — for instance, 562 → +, 415 → −, 860 → + — and any value exactly equal to 450 would be discarded. In this sample no entry matched 450, so nothing is discarded. The resulting signs: 7 pluses and 3 minuses. The test statistic is the number of plus signs,

$X = 7$

with $n = 10$ . Since $n \leq 20$ , this is the small-sample case and we use the binomial distribution. Assuming $H_{0}$ is true, the sampling distribution of the test statistic is binomial with $n = 10$ , $p = 0.5$ , and the binomial probabilities for $x = 0, 1, \dots, 10$ are computed. For a two-sided test, the p-value doubles the tail beyond the observed value:

$p -value = 2 \times P (X \geq 7) = 2 \times [P (X = 7) + P (X = 8) + P (X = 9) + P (X = 10)]$

The tail probability $P (X \geq 7)$ came out as 0.1719, so

$p -value = 2 \times 0.1719 = 0.3438$

Worked example — the binomial tail, computed in full. For $n = 10$ and $p = 0.5$ , each binomial probability is

$P (X = x) = (x 10) (0.5)^{x} (0.5)^{10 - x} = \frac{( x 10 )}{1024}$

because $(0.5)^{10} = 1/1024$ . The tail $P (X \geq 7)$ sums the four terms:

$P (X = 7) = \frac{( 7 10 )}{1024} = \frac{120}{1024} = 0.1172$

$P (X = 8) = \frac{45}{1024} = 0.0439, P (X = 9) = \frac{10}{1024} = 0.0098, P (X = 10) = \frac{1}{1024} = 0.0010$

$P (X \geq 7) = 0.1172 + 0.0439 + 0.0098 + 0.0010 = 0.1719$

An equivalent way to get the same tail: $P (X \geq 7) = 1 - P (X \leq 6)$ , where $P (X \leq 6) = 1 - 0.1719 = 0.8281$ . (The spoken account garbles this complement slightly; the standard complement of $P (X \geq 7)$ for a binomial with maximum value 10 is $1 - P (X \leq 6)$ .) Doubling for the two-sided test:

$p -value = 2 \times 0.1719 = 0.3438$

Sense-check: under a fair coin, seeing 7 or more heads in 10 flips happens with probability 0.1719 — about one time in six — so 7 pluses are not a surprising outcome, and the p-value confirms it.

Take $α = 0.05$ (the default level when nothing is given). Since the p-value 0.3438 is greater than $α$ — and indeed greater than 0.1 and greater than 0.01 — we cannot reject the null hypothesis; we accept it. There is not enough evidence in the sample to reject the assumption that the median weekly sales is 450 dollars. Real-world: the store chain keeps the decision to carry the product supported by the manufacturer's claim — the three months of sales data do not contradict the 450-dollar median. The identical example appears in the standard reference text with the same signs (7 plus, 3 minus), the same binomial tail 0.1719, and the same p-value 0.3438, so the numbers can be checked against the published solution.

Exam note. Small-sample sign tests ( $n \leq 20$ ) are the exception in this course — they need the binomial tables. The test statistic is always the number of plus signs $X$ ; the two-sided p-value is $2 \times P (X \geq x)$ . Remember the decision logic: p-value greater than $α$ means not enough evidence to reject the null.

15.3.6 Worked Example: Female Members' Median Age (Large Sample)

The second example moves to the large-sample case. A hypothesis test was conducted about the median age of the female members of a fitness center — the Trim Fitness Center — with the claim that the median age is 34 years. Instead of the raw sample, the conclusive information was given: in a sample of 40 members, 25 are older than 34, 14 are younger than 34, and 1 is exactly 34. Applying the sign procedure: values above the median become plus signs, values below become minus signs, and the value equal to 34 is discarded. So the effective sample size becomes

$n = 40 - 1 = 39$

with $X = 25$ plus signs (the members older than the claimed median age). The hypotheses are $H_{0}$ : median $= 34$ against $H_{A}$ : median $\neq = 34$ , equivalently $H_{0} : P = 0.5$ against $H_{A} : P \neq = 0.5$ . Since $n = 39 > 20$ , this is the large-sample case: we skip the binomial distribution and go straight to the standard normal (Z) tables.

The general conversion formula for the large-sample sign test converts the count of plus signs into a Z score:

$z = \frac{x - μ}{σ}, μ = n p, σ = n pq$

where $n$ is the effective sample size, $p = 0.5$ is the probability under $H_{0}$ , and $q = 1 - p$ . With $n = 39$ and $p = 0.5$ :

$μ = n p = 39 (0.5) = 19.5$

$σ = n pq = 39 (0.5) (0.5) = 9.75 = 3.1225$

The test statistic is then

$z = \frac{x - μ}{σ} = \frac{25 - 19.5}{3.1225} = \frac{5.5}{3.1225} = 1.76$

Because the alternative is two-sided, the p-value doubles the upper tail:

$p -value = 2 \times P (Z > 1.76) \approx 0.0784$

Worked example — the Z calculation, step by step. Step 1: effective sample size after discarding the tie: $n = 40 - 1 = 39$ . Step 2: the binomial mean under $H_{0}$ :

$μ = n p = 39 (0.5) = 19.5$

Step 3: the binomial standard deviation:

$σ = n pq = 39 (0.5) (0.5) = 9.75 = 3.1225$

Step 4: the Z score:

$z = \frac{x - μ}{σ} = \frac{25 - 19.5}{3.1225} = \frac{5.5}{3.1225} = 1.76$

Step 5: the p-value. From the normal tables, $P (Z < 1.76) = 0.9608$ , so the upper tail is $P (Z > 1.76) = 1 - 0.9608 = 0.0392$ . Doubling for the two-sided test:

$p -value = 2 \times 0.0392 = 0.0784$

(The spoken value "0.784" is a decimal slip; the standard value for $2 \times P (Z > 1.76)$ is 0.0784.) Sense-check: a Z of 1.76 is under the common cutoff of 1.96, so the p-value should be above 0.05 — and 0.0784 is indeed greater than 0.05, consistent with the textbook decision of not rejecting.

At $α = 0.05$ , the p-value is again greater than $α$ , so we do not reject the null hypothesis. There is not enough evidence in the sample to conclude that the median age of the female members is not 34 years. The decision rule throughout is the usual p-value approach: reject $H_{0}$ when the p-value is less than $α$ ; here the p-value is larger, so the null stands.

A note on the continuity correction. Some treatments of the large-sample sign test apply a continuity correction of 0.5 before computing Z (subtracting 0.5 from $X$ when $X > n p$ , adding when $X < n p$ ), because a discrete binomial is being approximated by a continuous normal curve. The reference chapter mentions this refinement, but the lecture does not use it: the professor computes $z = (x - μ) / σ$ directly from $X = 25$ , giving 1.76. Use the professor's form for the exam; the correction would change the Z score only slightly.

15.3.7 The Run Test: Preview

One more small technique remains for the non-parametric part of the course, to be covered in the final session: the test for randomness. In statistics we routinely say "consider a random sample" — but how do we actually validate whether a given sample is random or not? The procedure for that is the run test, and it will be discussed in the large-sample context to close out the non-parametric procedures. After that, the final session wraps up with a full review of everything from the beginning. It covers the syllabus as it stands and the paper pattern for the comprehensive examination.

Intuition — what "random" means in a sequence. In the run test, a run is a sequence of like elements preceded and followed by different elements (or nothing at all). Too few runs means the sequence clumps — signs bunch together — and too many runs means it alternates suspiciously; either pattern signals a lack of randomness. The final session will develop this formally for the large-sample case using the normal distribution.

15.3.8 Exam Notes for Non-Parametric Procedures

Exam note — non-parametric procedures.

The course focuses on the large-sample case for both the sign test and the run test. The knowledge of the normal distribution and the normal tables is enough for you to address those problems; you should be fluent in reading the normal tables, since the hypothesis-testing part of the syllabus relies on them.
The small-sample case is the exception: there you go back to the binomial distribution and read the binomial tables. The large-sample emphasis is deliberate — expect the same orientation in the examples.
Remember the sign test decision logic: p-value greater than $α$ means not enough evidence to reject the null; the p-value for a two-sided sign test is twice the tail probability $P (X \geq x)$ .
The hypotheses may be written either in terms of the median or in terms of $P = 0.5$ ; the two formulations are equivalent, and either is acceptable.
For the large-sample version, the quantities to compute are $μ = n p$ and $σ = n pq$ , then $z = (x - μ) / σ$ , then the tail probability from the normal tables.

Recap + bridge. The sign test replaces numeric data with plus and minus signs, counts the pluses, and checks the count against a binomial with $p = 0.5$ (small samples) or a normal approximation (large samples). In both worked examples the median claim survived: potato chip sales at 450 dollars and the fitness center's median age of 34 were not rejected. The run test previewed here — testing whether a sample is random — closes the non-parametric family in the final session, which is a full review from the beginning of the course.

Exam Guidance Summary

Exam note — the comprehensive examination.

Only what was actually discussed and delivered in the sessions can be expected in the comprehensive examination. The printed handout lists far more topics than the sessions could cover. The material presented in the course is the exam scope, so the expectation is to prepare from what was taught rather than everything in the handout.
The final session is a review, not a new delivery: it goes through everything from the beginning. It then gives clear information about the syllabus as it stands and about the paper pattern — how the comprehensive examination is structured and what to prepare for it. Plan for roughly 30 to 60 minutes.
No drawing of vectors or geometric representations is required on the exam. The geometric picture of eigenvalues and eigenvectors exists for conceptual understanding only; the representation itself is not an exam skill.
No software calculations are expected. The output will be given to you; your role is to understand the problem and interpret it, addressing the question as a recommendation. Expect a mixed paper: a few questions require working out the numbers, and a few require reading a given output, observing it, and answering from it. This is the same pattern as the midterm.
For the non-parametric part, the sign test and the run test are the two techniques; the large-sample case is the focus, with the normal tables as the tool. The small-sample case means binomial tables.
Cluster analysis exam skills: how clusters merge (similar pairs from the distance matrix), how the dendrogram and the agglomeration schedule are read (including the negative-sign convention), and how the number of clusters is decided (dendrogram inspection or height differences). Clusters are also labeled from their centroid profiles.
Practice material: a zip file with around 30 data sets was shared for the assignment case studies, and data sets for the remaining chapters are expected by Monday or Tuesday. If you have purchased the textbook, its data sets are also available for practice. The cluster analysis example sits in Chapter 18 of the Deepak Chawla book, with the data at page 628.

Key Industry Applications

Market segmentation for a car launch: the Nano purchase-intentions study clusters prospective four-wheeler buyers by their attitudes so that marketing can target each segment's needs — the core use of cluster analysis in marketing research. The same segmentation logic powers industry systems like ACORN and PRIZM, which classify neighborhoods and lifestyles for targeted campaigns.
Targeted marketing by customer profile: the three labeled clusters (patriotic Indian, cautious customer, innovative customer) show how a company can address each segment with the message it cares about. Patriotic appeal for one segment, value-and-safety emphasis for another, novelty for the third. After clustering, customer relationship management systems can route the right offer, channel, and timing to each group.
Retail assortment decisions: the potato chip example is a grocery store chain using a median hypothesis test on weekly sales to check a manufacturer's claim before continuing to carry a new product. Median-based tests are the natural tool here because weekly sales are typically skewed rather than normal.
Service industry demographics: the fitness center example tests claims about the median age of female members, a typical customer-profile question in the service sector — median age claims are checked non-parametrically when membership data are ordinal or skewed.
Dimensionality reduction and classification: the eigenvalue–eigenvector machinery reviewed here is the engine behind principal component analysis (used at scale for data compression and visualization) and Fisher's linear discriminant analysis (used for supervised classification). Both are standard tools in industry data science pipelines.
Textbook case study: the cluster analysis example is drawn from Research Methodology: Concepts and Cases by Deepak Chawla and Neena Sondhi (Chapter 18, page 628), a widely used research-methods text. Its case data sets double as practice material.

ASM Lecture 15 notes · Eigenvectors, Cluster Analysis, and Non-Parametric Procedures

Advanced Statistical Methods· postgraduate· 2026-08-11

Sections Breakdown

1Eigenvalues and Eigenvectors

Vector representation, matrix-vector multiplication, the eigenvalue equation, the characteristic equation, and how eigenvectors drive principal component analysis and Fisher's linear discriminant analysis.

2Cluster Analysis

Euclidean distance matrices, hierarchical agglomeration, the Nano purchase-intentions study, deciding the number of clusters, and labeling clusters from centroid profiles.

3Non-Parametric Procedures: The Sign Test

Parametric versus non-parametric procedures, the median as the target, sign assignment, the binomial model, and worked small- and large-sample sign tests.

4Exam Guidance Summary

The scope of the comprehensive examination, what is not examined, and the exam skills for cluster analysis and non-parametric procedures.

5Key Industry Applications

Real-world uses of cluster analysis, median-based sign tests, and the eigenvalue-eigenvector machinery in PCA and LDA.

Postgraduate students in Advanced Statistical Methods

Exam Revision Notes

Below is the distilled, exam-ready core. Every entry comes from the full explanation above. Use this section for rapid review; return to the main notes when a point needs more context.

Eigenvalues and Eigenvectors

Must-know: An eigenvector is a nonzero vector whose direction a matrix leaves unchanged: Ax = lambda x, with lambda the eigenvalue (stretch factor). To find them: solve |A - lambda I| = 0 for the eigenvalues, then (A - lambda I)x = 0 for each eigenvector. The largest-eigenvalue eigenvector gives the best-separation projection direction in LDA.

$∣ A - λ I ∣ = 0, (A - λ I) x = 0$

⚠️ Top pitfall: Counting the zero vector as an eigenvector (it is excluded by definition), or forgetting that any nonzero scalar multiple of an eigenvector is the same eigenvector direction. Also, eigenvalues can be negative in general, though not for covariance/scatter matrices.

Self-check: For A = [[1, 2], [2, 1]], verify that lambda = 3 with eigenvector (1, 1) satisfies Ax = lambda x.

Connects to: Cluster Analysis.

Cluster Analysis

Must-know: Cluster analysis: Euclidean distance d(x,y) = sqrt(sum of squared differences) builds the distance matrix; the closest pairs merge stage by stage (minus k = leaf k, positive k = cluster formed at stage k); the number of clusters is chosen where the largest jump in coefficients occurs (three for the Nano data); clusters are labeled from centroid profiles (patriotic, cautious, innovative).

$d (x, y) = j = 1 \sum 9 (x_{j} - y_{j})^{2}$

⚠️ Top pitfall: Labeling clusters from low-scoring attributes instead of the attributes scoring above 4; treating the number-of-clusters choice as unique when it is a judgment based on the largest height difference; confusing leaf (minus k) with cluster-formed-at-stage (positive k) in the agglomeration schedule.

Self-check: If C3 has 9 members whose 1A opinions sum to 18, what is the cluster centroid on 1A? (Answer: 18/9 = 2.0.)

Connects to: Eigenvalues and Eigenvectors.

Non-Parametric Procedures: The Sign Test

Must-know: Sign test: convert observations to + (above m0), - (below m0), discard ties; test statistic X = number of plus signs. Small sample (n <= 20): binomial tables, two-sided p-value = 2 x P(X >= x). Large sample (n > 20): z = (x - np)/sqrt(npq) with p = 0.5, p-value = 2 x P(Z > z). Reject H0 only when p-value < alpha.

$z = \frac{x - μ}{σ}, μ = n p, σ = n pq$

⚠️ Top pitfall: Forgetting to discard observations equal to the hypothesized median before computing n and X; using the binomial instead of the normal approximation for n > 20; forgetting to double the tail probability for the two-sided p-value.

Self-check: With n = 39 and X = 25 plus signs, what are mu, sigma, and z for the sign test? (Answer: mu = 19.5, sigma = 3.1225, z = 1.76.)

Connects to: Eigenvalues and Eigenvectors; Cluster Analysis.

Exam Guidance Summary

Must-know: The comprehensive examination covers only what was discussed in the sessions: eigenvector concepts (no drawing, no software), cluster analysis reading skills (distance matrix, agglomeration schedule with the minus-k convention, dendrogram, centroid labeling), and non-parametric procedures in the large-sample case with normal tables.

⚠️ Top pitfall: Preparing from the entire printed handout instead of the taught material — the sessions define the exam scope; also expecting to run software or draw vectors, neither of which is asked.

Self-check: For the sign test, which tables are needed for the large-sample case? (Answer: the normal tables.)

Connects to: Eigenvalues and Eigenvectors; Cluster Analysis; Non-Parametric Procedures: The Sign Test.

Key Industry Applications

Must-know: Cluster analysis segments customers for targeted marketing (each segment gets its own message); the sign test on medians backs retail and service decisions; PCA and LDA are the industry tools built on eigenvalues and eigenvectors.

⚠️ Top pitfall: Treating clustering as a purely academic exercise — in practice the labeled segments directly drive marketing, CRM, and product decisions.

Self-check: Which three customer segments emerged from the Nano purchase-intentions study? (Answer: patriotic, cautious, and innovative customers.)

Connects to: Eigenvalues and Eigenvectors; Cluster Analysis; Non-Parametric Procedures: The Sign Test.

Previous LectureLecture 14 · Linear Discriminant Analysis and Hierarchical Cluster Analysis

Next LectureLecture 16 · Run Test for Randomness

Was this lecture useful?

Loading comments…

🤖

BitsNotes AI Assistant

Subject Notes Assistant

Configure AI Chat

Choose how to access the chatbot

Have your own API key?

Switch to "Bring Your Own Key" tab above for unlimited access with any OpenAI-compatible provider.

AI Provider Preset

API Endpoint URL (OpenAI-Compatible)

API Key

Model Name

🔑 Enter API key above to fetch live models from provider, or enter model name manually.

Remember key for this session (until tab closes)

OpenAI-Compatible API Support

Choose any provider preset (Gemini, DeepSeek, Kimi, GLM, MiniMax, Qwen, OpenAI, Groq, Ollama, etc.) or enter a custom endpoint URL.

Security & Privacy First

Your API key is sent directly from your browser to your specified provider. BitsNotes servers never store or see your key.

Prerequisite Knowledge

Previously Covered in This Subject

15.1 Eigenvalues and Eigenvectors

15.1.1 Vector Representation: Two Equivalent Notations

15.1.2 Matrix–Vector Multiplication and Direction Change

15.1.3 Eigenvectors: Direction Fixed, Magnitude Scaled

15.1.4 Finding Eigenvalues and Eigenvectors

15.1.5 Eigenvectors in Principal Component Analysis and Fisher's LDA

15.1.6 Student Questions and Answers

15.2 Cluster Analysis

15.2.1 What Cluster Analysis Is

15.2.2 The Double Goal: Minimum Within, Maximum Between

15.2.3 Distance Matrix and Similar Pairs

15.2.4 Hierarchical Agglomeration: Schedule and Dendrogram

15.2.5 The Nano Purchase-Intentions Study: Data and Variables

15.2.6 Worked Walkthrough: Reading the Merges

15.2.7 Worked Walkthrough: Deciding the Number of Clusters

15.2.8 Worked Walkthrough: Centroids and Cluster Labels

15.2.9 Student Questions and Answers

15.3 Non-Parametric Procedures: The Sign Test

15.3.1 Parametric versus Non-Parametric Procedures

15.3.2 Why the Median: The Non-Parametric Target

15.3.3 The Sign Test Procedure

15.3.4 The Binomial Connection and Sample-Size Convention

15.3.5 Worked Example: Potato Chip Sales (Small Sample)

15.3.6 Worked Example: Female Members' Median Age (Large Sample)

15.3.7 The Run Test: Preview

15.3.8 Exam Notes for Non-Parametric Procedures

Exam Guidance Summary

Key Industry Applications

ASM Lecture 15 notes · Eigenvectors, Cluster Analysis, and Non-Parametric Procedures

Sections Breakdown

Exam Revision Notes

Eigenvalues and Eigenvectors

Cluster Analysis

Non-Parametric Procedures: The Sign Test

Exam Guidance Summary

Key Industry Applications

Was this lecture useful?

BitsNotes AI Assistant

Configure AI Chat

Sign in to use AI Chat