Groups, Vector Spaces, and Linear Combinations

3.1 Groups

3.1.1 What Is a Group

A group is one of the most fundamental structures in abstract algebra. It consists of exactly two things:

1. A set of elements — a collection of items. You already know many sets: the set of all integers,. the set of all whole numbers,. the set of all even numbers,. the set of square numbers. While we usually think of elements as numbers, they can be anything —. While we usually think of elements as numbers, they can be anything. — matrices, functions, symmetries of a shape, or moves in a puzzle.
2. A binary operation — an operation that takes two elements and produces a result. The basic binary operations are addition, subtraction, multiplication, and division.

When you take a set of elements together with a binary operation, the whole composition is called a group —. When you take a set of elements together with a binary operation, the whole composition is called a group. — provided it follows four basic rules. We write a group as $(G,\circ )$, pronounced "the group G under the operation circle."

Notation. A binary operation is often written with the symbol $\circ$ (used in pure mathematics). So for two elements $x,y$ in the group $G$, the operation is written $x\circ y$. For concrete operations like addition or multiplication, we write the familiar symbols $+$ or $\times$.

3.1.2 Symbol Registry

3.1.3 The Four Group Properties

For a set $G$ and an operation $\circ$ to form a group, all four of these properties must hold. If even one fails, it is not a group. Think of these as the four legs of a table — remove any one and the structure collapses.

Example (holds). Let $G=\mathbb{Z}$ (the set of all integers) and let the operation be addition. Take $2$ and $3$. Then $2+3=5$, and $5$ is also an integer. This works for any two integers — the sum of two integers is always an integer.

Example (holds). Same set $\mathbb{Z}$ with multiplication. $2\times 3=6$, still an integer.

Example (fails). $\mathbb{Z}$ with division. $3÷2=1.5$, which is not an integer. The output has left the set. So $(\mathbb{Z},÷)$ is not a group — it fails closure right away.

Example (fails). Let the set be the natural numbers $\mathbb{N}=\{1,2,3,\dots \}$ and the operation be subtraction. Take $2$ and $10$: $2-10=-8$, which is negative and not in $\mathbb{N}$. Fails closure.

Example (fails). Let the set be the square numbers $\{1,4,9,16,\dots \}$ with addition. $1+4=5$, which is not a perfect square. Fails closure.

Important nuance. If a property holds for some specific cases but fails for others, it does not hold. The property must be true for all elements in the set. A single counterexample is enough to disqualify the group.

Geometric intuition. Picture all the integers on a number line from $-\infty$ to $\infty$. If you add any two integers, the result is still a point on that same line. If you multiply any two integers, the result is still on that line. That is closure.

Example (holds). $\mathbb{Z}$ with addition. Take $a=1,b=3,c=5$:

The same holds for multiplication. For $\mathbb{Z}$ with multiplication, grouping $a(bc)$ vs $(ab)c$ always gives the same product.

Key distinction from commutativity. Associativity is about grouping when you have more than two elements. It is not about the order of the elements. Order is a separate property called commutativity. A group can be associative without being commutative — A group can be associative without being commutative. — for example, matrix multiplication: $(AB)C=A(BC)$ (associative), but A group can be associative without being commutative. — for example, matrix multiplication: $(AB)C=A(BC)$ (associative), but generally $AB\ne BA$ (not commutative).

Example. $\mathbb{Z}$ with addition. The identity is $0$, because $x+0=x$ for any integer $x$.

Example. $\mathbb{Z}$ with multiplication. The identity is $1$, because $x\times 1=x$ for any integer $x$.

So the identity element depends on the operation, not just the set. The same set $\mathbb{Z}$ has identity $0$ under addition but identity $1$ under multiplication.

Example. $\mathbb{Z}$ with addition. The inverse of $2$ is $-2$, because $2+(-2)=0$ (the identity). For any integer $x$, its additive inverse is $-x$.

Example. Multiplication. The inverse of $3$ is $\frac{1}{3}$, because $3\times \frac{1}{3}=1$ (the identity). But note: under the set of integers with multiplication, $\frac{1}{3}$ is not an integer. So $(\mathbb{Z},\times )$ fails the inverse property and is not a group. It passes closure, associativity, and identity — but fails inverse.

Why inverses matter. Inverses give you the ability to "go back" — to undo an operation and return to where you started. If you apply a transformation, you need a way to reverse it. This concept will be critical when we study matrix inverses and their role in solving linear systems $A\mathbf{x}=\mathbf{b}$.

3.1.4 Summary: The Group Test

A candidate $(G,\circ )$ is a group if and only if:

All four must hold. One failure and it is not a group.

3.1.5 Worked Examples: Group or Not?

3.1.6 Visual Intuition

Picture a number line stretching from $-\infty$ to $+\infty$. Mark every integer as a dot. Now imagine the operation of addition as a rule that takes any two dots and slides along the line to a third dot. Because the integers are evenly spaced and extend infinitely in both directions, any addition lands on another dot — closure holds. The zero sits at the center, unmoving — that is your identity. Every dot has a mirror image on the opposite side of zero — that is the inverse.

Now imagine instead the set of natural numbers $\{1,2,3,\dots \}$ on a ray starting at 1 and going right forever. If you subtract, you might slide left past 1 into negative territory — off the ray entirely. The ray cannot contain the result. That is closure failure.

Takeaway: a group's set must be "symmetric" around the identity under its operation. Integers stretch both ways, so addition works. Naturals only go one way, so subtraction breaks.

3.1.7 Real-World & Domain Connection

Group theory is not just abstract formalism. In cryptography, the security of RSA encryption rests on the group of integers modulo $n$ under multiplication —. In cryptography, the security of RSA encryption rests on the group of integers modulo $n$ under multiplication. — finding inverses (decrypting) is easy if you know the group structure (the private key) and computationally infeasible if you do not. In physics, Noether's theorem links every symmetry of a system to a conservation law, and. In physics, Noether's theorem links every symmetry of a system to a conservation law. , and those symmetries form groups. — In physics, Noether's theorem links every symmetry of a system to a conservation law. , and those symmetries form groups. — the group of rotations gives conservation of angular momentum. In machine learning, when. you study matrix factorizations and principal component analysis later in this course,. you are implicitly relying on the group properties of invertible matrices under multiplication. The "undo" operation — the matrix inverse — is what makes solving $A\mathbf{x}=\mathbf{b}$ possible when $A$ is invertible. Without the group properties holding for the space you are working in, every operation becomes a gamble.

3.2 Abelian Groups

3.2.1 Definition

Symbol Registry (new symbols in this section):

3.2.2 The Hierarchy

Think of these concepts as nested, like Russian dolls:

• Every abelian group is a group (it satisfies all four group properties plus commutativity).
• Not every group is an abelian group (it may lack commutativity).

The hierarchy goes: Group $\supset$ Abelian Group $\supset$ Vector Space — each layer adds more structure.

3.2.3 Examples

3.2.4 Geometric Intuition: Why $AB\ne BA$ for Matrices

Think of matrices as transformations of a 2D plane. A matrix can stretch, shear, rotate, or distort the plane.

• Applying matrix $A$ first (say,. a shear that tilts a square into a parallelogram) and then matrix $B$ (say,. a rotation) produces one final shape.
• Applying $B$ first (rotation) and then $A$ (shear) produces a different final shape.

Rotation-then-shear is not the same as shear-then-rotation because the shear acts on a different orientation in each case. First you rotate a square 45 degrees, then shear it — versus first you shear a square, then rotate the resulting parallelogram. The final shapes point in different directions and have different proportions.

This is why matrix multiplication is not commutative — the order of transformations matters. The same principle underlies why putting on socks then shoes works but shoes then socks does not.

3.2.5 Visual Intuition

Draw a 2D plane with a unit square at the center. Label the square's corners. Now apply transformation A (a horizontal shear): the square leans right, becoming a parallelogram. Then apply transformation B (a 90-degree rotation): the parallelogram rotates, pointing upward. The result is a tall, tilted parallelogram.

Now start over — same square. Apply B first: the square rotates 90 degrees, becoming a diamond. Then apply A: the shear tilts the diamond further. The result is a wide, tilted parallelogram — different from the first result.

Both axes show the plane coordinates $(x,y)$. The key landmark is the original square's top-right corner: under A-then-B it lands at one position;. The key landmark is the original square's top-right corner: under A-then-B it lands at one position. ; under B-then-A it lands at a different position. The takeaway: the order of transformations changes the destination — geometrically proving that $AB\ne BA$.

3.2.6 Real-World & Domain Connection

Abelian groups underpin much of modern cryptography and coding theory. Elliptic curve cryptography, used in Bitcoin and secure messaging, operates on an abelian group of points on a curve —. Elliptic curve cryptography, used in Bitcoin and secure messaging, operates on an abelian group of points on a curve. — commutativity ensures that shared secrets computed by two parties match. In physics, the abelian gauge theory (U(1)) describes electromagnetism — In physics, the abelian gauge theory (U(1)) describes electromagnetism. — the photon does not interact with itself because the underlying group is abelian. Meanwhile, the strong nuclear force uses a non-abelian group (SU(3)), where gluons do interact with each other. — Meanwhile, the strong nuclear force uses a non-abelian group (SU(3)). , where gluons do interact with each other. — the non-commutativity creates the complex behavior that binds quarks together. In machine learning, most feature-space operations (addition, weighted averaging) happen in abelian groups (${\mathbb{R}}^n$ under addition), which. In machine learning, most feature-space operations (addition, weighted averaging) happen in abelian groups (${\mathbb{R}}^n$ under addition). , which is why we can reorder and parallelize training computations safely. The non-abelian case appears in geometric deep learning, where transformations like 3D rotations (SO(3)) are not commutative and require special handling.

3.3 Vector Spaces

3.3.1 What Is a Vector Space

These two operations — vector addition and scalar multiplication — are the foundation of all of linear algebra. Roughly 90% of everything you will do with vectors in this course is built on these two basic operations.

Notation. Vectors are typically written in bold: $\mathbf{v}$, or with an arrow: $\vec{v}$. Scalars are written as plain letters: $c,\alpha ,\beta$.

Symbol Registry (new symbols in this section):

3.3.2 Vector Space ${\mathbb{R}}^2$

${\mathbb{R}}^2$ denotes the 2-dimensional plane. Think of it as an infinite flat sheet. Any point in this plane is a position vector — it tells you the position of that point relative to the origin.

A point in ${\mathbb{R}}^2$ is specified by two coordinates $(x,y)$. So a vector in ${\mathbb{R}}^2$ is a combination of two numbers. This is why you need both $x$ and $y$ to locate a point —. This is why you need both $x$ and $y$ to locate a point. — one number alone is not enough in two dimensions.

3.3.3 Vector Space ${\mathbb{R}}^3$

${\mathbb{R}}^3$ is the 3-dimensional space we live in. Any point requires three coordinates $(x,y,z)$. A position vector in ${\mathbb{R}}^3$ looks like $(x_1,y_1,z_1)$.

3.3.4 Vector Space ${\mathbb{R}}^n$

${\mathbb{R}}^n$ is the $n$-dimensional space. A vector in ${\mathbb{R}}^n$ has $n$ components:

You cannot visualize ${\mathbb{R}}^n$ for $n>3$ on a 2D sheet of paper. But you can do the exact same mathematics with $n$-dimensional vectors as you do with 2D or 3D vectors. This is the power of linear algebra — it lets you work with $n$-dimensional spaces even though you cannot picture them.

Real-world motivation. A house can be described by many features. Size (1000 sq ft), age (10 years), number of bedrooms (3), number of bathrooms (1). This is a 4-dimensional vector: $(1000,10,3,1)$. You could add more features — location score, garage size, floor number — and get a 5-dimensional, 6-dimensional, or $n$-dimensional vector. Linear algebra gives you the tools to do math with these high-dimensional feature vectors, which is exactly what happens in machine learning.

3.3.5 Properties a Vector Space Must Satisfy

In practice, for. proving or disproving whether a space is a vector space,. two properties most often fail: closure (under addition or scalar multiplication) and the presence of the zero vector. The other group properties (associativity, commutativity of addition, inverse) tend to hold automatically in most reasonable settings —. The other group properties (associativity, commutativity of addition, inverse) tend to hold automatically in most reasonable settings. — they are inherited from ${\mathbb{R}}^n$ when your set is a subset of ${\mathbb{R}}^n$.

3.3.6 Why ${\mathbb{R}}^2$ Is a Vector Space

Take any two vectors in the 2D plane. Their sum is another vector in the same plane — closure under addition holds. Scale any vector by any real number — it stays in the plane — closure under scalar multiplication holds. The origin $(0,0)$ is present. The vectors under addition form an abelian group (associativity, commutativity, identity, inverses all hold). So ${\mathbb{R}}^2$ is a vector space.

Similarly, ${\mathbb{R}}^3$ and ${\mathbb{R}}^n$ are vector spaces.

3.3.7 Worked Examples: Vector Space or Not?

3.3.8 The Role of the Origin

The origin (zero vector) serves a critical role beyond just being the identity element for addition:

• Reference point. Every position vector is measured from the origin. A vector $(2,3)$ means "2 units in the x-direction and 3 units in the y-direction from the origin ."
• Inversion. To invert a vector (get its negative), you "flip" it through the origin. The vector $(2,2)$ scaled by $-1$ becomes $(-2,-2)$. Adding them gives $(0,0)$ — you return to the origin.
• Observing transformations. When a linear transformation (via a matrix) deforms the space, you need a fixed reference point. That point lets you measure how much each point has moved. The origin is that fixed point. If the origin itself moved, you would lose the ability to quantify the transformation and to invert it.

Think of the origin as the anchor — everything is defined relative to it.

3.3.9 Visual Intuition

Picture the 2D plane with the origin at $(0,0)$. Draw the first quadrant as the top-right wedge — all points with $x>0,y>0$. Now take the point $(2,3)$ in that wedge. Draw an arrow from the origin to that point — that is your vector. Now multiply by $-1$: the arrow flips 180 degrees through the origin and lands at $(-2,-3)$ in the bottom-left wedge. The arrow has left the first quadrant. The quadrant cannot contain all scalar multiples — it is not scale-invariant.

Now draw the line $y=x$ as a 45-degree diagonal through the origin. Take the point $(2,2)$ on it. Multiply by $3$: you get $(6,6)$, still on the line. Multiply by $-1$: you get $(-2,-2)$, still on the line. Add $(2,2)$ and $(-1,-1)$: you get $(1,1)$, on the line. The line is closed under both operations and contains the origin — it is a vector space.

The takeaway: a vector space must be "radially complete" — The takeaway: a vector space must be "radially complete". — from the origin, you can go any distance in any direction the space allows, and. The takeaway: a vector space must be "radially complete". — from the origin, you can go any distance in any direction the space allows, and you stay inside. Bounded regions fail because they cap your travel distance. Shifted lines fail because they cap your reference point.

3.3.10 Real-World & Domain Connection

Vector spaces are the mathematical bedrock of machine learning. Every dataset of $n$ features lives in ${\mathbb{R}}^n$. When you train a linear regression model, you are finding a vector of weights $\mathbf{w}\in {\mathbb{R}}^n$ that operates on feature vectors. When you apply Principal Component Analysis (PCA), you are finding a new basis —. When you apply Principal Component Analysis (PCA), you are finding a new basis. — a rotated coordinate system. — When you apply Principal Component Analysis (PCA), you are finding a new basis. — a rotated coordinate system. — within the same vector space that captures maximum variance. When a neural network layer applies a weight matrix $W$ to an input vector $\mathbf{x}$,. the output $W\mathbf{x}$ is a linear combination of the columns of $W$ —. When a neural network layer applies a weight matrix $W$ to an input vector $\mathbf{x}$,. the output $W\mathbf{x}$ is a linear combination of the columns of $W$. — staying squarely inside ${\mathbb{R}}^m$. The closure property guarantees that these operations never produce undefined results. In computer graphics,. 3D rendering engines operate in ${\mathbb{R}}^3$ and ${\mathbb{R}}^4$ (homogeneous coordinates),. using vector addition for translation and scalar multiplication for scaling objects. In economics, portfolio theory treats asset allocations as vectors in ${\mathbb{R}}^n$, and. In economics, portfolio theory treats asset allocations as vectors in ${\mathbb{R}}^n$. , and the vector space structure ensures that combining portfolios yields another valid portfolio.

3.4 Linear Combinations

3.4.1 Definition

Symbol Registry (new symbols in this section):

3.4.2 Geometric Interpretation

A linear combination is the set of all points you can reach by scaling given vectors and adding the results.

Imagine you have two vectors in ${\mathbb{R}}^2$: ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$. You can change their lengths by multiplying with scalars — stretching or shrinking, and flipping direction if the scalar is negative. Then you add the scaled versions tip-to-tail. The resulting vector points to a new location in the plane.

By varying the scalars $c_1$ and $c_2$ over all real numbers, you can reach many different points in ${\mathbb{R}}^2$. The collection of all points reachable this way is the span of $\{{\mathbf{v}}_1,{\mathbf{v}}_2\}$ (discussed in Section 3.6).

3.4.4 Linear Combination Is the Fundamental Building Block

Almost everything that follows in linear algebra is built on the idea of linear combinations. Span, basis, linear independence, dimension, matrix multiplication, solving $A\mathbf{x}=\mathbf{b}$ — all of it. When you multiply a matrix $A$ by a vector $\mathbf{x}$,. the product $A\mathbf{x}$ is exactly a linear combination of the columns of $A$,. weighted by the entries of $\mathbf{x}$:.

If you deeply understand linear combinations as "scale some vectors and add them up," you have the key. It unlocks the rest of the course.

3.4.5 Visual Intuition

Draw the 2D plane with origin at $(0,0)$. Draw vector ${\mathbf{v}}_1=(2,1)$ as an arrow pointing right-and-up. Draw vector ${\mathbf{v}}_2=(-1,2)$ as an arrow pointing left-and-up. Now vary $c_1$ and $c_2$. When $c_1=1,c_2=1$, place ${\mathbf{v}}_2$'s tail at ${\mathbf{v}}_1$'s tip —. When $c_1=1,c_2=1$, place ${\mathbf{v}}_2$'s tail at ${\mathbf{v}}_1$'s tip. — the resulting arrow from the origin to the final tip is the sum $(1,3)$. When $c_1=-0.5,c_2=2$, flip ${\mathbf{v}}_1$ halfway backward and double ${\mathbf{v}}_2$ — the sum points somewhere else entirely. By sampling many $(c_1,c_2)$ pairs, the tips of the resulting arrows fill the entire plane. The takeaway: two independent vectors in ${\mathbb{R}}^2$ can reach every point through linear combinations.

3.4.6 Real-World & Domain Connection

Linear combinations are the engine behind dimensionality reduction techniques like PCA, where. each principal component is a linear combination of the original features. In recommender systems (Netflix, Spotify), a user's predicted rating for an item is a linear combination of latent feature vectors —. In recommender systems (Netflix, Spotify), a user's predicted rating for an item is a linear combination of latent feature vectors. — the scalars are the user's preferences, the vectors are item embeddings. In finance, a portfolio's return is a linear combination of individual asset returns weighted by allocation percentages. In signal processing, any sound wave can be expressed as a linear combination of sine and cosine waves of different frequencies —. In signal processing, any sound wave can be expressed as a linear combination of sine and cosine waves of different frequencies. — the Fourier transform finds those scalars.

3.5 Linear Independence and Dependence

3.5.1 Formal Definition

Symbol Registry (new symbols in this section):

3.5.2 What Linear Dependence Really Means

If the vectors are linearly dependent, at least one vector in the set is redundant —. If the vectors are linearly dependent, at least one vector in the set is redundant. — it can be written as a linear combination of the others.

Suppose $c_1{\mathbf{v}}_1+c_2{\mathbf{v}}_2=0$ with $c_1\ne 0$. Then you can solve for ${\mathbf{v}}_1$:

So ${\mathbf{v}}_1$ is just a scaled version of ${\mathbf{v}}_2$. The two vectors lie on the same line through the origin — one is a stretched or flipped copy of the other.

With three vectors,. linear dependence means one vector can be expressed as a linear combination of the other two: ${\mathbf{v}}_1=\alpha {\mathbf{v}}_2+\beta {\mathbf{v}}_3$. All three vectors lie in the same plane (or even the same line).

3.5.3 Geometric Interpretation

Linearly dependent vectors in ${\mathbb{R}}^2$: Take ${\mathbf{v}}_1=(2,3)$ and ${\mathbf{v}}_2=(4,6)$. Notice that ${\mathbf{v}}_2=2\cdot {\mathbf{v}}_1$. Both vectors lie on the exact same line through the origin. No matter how you scale and add them, you can never leave that line. You are trapped on a 1-dimensional subspace of ${\mathbb{R}}^2$.

Linearly independent vectors in ${\mathbb{R}}^2$: Take ${\mathbf{v}}_1=(2,3)$ and ${\mathbf{v}}_2=(1,2)$. These vectors point in different directions — they do not lie on the same line. You cannot express one as a scalar multiple of the other. By scaling and adding them, you can reach any point in the entire 2D plane.

3.5.4 The Geometric Test

A quick geometric test for linear independence in ${\mathbb{R}}^2$ and ${\mathbb{R}}^3$:

• Two vectors in ${\mathbb{R}}^2$ are linearly independent if they do not lie on the same line through the origin. If they do, one is a scalar multiple of the other — dependent.
• Three vectors in ${\mathbb{R}}^3$ are linearly independent if they do not all lie in the same plane through the origin. If they do, at least one is a linear combination of the others — dependent.
• In general , a set of $k$ vectors is linearly dependent if at least one vector falls within the span of the others — , a set of $k$ vectors is linearly dependent if at least one vector falls within the span of the others —. it lies on the line, plane, or hyperplane generated by the rest.

3.5.6 Visual Intuition

Draw the 2D plane with origin at $(0,0)$. Plot ${\mathbf{v}}_1=(2,3)$ and ${\mathbf{v}}_2=(4,6)$ — both arrows lie on the same diagonal line from the origin. Draw ${\mathbf{v}}_2$'s tip at $(4,6)$ and ${\mathbf{v}}_1$'s tip at $(2,3)$ — they are collinear. Now plot ${\mathbf{v}}_1=(2,3)$ and a new ${\mathbf{v}}_2=(1,2)$ —. Now plot ${\mathbf{v}}_1=(2,3)$ and a new ${\mathbf{v}}_2=(1,2)$. — the arrows fork away from each other, creating a V-shape from the origin. The collinear pair can only reach points on their shared line (a 1D subspace). The forked pair can reach every point in the plane through linear combinations. The takeaway: independent vectors "fan out" into new dimensions; dependent ones "stack" on the same line or plane.

3.5.7 Real-World & Domain Connection

Linear independence is at the heart of feature selection in machine learning. When building a regression model, you check whether your feature matrix has full column rank —. When building a regression model, you check whether your feature matrix has full column rank. — meaning all feature columns are linearly independent. Dependent features cause multicollinearity, inflating variance and making coefficient estimates unstable. In compressed sensing,. the key insight is that a sparse signal can be recovered from far fewer measurements than the Nyquist rate,. provided the measurement matrix's columns are sufficiently "incoherent" (nearly independent). In network analysis, finding a maximal set of linearly independent paths in a graph reveals the network's throughput capacity. The concept also underpins the invertibility of matrices: an $n\times n$ matrix $A$ is invertible if and only if its columns (and rows) are linearly independent.

3.6 Span

3.6.1 Definition

Symbol Registry (new symbols in this section):

3.6.2 Geometric Interpretation

Span of one non-zero vector in ${\mathbb{R}}^2$. The span of a single vector $\mathbf{v}=(2,1)$ is the line through the origin in the direction of $\mathbf{v}$. Every linear combination is just $c\mathbf{v}$ — Every linear combination is just $c\mathbf{v}$. — a scalar multiple. — Every linear combination is just $c\mathbf{v}$. — a scalar multiple. — so you trace out the entire line through the origin in that direction. A 1-dimensional span.

Span of two linearly independent vectors in ${\mathbb{R}}^2$. With ${\mathbf{v}}_1=(1,0)$ and ${\mathbf{v}}_2=(0,1)$, their span is the entire plane ${\mathbb{R}}^2$. You can reach any point $(x,y)$ by setting $c_1=x,c_2=y$. A 2-dimensional span.

Span of two linearly dependent vectors in ${\mathbb{R}}^2$. If ${\mathbf{v}}_1=(2,3)$ and ${\mathbf{v}}_2=(4,6)$ (where ${\mathbf{v}}_2=2{\mathbf{v}}_1$), both lie on the same line. Their span is just that line — a 1-dimensional subspace. You cannot leave the line no matter how you scale and add.

Span of three vectors in ${\mathbb{R}}^3$.

• All three independent (not coplanar) → span is ${\mathbb{R}}^3$ (3D).
• All three in the same plane → span is that plane (2D).
• All three on the same line → span is that line (1D).

3.6.4 Span, Independence, and Dimension

These three concepts are intimately connected:

• The span tells you what subspace is generated — the "territory" covered by your vectors.
• Linear independence tells you whether any vector in the set is redundant — already inside the span of the others.
• The dimension of the span is the number of linearly independent vectors in the set. If you have $k$ vectors but only $r$ are independent, the span has dimension $r$.

A basis is a set of vectors that are both linearly independent and span the entire space. The number of vectors in a basis is the dimension of the space. In ${\mathbb{R}}^n$, the standard basis is $\{{\mathbf{e}}_1,\dots ,{\mathbf{e}}_n\}$ where ${\mathbf{e}}_i$ has 1 in the $i$-th position and 0 elsewhere.

3.6.5 Connection to Solving Linear Systems

If you have a system of linear equations $A\mathbf{x}=\mathbf{b}$:

• The product $A\mathbf{x}$ is a linear combination of the columns of $A$.
• A solution $\mathbf{x}$ exists if and only if $\mathbf{b}$ is in the span of the columns of $A$ (the column space of $A$).
• If $\mathbf{b}$ is not in the column space, the system has no solution — $\mathbf{b}$ is unreachable.
• If the columns are linearly independent, every $\mathbf{b}$ in the column space has a unique $\mathbf{x}$.

This connection — span = solvability condition — is one of the central ideas of linear algebra.

3.6.6 Visual Intuition

Picture ${\mathbb{R}}^3$ as a room. The origin is a point on the floor. A single vector ${\mathbf{v}}_1$ pointing diagonally up defines a line through the origin — its span. Add a second vector ${\mathbf{v}}_2$ pointing forward — Add a second vector ${\mathbf{v}}_2$ pointing forward. — if it is not collinear with ${\mathbf{v}}_1$,. the two together define a plane through the origin (the floor,. a wall,. or a tilted surface). Add a third vector ${\mathbf{v}}_3$ pointing to the side — Add a third vector ${\mathbf{v}}_3$ pointing to the side. — if it is not in the plane of the first two, the three together span the entire room. The key landmarks: each new independent vector adds a dimension to the span. A dependent vector (already in the current span) adds nothing — it lies flat in the existing subspace. The takeaway: the dimension of the span equals the number of independent vectors you have; every dependent vector is a wasted opportunity.

3.6.7 Real-World & Domain Connection

Span directly translates to the concept of representational capacity in machine learning. The column space of a data matrix is the span of its feature vectors —. The column space of a data matrix is the span of its feature vectors. — it is the set of all predictions a linear model. can make. If the target vector $\mathbf{b}$ lies in this span, the model can fit the training data perfectly (zero training error). If not, you have irreducible error. In deep learning, the activations of a hidden layer span a subspace, and. In deep learning, the activations of a hidden layer span a subspace. , and the layer's width determines the maximum dimension of that subspace. In computer graphics, the span of two texture basis vectors defines all possible surface colors achievable by blending them. In robotics, the span of a robot arm's joint-angle vectors defines its reachable workspace —. In robotics, the span of a robot arm's joint-angle vectors defines its reachable workspace. — the set of all positions the end-effector can attain.

3.7 The Big Picture: Math as Assumptions

3.7.1 Why These Abstract Structures Matter

Groups, vector spaces, and the properties they enforce are not arbitrary mathematical formalism. They are a system of assumptions — a contract. When you do mathematics on vectors, you want guarantees. Scaling them, adding them, combining them — you need to know what can and cannot happen.

The closure property guarantees that your operations never produce something outside the space you are working in. You never have to worry about a vector addition spilling into an undefined territory. The identity and inverse properties guarantee you can "undo" operations — every transformation has a reverse gear. Associativity and commutativity guarantee that the order and grouping of operations do not introduce hidden complexity —. Associativity and commutativity guarantee that the order and grouping of operations do not introduce hidden complexity. — you can reorganize calculations without changing the result.

When these properties hold, you can reason about systems with confidence. When they fail, you know exactly where the math breaks and why. This is the engineer's mindset: know your assumptions, verify them, and understand what happens at the boundaries.

3.7.2 The Hierarchy Revisited

Group
 └─ Abelian Group (Group + Commutativity)
     └─ Vector Space (Abelian Group on vectors + scalar multiplication field)

Every vector space is an abelian group under vector addition. Every abelian group is a group. But the reverse implications do not hold:

• A group lacks commutativity (e.g., invertible matrices under multiplication).
• An abelian group lacks scalar multiplication (e.g., integers under addition — An abelian group lacks scalar multiplication (e.g., integers under addition. — you can scale by integers but not by all real numbers in a way that stays within the integers).

Each level adds structure, and with it, more power. By the time you reach a vector space, you have two operations harmonizing across two sets —. By the time you reach a vector space, you have two operations harmonizing across two sets. — enough machinery to build all of linear algebra.

3.7.3 Connection to Machine Learning

When you work with feature vectors in machine learning — describing houses, images, text embeddings — you are working inside ${\mathbb{R}}^n$. You implicitly assume:

• Adding two feature vectors gives another valid feature vector (closure under addition).
• You can scale features — normalization and standardization depend on scalar multiplication.
• There is a meaningful zero vector (all features zero), which serves as the reference for centering data.
• You can form linear combinations to create new representations — You can form linear combinations to create new representations. — every layer of a neural network computes linear combinations followed by a non-linearity.

These are the same vector space properties discussed throughout this lecture. They are not abstract curiosities — They are not abstract curiosities. — they are the mathematical foundation that makes operations like PCA, linear regression, and They are not abstract curiosities. — they are the mathematical foundation that makes operations like PCA, linear regression, and neural network weight updates well-defined.

3.7.4 What Comes Next

With groups and vector spaces established, the next lectures build on this foundation directly:

• Linear transformations and matrices use the vector space structure to map between spaces.
• Solving $A\mathbf{x}=\mathbf{b}$ relies on the span of columns (existence) and linear independence (uniqueness).
• Determinants and eigenvalues characterize how matrices stretch and rotate the vector space.
• Inner products (Lecture 4) add the concept of angle and length, turning a vector space into an inner product space — a further enrichment of the hierarchy.

3.8 Student Questions and Answers

The student questions from the lecture have been integrated into their respective concept sections for better context. Use this index to locate them quickly.

3.8.1 Groups and Abelian Groups — see Section 3.1

• Division and closure for integers → Section 3.1, Student Q&A
• Specific cases vs. universal properties (single counterexample rule) → Section 3.1, Student Q&A
• Associativity vs. commutativity → Section 3.1, Student Q&A
• Mixing operations in a group definition → Section 3.1, Student Q&A
• Matrix multiplication and group properties → Section 3.1, Student Q&A
• Multiplication of two negatives and closure → Section 3.1, Student Q&A

3.8.2 Vector Spaces and Linear Combinations — see Sections 3.3 and 3.5

• First quadrant as a vector space → Section 3.3, Student Q&A
• Boundaries of a vector space → Section 3.3, Student Q&A
• Moving the origin → Section 3.3, Student Q&A
• Which properties to check when proving a vector space → Section 3.3, Student Q&A
• Scalar multiplication: vector-by-vector or scalar-by-vector → Section 3.3, Student Q&A
• Why only addition and multiplication, not division or subtraction → Section 3.3, Student Q&A
• Zero vector vs. zero scalar in independence definition → Section 3.5, Student Q&A
• Can independent vectors reach the origin → Section 3.5, Student Q&A
• Adding independent vectors and dependence → Section 3.5, Student Q&A
• Zero vector direction → Section 3.5, Student Q&A
• Linear independence and solving $A\mathbf{x}=0$ → Section 3.5, Student Q&A
• Dependent columns: no solution or infinite solutions → Section 3.5, Student Q&A

3.9 Key Concepts and Summary

3.9.1 Summary Table

3.9.2 The Hierarchy at a Glance

Group (4 axioms: closure, associativity, identity, inverse)
  │
  └─ Abelian Group (+ commutativity: x∘y = y∘x)
       │
       └─ Vector Space (+ scalar multiplication over a field)

• Every vector space is an abelian group (under vector addition).
• Every abelian group is a group.
• Reverse implications do not hold.

3.9.3 Quick Counterexample Bank

3.10 Exam Guidance Summary

3.10.1 What to Expect

• Quizzes and midterm: Questions asking you to prove whether a given set with a given operation forms a group typically carry one or two marks. These were explicitly mentioned as common quiz and exam content. Be prepared to verify all four properties systematically.
• Vector space questions: Expect questions that ask you to determine whether a given set (line, quadrant, bounded region) is a vector space. The key is to test closure properties and presence of the zero vector. Show your reasoning — a yes/no answer alone gets no credit.
• Line $y=x$ as a vector space: This exact example has appeared in a previous semester's examination. Know how to verify: any point is $(a,a)$, addition gives $(a+b,a+b)$, scalar multiplication gives $(ca,ca)$ —. Know how to verify: any point is $(a,a)$, addition gives $(a+b,a+b)$, scalar multiplication gives $(ca,ca)$. — all on the line. Zero vector $(0,0)$ is present.
• Bounded region $[-10,10]\times [-10,10]$ as a non-vector-space: Also appeared in a previous year's examination. The counterexample is scaling: pick $(2,5)$ in the region, scale by $10$, get $(20,50)$ — outside. One counterexample is enough.
• The four group properties (closure, associativity, identity, inverse) must be memorized in order. Questions may ask you to verify all four for a given $(G,\circ )$. Use the checklist approach shown in Section 3.1.4.
• Linear independence definition: Expect a question asking you to state the formal definition and apply it to a set of two or three vectors. Remember: set up $c_1{\mathbf{v}}_1+\cdots +c_k{\mathbf{v}}_k=0$ and solve for $c_i$. If only the trivial solution exists, the vectors are independent.
• Abelian/commutative distinction: You may be asked whether a specific group is abelian. Show one counterexample if it is not (e.g., two matrices where $AB\ne BA$). For proof that it is abelian, the operation must commute for all pairs.
• Sample questions will be shared separately for practice.