Unsupervised Deep Learning: Course Overview

This opening session maps the whole course before any technical depth arrives. It answers three questions: what unsupervised deep learning is, what you will build, and how the sixteen planned sessions unfold. Every model family gets a preview here: principal component analysis, autoencoders, autoregressive models, normalizing flows, variational autoencoders, generative adversarial networks, diffusion models, energy-based models, and pretrained multimodal systems.

Treat this document as the course map you will return to all term. Each later session picks one stop on this map and goes deep; when that happens, come back to the matching section here to remind yourself why that model family exists and where it sits next to its neighbors.

1.1 From Supervised Deep Learning to Unsupervised Deep Learning

1.1.1 What Changes When Labels Disappear

Your deep neural networks course was a supervised story: along with each training input you had a training label, and you used both to build models. You started with the multi-layer perceptron, moved through convolutional neural networks and their many variants, and finished with LSTM networks, attention mechanisms, and related machinery. Supervised learning works because labels tell the model what the right answer looks like.

Unsupervised learning removes that comfort. You still have training data — often enormous amounts of it — but no labels come attached. The question this course asks is: with only raw data in hand, what can deep neural networks still accomplish? Two big answers organize everything ahead:

1. Learn features. The data can teach you a new, more succinct representation of itself, known broadly as features (a feature is a number or short list of numbers that summarizes the useful content of an input). A good feature list is shorter than the raw input but keeps what matters.
2. Model the data distribution. The data can teach you its own probability distribution (a rule that says which values are common and which are rare), which you can then sample to create fresh data.

A related idea, self-supervised learning, pushes one step further: by using the training data itself, you can supervise the building of these models without any human-provided labels. For example, hide part of an image and train the network to fill in the missing piece — the image supplies its own label. Time permitting, the course touches self-supervised learning near the end.

The contrast between the two learning settings is worth pinning down side by side:

When to pick which: if someone will pay for labels and you need a specific prediction, stay supervised. If labels are missing, too costly, or the goal is to understand or recreate the data itself, unsupervised methods take over.

The course itself carries history worth knowing. It began as a deep learning specialization created around 2022 with two tracks (deep learning and natural language processing); a 2024 revision added an audio-and-vision track. The subject ran under the name Advanced Deep Learning until the title was made explicit about its content — Unsupervised Deep Learning — with essentially the same material inside.

1.1.2 The Data Landscape: Dough, Icing, and Cherry

A memorable picture explains why this course exists at all. Think of all data around us as a cake.

• The dough is most of the world's data — images, text, documents, media — none of it carrying training labels.
• The icing is the thinner layer of data that comes with labels; supervised learning consumes exactly this.
• The cherry is the sparsest tier: data with reward signals, which reinforcement learning needs.

Visualize the cake as three horizontal bands drawn to scale: a thick base (dough), a thin middle stripe (icing), and one small red dot on top (cherry). The vertical axis is "amount of data," and the takeaway line reads: the biggest data reservoir has no labels, so methods that drink from it unlock the most value.

1.1.3 Where This Course Sits in the Program

This subject follows the machine learning course and the deep neural networks course, and those two are the entire prerequisite list — nothing else is assumed. Concretely, that means you should already be comfortable training an MLP and a CNN in Python, reading a loss curve, and treating images as matrices of numbers. Anything beyond that gets built from scratch here.

Everyone here balances full-time employment with study. So the course is designed for working professionals: broad objectives first, technical detail from the next class onward, and Python coding woven throughout. A constant eye stays on applying deep unsupervised learning to industry problems. No single batch member's industry can be catered to individually, but the methods are taught broadly enough that, with some imagination, they transfer to your own domain — anomaly detection in manufacturing sensor streams, document deduplication in legal tech, and synthetic test data generation in banking analytics are all instances of the same two pillars described above.

Real-world connection: recommendation engines at streaming platforms compress user behavior into learned feature vectors before predicting preferences — pillar one at industrial scale. Image-generation products that create artwork from text prompts are pillar two. This course walks the road from classical PCA to those modern systems.

1.2 Course Scope, Roadmap, and Working Principles

1.2.1 Two Pillars: Feature Representation and Generative Modeling

Everything in this course hangs on two pillars.

Pillar one — representation learning. From the raw data alone, with no information about labels, you learn a new representation of the original inputs that is more succinct. Concretely, a 65,536-pixel image might come out of the pipeline as 5,000 numbers that keep everything important. These learned features can then plug straight into a standard machine learning pipeline downstream — a classifier, a clusterer, or an anomaly detector trained on the features instead of the raw pixels.

Pillar two — generative modeling. Here you try to model probability distributions based on the data. Once the distribution is learned, you sample from it and generate new data that resembles your training set yet is not the training data itself. That loop — model the distribution, then sample — is the core of generative AI. Close to twenty hours of the course go into this distribution-modeling business alone.

The pillars connect rather than compete: a good feature space (pillar one) often becomes the latent space a generative model samples from (pillar two). PCA and autoencoders build pillar one; autoregressive models, flows, VAEs, GANs, diffusion models, and energy-based models are six different routes up pillar two.

1.2.2 Session-by-Session Roadmap

The sixteen planned sessions break down as follows:

1. Session 1 (today): overview, evaluation plan, and previews of every topic.
2. Session 2: principal component analysis and its variants — the classical benchmark for feature representation, rooted in linear algebra.
3. Session 3: autoencoders — deep neural network architectures for creating feature representations.
4. Sessions 4–5: likelihood-based generative models, starting with autoregressive models.
5. Sessions 6–7: normalizing flow models (sometimes the autoregressive discussion borrows a little of this time).
6. Session 8: revision. This point marks the mid-semester boundary.
7. Session 9: variational autoencoders — an autoencoder variant particularly suited to probability distribution estimation.
8. Sessions 10–12: generative adversarial networks and their many variants.
9. Sessions 13–14: diffusion models plus energy-based models.
10. Sessions 15–16: pretrained language models and multimodal pre-training — the culmination tying many threads together.

Read the roadmap as a ladder: sessions 2–3 climb pillar one, sessions 4–14 climb pillar two family by family, and sessions 15–16 stand on both. Some sessions run math-heavy, heavier than what the earlier courses demanded, though you are never expected to reproduce theorem proofs. Because of that weight, the schedule sometimes slips. Topics such as semi-supervised learning and explicit time-series handling usually fall off the delivered syllabus, even though they appear in the posted description.

1.2.3 What Is Not Covered

Three things sit outside the delivered syllabus. Semi-supervised learning is acknowledged but not taught in full (the extra-reading slides exist). Time-series data is not treated explicitly. And graph-structured data belongs to a separate graph neural network course offered in a later semester. What the course does cover deeply is images and text, because those form the bulk of today's digital data across social media and digital documents.

1.2.4 The No-Free-Lunch Principle

One high-level principle governs the whole degree: there is no free lunch in machine learning. Classical techniques are cheap and often enough; deep learning promises more, but better results demand payment — more data, more processing power, more memory.

A quick illustration of the trade-off: PCA on a 65,536-pixel image needs one eigen-decomposition and runs on a laptop; a convolutional autoencoder may need hours on a GPU to beat it by a few percent of reconstruction quality. The deep option wins on quality and loses on cost — no free lunch means you decide which currency to pay.

1.2.5 Books, Notebooks, and Study Resources

The primary textbook is Understanding Deep Learning by Simon J. D. Prince. The full PDF is downloadable free online, complete with answers to selected questions and — most useful of all — a large bank of companion notebooks. Several notebooks map straight onto course modules: one-dimensional normalizing flows, autoregressive flows, latent variable models, and one-dimensional diffusion models. Fair warning: the book itself reads dense for time-starved professionals, so lean on the specific linked chapters tied to each topic rather than reading cover to cover.

The first reference is Deep Learning by Goodfellow and Bengio. Goodfellow invented GANs in the 2014 timeframe — the same researcher who authored this reference book. Two further recommendations round out the shelf. Deep Learning with Python packs in coding examples while keeping strong fundamental discussions alongside the code. And the hands-on O'Reilly volume covering Scikit-Learn, Keras, and TensorFlow supplies the source material for the PCA-and-variants module, which borrows heavily from one of its chapters.

1.3 Generative Modeling: The Central Idea

1.3.1 Model the Distribution, Then Sample From It

Generative modeling means estimating the probability distribution that represents your data. Write $p(\mathbf{x})$ for that distribution: it assigns a high value to inputs $\mathbf{x}$ shaped like your training data and a low value to anything unlike it. Once you hold $p(\mathbf{x})$, two moves become possible:

1. Evaluate. Given a new data point, plug its values into the estimated density function and read off how probable that point is with respect to your training data.
2. Generate. Sample from the learned distribution to create new data — similar to your training data, yet not a copy of any training item.

That second move is the heart of generative AI.

To make "evaluate" concrete with real numbers, suppose a feature of your data (say, pixel brightness averaged over an image) was estimated to follow a standard Gaussian with mean $\mu =0$ and standard deviation $\sigma =1$. Its density is

$$p(x)=\frac{1}{\sqrt{2\pi }\sigma }\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)$$

Plug in a new observation $x=1.0$:

$$p(1.0)=\frac{1}{\sqrt{2\pi }}\exp (-0.5)\approx 0.399\times 0.607\approx 0.242$$

while $x=4.0$ gives $p(4.0)\approx 0.399\times \exp (-8)\approx 0.0001$. The first point reads as typical of the training data; the second reads as nearly impossible. That reading-off move is exactly what "evaluate" means for any learned distribution, however many dimensions it has.

Real-world connection: much of the media circulating on social platforms today is synthetically generated. The techniques in this course are the machinery behind that shift, applied mainly to image and text data.

1.3.2 Prior Exposure: Where You Have Seen Distribution Fitting Before

Distribution estimation is not new to you, and the classroom discussion surfaced exactly where it hides in your past courses.

The Gaussian mixture model plus expectation maximization pairing is, by common admission, the most memorable method from that course. Maybe not the easiest or the most intuitive — but the one that sticks. Hold onto it. In that world, a mixture of, say, three Gaussians approximates a dataset using about a dozen numbers per component. This course spends close to twenty hours pushing the same modeling instinct much further, toward distributions that approximate real data far more closely than a handful of Gaussians can — neural networks replace the fixed Gaussian bumps, so the expressible family grows from a few ellipsoid blobs to essentially arbitrary shapes.

1.3.3 Student Questions: Text Generation Background

Both threads matter because the course's first generative models extend them. RNNs return as the opening example of autoregressive modeling, before elaborate successors such as PixelCNN and PixelRNN take the stage.

Two beginner traps deserve flagging here. First, do not confuse generative with discriminative modeling: a discriminative classifier learns $p(y\mid \mathbf{x})$ — the label given the input — while a generative model learns $p(\mathbf{x})$ over the inputs themselves; only the second can create new data. Second, do not equate generation with retrieval: a good generator produces samples that match the distribution of training data, never copies of stored examples.

1.4 Principal Component Analysis: The Classical Baseline

1.4.1 Geometry: Axes of Maximal Variance

PCA starts from a geometric picture. Picture data scattered as a cloud of points. PCA finds the axis along which the data varies the most — the axis of maximal variation. Mathematically these axes are eigenvectors, and you already own the tool to find them: the eigenvalue-eigenvector computation from your mathematical foundations for machine learning course in the first semester. Each eigenvector ${\mathbf{v}}_i$ pairs with an eigenvalue ${\lambda }_i$ measuring how much spread lives along that direction.

To compress two-dimensional data into one dimension, you project the data onto the dominant direction. As stated in class, you project the data onto the axis of maximum variation — that axis is "the eigenvector which is associated with the maximum eigenvalue." Projecting a data point $\mathbf{x}$ onto the leading eigenvector gives the reduced coordinate

$$z={\mathbf{v}}_1^{\top }\mathbf{x}$$

where ${\mathbf{v}}_1$ is the unit-norm eigenvector carrying the maximum eigenvalue ${\lambda }_1={\max }_i{\lambda }_i$, and $z$ is a scalar — the low-dimensional feature representation of the original point. This matches the standard construction in every dimensionality-reduction reference: project onto the first principal component.

Where does ${\mathbf{v}}_1$ come from? Build it step by step. Given $n$ data points ${\mathbf{x}}^{(1)},\dots ,{\mathbf{x}}^{(n)}$ in $d$ dimensions:

1. Center the data. Compute the mean $\bar{\mathbf{x}}=\frac{1}{n}{\sum }_{m=1}^n{\mathbf{x}}^{(m)}$ and subtract it from every point, so the cloud sits at the origin. Centering matters because PCA measures spread around the mean, not absolute location.
2. Form the covariance matrix. With centered points ${\tilde{\mathbf{x}}}^{(m)}={\mathbf{x}}^{(m)}-\bar{\mathbf{x}}$,

$$\mathbf{C}=\frac{1}{n}\sum _{m=1}^n{\tilde{\mathbf{x}}}^{(m)}{\tilde{\mathbf{x}}}^{(m)\top }$$

where $\mathbf{C}\in {\mathbb{R}}^{d\times d}$; entry $C_{ij}$ says how feature $i$ co-varies with feature $j$.

1. Diagonalize. Solve the eigenproblem

$$\mathbf{C}{\mathbf{v}}_i={\lambda }_i{\mathbf{v}}_i,i=1,\dots ,d$$

with eigenvalues sorted so ${\lambda }_1\ge {\lambda }_2\ge \cdots \ge {\lambda }_d\ge 0$. The covariance matrix is symmetric and positive semi-definite, which guarantees all eigenvalues are real and non-negative and the eigenvectors can be chosen mutually orthogonal.

1. Project. Each point collapses through $z={\mathbf{v}}_1^{\top }\mathbf{x}$ (one coordinate) or, keeping $R$ axes, through $\mathbf{z}={\mathbf{V}}_R^{\top }(\mathbf{x}-\bar{\mathbf{x}})$, where ${\mathbf{V}}_R$ stacks the top $R$ eigenvectors as columns, giving shape ${\mathbb{R}}^{d\times R}$ times ${\mathbb{R}}^d$ equals ${\mathbb{R}}^R$.

Why does maximizing variance lead here? The projected variance along a unit direction $\mathbf{u}$ is ${\mathbf{u}}^{\top }\mathbf{C}\mathbf{u}$. Maximizing that quadratic subject to $\Vert \mathbf{u}\Vert =1$ is a classic constrained-optimization problem; setting its gradient to zero yields exactly $\mathbf{Cu}=\lambda \mathbf{u}$ with the best $\lambda$ being the largest eigenvalue. So "axis of maximal variation" and "leading eigenvector" are two names for one object.

1.4.2 Dimensionality Reduction Arithmetic

Now scale up to real images. Real data lives in dimensions like 256 by 256, 512 by 512, or even 1024 by 1024 pixels. Take a moderate smallish image of 256 by 256 pixels. That is two-dimensional data with

$$256\times 256=65,536$$

values per image. Eigenvectors of such data live in the same dimension: each eigenvector has 65,536 components. Computing them uses the standard eigen-decomposition you already know. What you typically discover for physical data is striking: only a very small fraction of the eigenvalues carry significant value, and the rest are tiny. The spectrum is skewed, and skew means redundancy — a lot of redundancy in the data.

1.4.3 Reconstruction as the Quality Check

How do you know the compressed representation is good? Try to reconstruct your original data from those roughly five thousand values and compare the reconstruction against the original. Formally, with $\mathbf{z}={\mathbf{V}}_R^{\top }(\mathbf{x}-\bar{\mathbf{x}})$ stored, rebuild with

$$\hat{\mathbf{x}}={\mathbf{V}}_R\mathbf{z}+\bar{\mathbf{x}}$$

and measure the reconstruction error, for instance $\Vert \mathbf{x}-\hat{\mathbf{x}}{\Vert }^2$, averaged over the dataset. If the reconstruction resembles the original closely, your PCA is working and the retained dimensions capture the main properties of the data. A useful fact hides in this setup: because PCA keeps orthogonal directions of largest variance, it is provably the best linear compressor of rank $R$ — no other linear projection of the same size reconstructs with smaller squared error. Reconstruction error is the standing quality measure for any feature representation — keep it in mind, because the same test judges autoencoders later.

1.4.4 Scalable Variants and When PCA Fails

Two weaknesses motivate the session 2 discussion. First, scale: when the dimension is huge, forming and diagonalizing the covariance matrices becomes unwieldy — a $d\times d$ matrix at $d=65,536$ holds over four billion entries. Approximate variants rescue practicality — randomized PCA and incremental PCA alleviate the unwieldy nature of classical PCA when data abounds; randomized variants sketch the spectrum cheaply, while incremental versions fit a stream of mini-batches without holding everything in memory.

Second, shape: PCA rests on the principle of maximum variance and implicitly assumes the data obeys unimodality — one blob, like a single Gaussian.

Visual intuition for choosing components: draw the scree plot — eigenvalues ${\lambda }_i$ on the vertical axis against component index $i$ on the horizontal axis. The curve plunges steeply, then flattens into a long low tail. The elbow marks the boundary between signal directions (steep part) and noise directions (flat part); keep the components left of the elbow.

1.4.5 Eigenfaces: A Visual Worked Example

The classic demonstration uses human faces. Take a dataset of many human face shots, each image 64 by 64 grayscale — so each image is a vector of $64\times 64=4,096$ brightness values, and the covariance matrix lives in $4,096$ dimensions. Compute the eigenvectors of the face dataset and display the leading ones as images — these are eigenfaces. A mosaic of eight columns by eight rows shows sixty-four eigenfaces in all, ordered left-to-right, top-to-bottom by decreasing eigenvalue.

The progression teaches the concept visually, and the skew of the eigenvalue spectrum explains every step of it. The very first eigenface — the one paired with the maximum eigenvalue — looks mostly like a blurred average face, carrying overall appearance but little discriminating power: it encodes the gross lighting-and-head-shape variation shared by everyone. Moving through the middle of the gallery, distinct facial features gain prominence: this eigenface highlights eyes, that one shadows around the nose, another frames the mouth region. Toward the tail, the images start looking noisy. Those eigenfaces encode only very fine variations of the facial images. Beyond them lie ever-noisier directions, useless as features — practitioners discard them. Reading the mosaic left to right, top to bottom, the average face visibly dissolves into recognizable facial parts and finally into pure noise: that drift toward the noisy tail is the eigenvalue spectrum made visible.

Real-world connection: eigenfaces are the historical bridge between linear algebra and face recognition systems — the same projection logic you will formalize next session powered early industrial face pipelines in the 1990s, and the descendents of that idea still run compact face embeddings inside phone unlock systems today. In quantitative finance, the identical mathematics extracts the leading factors driving yield-curve movements.

1.4.6 How Many Components to Keep

The stopping rule heard in class, quoted: "sum total of the first r eigenvalues divided by sum total over all eigenvalues must be greater than or equal to 90 percent." Reconstructed:

$$\frac{{\sum }_{i=1}^R{\lambda }_i}{{\sum }_{j=1}^d{\lambda }_j}\ge 0.90\text{(sometimes the bar is set at }0.95\text{)}$$

Here $R$ is the number of retained components, $d$ the full data dimension, and ${\lambda }_1\ge {\lambda }_2\ge \cdots \ge {\lambda }_d$ the eigenvalues sorted from largest to smallest. Informally: these $R$ eigenvalues capture ninety percent of the variation in the images. This ratio-of-sums form is the standard cumulative-variance criterion — it is exactly what scikit-learn's explained-variance-ratio reports when you ask PCA to preserve a fixed share of variance such as ninety-five percent. In the eigenface gallery, keeping the top 64 components satisfied this rule for the 64-by-64 face dataset shown.

1.5 Autoencoders: Learned Feature Extraction

1.5.1 Architecture and Training Objective

An autoencoder looks like a standard multi-layer perceptron drawn symmetrically. Inputs feed into one or two hidden layers, and the hidden layers feed outputs. Every input node connects to every hidden node, and every hidden node connects to every output node — the usual MLP wiring. The twist is the target. You train this network so that its output lands about equal to each input. Put as a rule: the target outputs are roughly equal to the inputs themselves. Written as an objective,

$$f(\mathbf{x})\approx \mathbf{x}$$

where $\mathbf{x}$ is the input vector and $f$ is the full network mapping input to output. In training you minimize the reconstruction loss

$$L=\frac{1}{n}\sum _{m=1}^n{\Vert {\mathbf{x}}^{(m)}-f\left({\mathbf{x}}^{(m)}\right)\Vert }^2$$

so training pushes each reconstructed output back toward its original input despite the narrow hidden bottleneck in between. Split $f$ into two halves: an encoder $g$ that maps input to the hidden code $\mathbf{h}=g(\mathbf{x})$, and a decoder $h$ that rebuilds from the code, so $f=h\circ g$. One honest caveat given upfront: the MLP picture is the general-purpose skeleton, not the exact canonical architecture — the idea matters more than the precise wiring diagram.

Everyday picture: packing for a flight with one tiny suitcase. The encoder decides what goes in; the suitcase's size is the bottleneck; the decoder reassembles your outfit at the destination. A good packer keeps the essentials and folds away the rest — exactly what reconstruction loss rewards. The analogy breaks in one place: a neural packer learns what to keep by gradient descent on rebuilding quality, not by a checklist anyone wrote down.

1.5.2 Hidden Activations as Features, and PCA as a Special Case

After training completes, look at the hidden nodes. Their outputs are the features — the autoencoded features — because from those hidden activations alone you can recreate the original image to a large extent. That recreating power is why they deserve the name features. At inference time you throw away the output side entirely: the remaining encoder part spits out features for any new input.

Why bother, when PCA exists? Because nonlinearity buys reconstruction power. When the autoencoder uses nonlinear activation functions in its hidden layers, the learned features reconstruct better than PCA-based features — curved manifolds of images get hugged more tightly than any flat plane can. And the connection runs deeper: PCA falls out as a special case of this exact architecture. Normalize the input in a certain way, use plain linear activation functions in every hidden and output node, train on the reconstruction objective — and the autoencoder realizes PCA: the optimal linear encoder's weights span the same leading eigenvector space you met in section 4. Session 3 walks through this simple way of seeing PCA inside the autoencoder framework, then switches on nonlinearities and watches reconstruction improve.

When to pick which: start with PCA to set the bar; move to autoencoders when the reconstruction gap justifies the extra cost — the assignment in this course makes you run both sides of that comparison yourself.

The two conditions making the special case true are worth memorizing: linear activations everywhere, plus the particular input normalization used in that derivation. Drop either condition and the equivalence breaks.

1.5.3 Convolutional Autoencoders for Images

For image data, a specialized build swaps fully connected layers for deep convolutional layers. Deep CNN-based autoencoder architectures perform a lot better than the plain version on images, especially as measured by reconstruction error — convolution filters respect the local, translation-friendly structure of pixels that a fully connected mesh ignores. The dedicated session also shows how to extract features from the convolutional encoder, then compares reconstructions against PCA baselines. That comparison is exactly what the group assignment later asks you to run yourself.

Real-world connection: the same reconstruct-and-flag loop powers industrial anomaly detection — a CNN autoencoder trained only on defect-free parts (sheet metal, PCB boards) reconstructs normal samples well and defective ones badly, so a spike in reconstruction error flags the fault automatically. Remember the no-free-lunch principle: the better features come with higher processing power, memory, and data requirements.

1.6 Autoregressive Generative Models

1.6.1 Factorizing the Joint Probability of an Image

Autoregressive models turn generation into probability bookkeeping. Suppose the data is an image, though text works identically. Treat the image as made of n-by-n pixels. A naive first framing assumes all individual pixels are independent and identically distributed, so the probability of the whole image collapses into a product of per-pixel probabilities:

$$p(\mathbf{X})=\prod _{k=1}^{n^2}p(x_k)$$

where $\mathbf{X}$ is the full pixel grid and each $x_k$ is one pixel's intensity. This independence assumption throws away everything images are made of — neighboring pixels correlate strongly — so it serves only as the straw-man baseline.

The autoregressive refinement conditions every factor on the past instead. Start from the exact chain rule of probability, which holds for any joint distribution:

$$p(\mathbf{X})=p(x_1)p(x_2\mid x_1)p(x_3\mid x_1,x_2)\cdots$$

each step peeling off one variable conditioned on everything before it — no approximation has entered yet. Two spoken lines anchor what comes next. First: "probability of this particular pixel depends... based on what has happened in the past." Second: "autoregressive basically means probability of a particular point depends on all the points that have happened in the past, but not on the future points." Applying the chain rule over the $n^2$ pixels, the joint distribution factorizes into a chain:

$$p(\mathbf{X})=\prod _{k=1}^{n^2}p(x_k\mid x_1,x_2,\dots ,x_{k-1})$$

so the joint probability factorizes into a product of pixel conditionals given past pixels. One convention makes this precise: the pixels are visited in a fixed order, and the standard choice is raster-scan order — left to right along each row, rows proceeding top to bottom — which is exactly how PixelRNN- and PixelCNN-style successors process an image.

1.6.2 Training by Maximum Likelihood, and Exact Densities

How do the per-pixel conditionals get learned? By the maximum likelihood principle: maximize the probability of the data. Expectation maximization or likelihood maximization forms the basic framework for these autoregressive models. Written directly from the words "maximize the probability of the data,"

$$\hat{\theta }=\arg \underset{\theta }{\max }\prod _{m=1}^M{p}_{\theta }\left({\mathbf{x}}^{(m)}\right)$$

where $\theta$ collects the model parameters and the product runs over the $M$ training examples. In practice everyone maximizes the log-likelihood instead,

$$\hat{\theta }=\arg \underset{\theta }{\max }\sum _{m=1}^M\log p_{\theta }\left({\mathbf{x}}^{(m)}\right)$$

and the two objectives have identical maximizers because the logarithm is a monotonically increasing function — whatever argmax wins the first wins the second. Why prefer logs: multiplying millions of per-pixel probabilities collapses toward zero until floating-point arithmetic gives up, while the log turns the long product into a comfortable sum of per-example terms that gradient descent handles batch by batch.

The payoff of this likelihood route is exactness. Autoregressive models give an exact method for probability density calculation. The model defines the PDF of the data itself. Plug a new data point's values into that density function and read off its probability with respect to the training data. No approximation sneaks in anywhere — multiply the stored conditional probabilities and you hold the model's true probability for that image under its learned distribution.

Sampling closes the loop. Draw from the learned distribution and you generate fresh data. In the worked examples shown, generated animal images — with all training data drawn from one animal category dataset — looked quite realistic. Named successors of the RNN opening act include PixelCNN and PixelRNN, which apply the autoregressive principle pixel by pixel on images.

1.6.3 Sequential Generation: The Main Drawback

Ask what weakness the autoregressive design carries and the answer is its generation loop.

Real-world connection: GPT-family systems are autoregressive models underneath, with bells and whistles layered on top — assumptions, engineering innovations, transformer architectures, pre-training. At heart they ride this exact principle. Knowing the plain autoregressive core tells you what all the engineering surrounds.

1.7 Normalizing Flow Models

1.7.1 Transforming a Simple Base Distribution

Normalizing flows attack generation from the opposite end to autoregressive models. Start by assuming a basic, easy-to-sample probability distribution — a Gaussian or even a triangular distribution. Call samples from it $\mathbf{z}$. Then push those samples through a stack of transformations of a certain type, transformation one, transformation two, and onward, producing a new transformed distribution. Compare that transformed density against your actual data distribution. Assume the training data comes from the distribution being shaped, then train the transformations on the deviation between transformed density and data. In other words, you learn transformations that bend the simple base density until it estimates the original data distribution. Schematically,

$$\mathbf{z}\sim \mathcal{N}(0,\mathbf{I}),\mathbf{x}=T_K\circ T_{K-1}\circ \cdots \circ T_1(\mathbf{z})$$

where each $T_i$ is a learned invertible transformation, $\mathcal{N}(0,\mathbf{I})$ is the multivariate standard normal with zero mean and identity covariance matrix, and the composition turns a simple base distribution into the data distribution. Like autoregressive models, this is a likelihood-based method: maximizing likelihood is what trains the transformations. Generation then runs the pipe forward: sample from the original simple distribution, send the sampled value through the transformations, and collect new data.

Everyday picture: press a flat sheet of rubber onto an irregular statue. Stretching and squeezing moves the material around until it hugs every contour. Bijectivity means you may stretch or squeeze anywhere, but never tear the sheet (one piece splitting into two) and never glue regions together (two points merging) — every original point must stay findable. Where the analogy breaks: rubber has stiffness limits, while flows accept any smooth invertible warping, however extreme.

The missing link — how densities transform under such a bend — comes from the change-of-variables rule of probability. For one transformation $f$,

$$Pr(\mathbf{x}\mid \mathbf{\varphi })={\left|\det \frac{\partial f[\mathbf{z},\mathbf{\varphi }]}{\partial \mathbf{z}}\right|}^{-1}Pr(\mathbf{z}),\mathbf{z}=f^{-1}[\mathbf{x},\mathbf{\varphi }]$$

where $\partial f/\partial \mathbf{z}$ is the $D\times D$ Jacobian matrix of partial derivatives and its absolute determinant measures how the transformation stretches volume at that point. Intuition: probability mass behaves like an incompressible fluid. Where $f$ expands space, the same mass spreads thinner, so density drops by exactly the volume-stretch factor; where $f$ compresses, density rises. For the full stack, each layer contributes its own factor, and the total correction is the product of per-layer absolute determinants — which is what training actually maximizes against the data likelihood. The dedicated sessions build this machinery carefully; here, hold onto the picture: invertible bends plus bookkeeping of local volume change equals an exact, trainable density.

1.7.2 The Bijectivity Constraint and Its Costs

The transformations cannot be arbitrary. They must be bijective — one-to-one mappings, so that one data point never maps to two different points. As long as every transformation stays one-to-one and not one-to-many, the flow model works: the inverse path back to the base distribution exists, so the likelihood formula above always evaluates cleanly. But that bijectivity requirement also caps quality: restricting the admissible transformations puts a restriction on how well the flow can approximate complicated real data distributions — any architecture trick that breaks invertibility is off the table, however expressive it would be. The speed profile splits too — generation of the data is very fast (one pass through the stack), but training is not (every likelihood step drags the Jacobian determinant along).

When to pick which: need exact densities plus fast sampling and can afford expensive training? Flows. Need simple training and can stream tokens slowly? Autoregressive.

Real-world connection: researchers at Apple built StarFlow, a transformer-based flow model, showing flows can also generate very high-quality new images — evidence the family remains competitive alongside GANs and diffusion. The same exact-likelihood property makes flows natural anomaly scorers in fraud pipelines: transactions far from the learned high-density region score low and get flagged.

1.8 Variational Autoencoders

1.8.1 Latent Spaces With a Gaussian Constraint

After mid-semester the course returns to autoencoders with a probabilistic twist: the variational autoencoder, an autoencoder variant particularly suited to probability distribution estimation. Unlike the exact likelihood methods before it, the VAE is explicitly an approximate method — a latent space method. The approximation is a constraint: the distribution of the autoencoded feature space is forced to be of a certain nature — Gaussian distributed along the axes. Written compactly,

$$\mathbf{z}\sim \mathcal{N}(0,\mathbf{I})$$

where $\mathbf{z}$ is the latent code produced by the encoder, drawn from a standard normal with zero mean and identity covariance. The word latent means unobserved: the latent variables are not present in the data; they are calculated from the original data. To generate, sample codes from the constrained latent space and pass those samples through the decoder — new data emerges.

The constraint does not enforce itself. Training balances two demands at once, summarized by one objective called the evidence lower bound (session 9 derives it fully):

$$\text{ELBO}=\underset{\text{reconstruction quality}}{\underbrace{{\mathbb{E}}_{\mathbf{z}\sim q(\mathbf{z}\mid \mathbf{x})}[\log p(\mathbf{x}\mid \mathbf{z})]}}-\underset{\text{stay close to the standard normal}}{\underbrace{D_{\mathrm{KL}}(q(\mathbf{z}\mid \mathbf{x})\Vert \mathcal{N}(0,\mathbf{I}))}}$$

Here $q(\mathbf{z}\mid \mathbf{x})$ is the encoder's distribution over codes for input $\mathbf{x}$, $p(\mathbf{x}\mid \mathbf{z})$ scores how well the decoder rebuilds $\mathbf{x}$ from code $\mathbf{z}$, and $D_{\mathrm{KL}}$ is the Kullback-Leibler divergence — a non-negative measure of how much one distribution differs from another, equal to zero only when they match. The first term rewards faithful rebuilding; the second pulls every encoded region toward the same standard-Gaussian template. Squeeze both and the latent space fills in uniformly: no voids, no rogue corners.

Everyday picture: think of the latent space as a planned parking garage on a perfect grid. A plain autoencoder parks cars anywhere, leaving gaps nobody can decode into anything. The VAE's KL term acts like the garage attendant, nudging every car toward its designated slot so that picking any spot at random still yields a legitimate vehicle. Where the analogy breaks: the attendant never finishes — the pull is soft and permanent, balancing against reconstruction pressure forever.

1.8.2 Semantic Axes and Structured Latent Space

What makes the Gaussian constraint precious is structure. Train a VAE on varied human faces and some latent dimensions start capturing semantic attributes. One axis might run from neutral to smiling — pick points along it and faces shift from morose to visibly happy. Another axis separates more masculine from more feminine appearances. Others toggle wearing glasses or not, blonde hair or not, curly hair or not, beard or not. Move along a chosen axis and a generated face alters its appearance smoothly — more smile, less smile, curlier hair.

The same trick transfers to text: a VAE variant can rewrite textual data, changing sentiment and style — aggressive writing versus neutral versus submissive. Historically, the variational autoencoder was one of the initial techniques that gave reason to believe semantically meaningful modified data could be created at will.

And note why the Gaussian restriction matters rather than merely constrains. It keeps the latent space regular and structured. Two points sitting close together there decode into semantically similar outputs in the original data space. Without that regularity, neighboring codes could decode to unrelated images, and axis-walking would fail.

When to pick which: for pure feature compression stay with the plain version; when you need to sample new data or walk smoothly between meanings, pay the VAE's price.

Real-world connection: interactive face-editing sliders in photo tools descend directly from VAE axis-walking — drag one control and the smile deepens while everything else holds steady. In drug discovery pipelines, VAE-style structured latent spaces let chemists walk between molecular structures while staying inside the region of chemically plausible candidates.

1.9 Generative Adversarial Networks

1.9.1 Generator Versus Discriminator

Generative adversarial networks borrow from game theory — adversarial learning between two modules locked in a duel. The generator takes a random noise signal, say a 128-dimensional random Gaussian vector, and passes it through a neural network — often convolutional — whose output is an image. Because the input is random noise, that output initially looks random too. The discriminator sees pairs of images: fakes from the generator and reals from a genuine training set, say a dataset of dog photos. Its job is detection — identify which images are fake and which are real.

Training sharpens both sides against each other. The discriminator learns to catch generated images as fake. Meanwhile the generator trains itself so its outputs begin looking slowly, slowly like the dog images. Training ends when the discriminator can no longer distinguish fakes from real training images. When this adversarial process stabilizes, the generator produces awfully realistic images starting from pure noise. Structurally,

$$\mathbf{z}\sim \mathcal{N}(0,\mathbf{I}),\tilde{\mathbf{x}}=G(\mathbf{z}),D(\cdot )\in \{\text{real},\text{fake}\}$$

with $G$ the generator network and $D$ the discriminator network scoring how real an input looks. Where the counterfeiter analogy breaks: both players are retrained after every round — no human detective rebuilds their entire visual system between cases, but gradient descent updates both networks continuously.

Goodfellow — the same researcher who authored the famous Deep Learning reference book — invented GANs in the 2014 timeframe. Publication volume tells the impact story. Around 2019, roughly 350 papers per year still appeared in the GAN area. Many more variants have followed since, each pushing more realistic and larger images. The course budgets a solid seven to eight hours on GANs and their variants.

1.9.2 Min-Max Training and Brittleness

GANs introduce a first in your machine learning education. Every algorithm you have studied so far reduced to one maximization or one minimization. Adversarial training does both at once: you are maximizing one part — the discriminator's skill — while minimizing another — the generator's error against it. Schematically,

$$\underset{G}{\min }\underset{D}{\max }V(G,D)$$

where $V(G,D)$ scores how well the discriminator distinguishes while the generator undermines it. The explicit value function behind that schematic is worth writing out once. Let $D(\mathbf{x})$ output, through a logistic sigmoid, the estimated probability that $\mathbf{x}$ is a real training image. Then

$$V(G,D)={\mathbb{E}}_{\mathbf{x}\sim \text{data}}[\log D(\mathbf{x})]+{\mathbb{E}}_{\mathbf{z}\sim \mathcal{N}(0,\mathbf{I})}\left[\log \right(1-D(G(\mathbf{z}))\left)\right]$$

Read it term by term. The first expectation rewards the discriminator for assigning high real-probability to genuine data. The second rewards it for assigning low real-probability to generator fakes — so maximizing $V$ sharpens detection. The generator attacks exactly those same terms: pushing its samples toward high $D$-scores makes $1-D(G(\mathbf{z}))$ small and drives the second expectation down, which is why the outer problem minimizes over $G$. At equilibrium, textbook analysis shows the discriminator sits at chance level — outputting about 0.5 on everything — because perfect fakes carry no detectable signal.

When to pick which: want exact probabilities? Flows or autoregressive. Want fast, stunning samples and can stomach unstable training? GANs.

1.9.3 Transposed Convolutions and Parallel Generation

Look closer at the generator's internals and a curious operator appears. Ordinary convolutions with strides shrink spatial size. Generation needs the reverse: growing a tiny seed into a full picture.

The second structural advantage answers autoregression's weakness. GAN generation is parallel, not sequential — the whole image pops out at once from one forward pass through the generator, with no token-by-token dependency chain. Fast parallel sampling plus implicit density learning, traded against brittle min-max training, is the GAN bargain.

Real-world connection: StyleGAN-family generators produce the photorealistic synthetic faces behind many digital avatars, and GAN-based super-resolution sharpens low-resolution medical and satellite imagery — both industrial jobs that need parallel one-shot synthesis rather than token streams.

1.10 Diffusion Models

1.10.1 Noising Forward, Denoising Backward

Diffusion models arrive from statistical physics with a disarmingly simple recipe. Take an original image and add a little noise to it. Add noise again, and again — multiple steps — until what remains is pure Gaussian-distributed noise. Forward chain sketched,

$${\mathbf{x}}_0\to {\mathbf{x}}_1\to \cdots \to {\mathbf{x}}_T\sim \mathcal{N}(0,\mathbf{I})$$

where ${\mathbf{x}}_0$ is the clean image and each step degrades it further until ${\mathbf{x}}_T$ is indistinguishable from Gaussian noise. Then learn the reverse of the noise addition process. Generation follows: sample noise from the Gaussian, pass it through the learned denoising process step by step, and out comes new data similar to your original data.

The narrative becomes precise with the per-step transition rule used across diffusion references. Each step attenuates the current image slightly and injects fresh Gaussian noise:

$${\mathbf{x}}_t=\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1}+\sqrt{{\beta }_t}{\mathbf{ϵ}}_t,{\mathbf{ϵ}}_t\sim \mathcal{N}(0,\mathbf{I})$$

equivalently written as a conditional distribution,

$$q({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1})=\mathcal{N}({\mathbf{x}}_t;\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1},{\beta }_t\mathbf{I})$$

where each ${\beta }_t$ is a small number between 0 and 1, chosen ahead of time; the full sequence ${\beta }_1,\dots ,{\beta }_T$ is called the noise schedule because it sets how fast structure dissolves. Two properties make this design trainable. First, the chain is Markov: step $t$ depends only on step $t-1$, never further back, which keeps every learning problem local. Second, with enough steps $T$, all traces of the original image wash out completely, leaving a pure standard normal — so generation can always start from noise you know how to sample. The network's job during training is simply to predict the noise ${\mathbf{ϵ}}_t$ mixed into a given noisy image, which turns denoising into ordinary supervised regression against known answers.

Visual intuition: imagine a stack of horizontal strips showing the histogram of pixel values after more and more steps. The top strip — the raw data histogram — looks lumpy and multi-modal. Moving down the stack, the lumps melt outward into a smooth symmetric bell centered at zero; the last strip is indistinguishable from the standard normal curve. That slow melting from any shape into one universal bell is what the reverse process must undo.

Why the name diffusion? Watch a glass of water. Drop a bead of red dye in. For a while you see the color moving and swirling inside the water. Eventually the color vanishes into uniform tinting, the whole glass shifting shade with how much dye entered. Adding noise to images diffuses the same way — structure dissolves gradually until only homogeneous randomness remains — and the model learns to run that film backward.

On speed: iterative denoising over hundreds of steps is the family's standing weakness, but the reverse process does not have to crawl. Modern variants define the same model over a shortened ladder of time points, letting sampling skip most steps — good samples in roughly fifty denoising passes instead of several hundred. The textbook treatment names these denoising diffusion implicit models and related accelerated samplers; the course covers them when diffusion gets its dedicated sessions.

1.10.2 Real Systems and Practical Trade-offs

Today, denoising diffusion is considered one of the most promising methods for creating new data, showing great potential especially for images. Named systems include Stable Diffusion and DALL-E — working proof that diffusion scales. Text is reachable too, but needs adaptation: images are modeled by continuous distributions, whereas text normally uses discrete distribution functions, so specific modifications adapt the diffusion machinery to discrete tokens. The standing weakness mirrors flows: the iterative denoising process is slow, though ways exist to expedite it.

When to pick which: need instant samples and accept training drama — GANs; want stable training and top fidelity with patience — diffusion.

Real-world connection: text-to-image products such as Stable Diffusion and DALL-E run exactly this reverse-noising loop behind their prompt boxes, refining pure static into artwork over dozens of denoising rounds. Beyond media, diffusion-based generators design protein structures and drug candidates, where gradual refinement beats one-shot guessing.

One adjacent family shares the physics root. Energy-based models borrow the notion of energy from statistical mechanics — the energetics of atomic movements. They import statistical physics principles into generative modeling, granting tremendous flexibility in density estimation. Sessions 13–14 cover both families.

1.11 Energy-Based Models and Statistical Physics

1.11.1 Energy Ideas From Statistical Mechanics

Energy-based models complete the physics-flavored corner of the course. The concept of energy here comes straight from statistical mechanics and the mathematics of atomic movements: systems settle into low-energy configurations, and probabilities of configurations follow from their energies. In physics, hot atoms explore many arrangements but prefer stable, low-energy ones; the probability of a configuration falls off exponentially as its energy rises. Generative modeling borrows exactly that law: attach an energy $E(\mathbf{x})$ — just a scoring network outputting low numbers to data-like inputs and high numbers to unlike ones — and define the model's distribution through

$$p(\mathbf{x})\propto e^{-E(\mathbf{x})}$$

where the symbol $\propto$ hides a normalizing constant that makes everything sum to one. Read it in plain words: halve the energy and the configuration becomes far more probable; a face-shaped input should land in a deep valley of $E$, while static noise sits high on the hills. Learning means carving the landscape until real data occupies the valleys.

Everyday picture: think of an egg carton shaken gently on a table. The eggs always end up nested in dents — the low-energy spots — never balanced on ridges. Training an energy-based model shapes the carton so your dataset's examples are precisely the dents. Where the analogy breaks: the mathematical carton has thousands of dimensions, and its dents move during training rather than being fixed at manufacture.

The catch hiding behind the proportionality sign: computing the normalizing constant (physicists call it the partition function) requires summing over every possible configuration — impossible directly at image scale. So these models lean on sampling-based tricks that avoid the sum, which is also why training them earns its own dedicated treatment. Importing those principles into generative modeling provides another lever for learning probability distributions from data, alongside likelihood maximization, adversarial play, and iterative denoising. The promised payoff is a tremendous amount of flexibility in how density models get estimated — the energy function can be almost any network, unburdened by invertibility or tractable-conditionals requirements that handcuff flows and autoregressive models.

Compared with the other families, this thread receives lighter treatment in the schedule, sitting beside diffusion in the post-midterm sessions.

Real-world connection: restricted Boltzmann machines — energy-based models with a two-layer architecture — powered the recommender ensembles that made headlines during the Netflix Prize competition, and the same energy-and-sampling thinking survives today inside contrastive representation-learning objectives used across vision and language pre-training.

1.12 Pretrained Models and Multimodal Learning

1.12.1 Pretrained Language Models: GPT, BERT, ELMo

The last two sessions gather everything into modern pre-trained systems. On the language side, the course walks the overall architecture of GPT-style models, alongside BERT — which came from Google — and ELMo among others. A one-line orientation keeps the three straight: GPT-class systems generate text left to right using exactly the autoregressive principle from section 6; BERT reads in both directions and fills in deliberately hidden words, excelling at understanding tasks; ELMo contributed the earlier idea that word representations should shift with context — one word, different meanings, different vectors. All three earn their power from pre-training: first absorbing statistical structure from enormous unlabeled corpora through self-supervised objectives, then adapting cheaply to specific jobs. Your earlier brush with that idea came through transfer learning — reuse a model trained on a big problem as the starting point for a small one — and these sessions finally give pre-training proper treatment.

A dedicated LLM course exists elsewhere in the program, covering language models exclusively with minimal overlap. Here instead, pretrained giants close the course once the generic generative machinery is built. LLMs are best understood as one application of the broader unsupervised toolkit — swap "image pixels" for "word tokens" and the distribution-modeling loop you already know reappears underneath.

1.12.2 Vision-Language Applications and Molecule Generation

Multimodal pre-training extends the same ideas across media. Visual captioning maps a given image to a textual description of it — an encoder digests pixels while a decoder emits words, two of the course's representation families shaking hands. CycleGAN converts between visual domains — zebras synthesized out of horses — with a loss function enforcing cycle consistency: the translated image must translate back to something matching where it started, so a horse turned zebra and returned must still be the same horse. That round-trip constraint lets the network learn translation between unmatched collections, no paired examples needed. Generated audio, generated video, generated mathematical symbols, and generated code all fall within reach of these methods.

Generating molecules deserves special mention: it matters enormously for computational biology, pharmaceuticals, and drug discovery — propose candidate structures in silico before any lab synthesizes them. Molecule work leans on graph representations, connecting directly to the graph neural network course — students heading there will reuse principles taught in this course, applied to graphical structures. The course teaches the generic principles; molecule generation is one domain where those principles pay off spectacularly.

Real-world connection: captioning systems describe photographs for accessibility tools and newsroom automation; CycleGAN-style domain translation handles style transfer and seasonal imagery correction in mapping products; generative molecule models now sit inside pharmaceutical screening pipelines at major research labs.

1.13 Course Boundaries: Questions From Students

1.13.1 Applications You Can Build After This Course

The closing stretch of the session belonged to open-floor questions. The first set asked what this course makes you capable of building.

Notice the pattern in that answer list: translation, summarization, captioning are all sequence-to-sequence or distribution-modeling jobs — the two pillars from section 2 wearing application clothing.

1.13.2 Code Generation and Software Engineering Questions

Both questions share one confusion point worth naming: conversion between modalities is taught generically here, while converting into engineering artifacts needs domain machinery the syllabus deliberately leaves out.

1.13.3 Three-Dimensional Design and Graph Neural Networks

That answer carries the single most important sentence of the session: representation first, then distribution learning. Every model family ahead assumes you can phrase your data as numbers or structures a network can ingest — once that bridge exists, the generative machinery applies unchanged.

1.13.4 Prerequisites and the NLP Course Distinction

1.13.5 Building Models Versus Fine-Tuning Pretrained Ones

Read that last answer as career advice layered on course logistics: learn how the models work here so that fine-tuning later is informed choice rather than guesswork.

Real-world connection: the questions themselves mirror industry demand — translation services, code assistants, and generative design shops all hire for exactly the boundaries discussed above, and knowing what a course does not teach is as valuable as knowing what it does.

Exam Guidance Summary

• Evaluation plan: quizzes carry 10 percent, with two quizzes scheduled — one before the mid-semester examination, one after, toward the semester's end. No makeup examinations will be conducted for any reasons. Calendar conflicts lose to this rule, so plan work travel around both quiz dates.
• Group assignments: two assignments, 10 percent each, heavy-duty programming exercises requiring the institute-provided computing infrastructure. Groups formed through the operations team; expect about five members per group. Assignment one lands before midterm on feature representation; assignment two follows midterm on generating data.
• Assignment shape: expect a dataset plus staged tasks: run various types of PCA, build classifiers on PCA features, construct a CNN-based autoencoder, extract its features, reconstruct images, judge reconstruction quality, then compare autoencoded-feature classifiers against PCA-feature classifiers. All coding uses scikit-learn plus libraries like Keras and TensorFlow. Treat the assignments as mini-projects: with five members, do not assume someone else carries the load — from an industry standpoint these assignments may outvalue months of ordinary coursework. The staged structure mirrors exactly what sections 4 and 5 of these notes preview, so finishing the readings early doubles as assignment preparation.
• Examinations: midterm plus comprehensive exam together carry 40 percent. Grading is strictly relative. An A might arrive at 70 marks or demand above 90, depending wholly on the cohort. Absolute marks mean nothing — only your position within the population counts; studying to a personal target score misses how the curve works.
• Reading policy: questions never come blanket-fashion from reading material not covered in class. Papers enter only when explicitly assigned — possibly one or two must-read papers anchoring an assignment topic. Read the linked blog posts and chapters attached to slides; the textbook's relevant chapters map directly onto course topics.
• Webinars and labs: four webinars run — two before mid-semester, two after — mixing lab sheets (helpful if Python experience is thin) with problem solving aimed squarely at exams. Formal lab topics: autoencoders, deep autoencoders, and convolutional autoencoders, typically in webinar one, with variational autoencoders and GANs following. Labs are not graded, and recordings exist if evening timing conflicts with work — but the problem-solving practice they rehearse is precisely what exam questions imitate.
• Prerequisites: only the machine learning and deep neural networks courses. No computer vision background needed beyond treating images as numeric matrices.
• Section policy: sections share identical slides and examinations; no section gains hidden advantages, so attend whichever fits.

Key Industry Applications

• Real-world: machine translation between languages — Hindi to English, English to Hindi, English to others — built with sequence-to-sequence generative models; the same machinery drives localization pipelines at global product companies.
• Real-world: document summarization systems — condensing contracts, filings, and news digests where human reading time is the bottleneck.
• Real-world: visual captioning — generating textual descriptions of images; plus cross-modal conversions generally (audio to text, text to image, image to text, audio to image), powering accessibility tools and media search.
• Real-world: ChatGPT-style assistants — fundamentally autoregressive models wrapped in transformers, pre-training, and heavy engineering.
• Real-world: Stable Diffusion and DALL-E — production diffusion models proving iterative denoising scales to stunning image synthesis for design, advertising, and entertainment workflows.
• Real-world: synthetic media across social platforms — much circulating content is already machine-generated, which is why detection skills pair with generation skills in this syllabus.
• Real-world: generative automobile design — MIT Technology Review documented designs optimized for aesthetics and aerodynamics, improving mileage.
• Real-world: molecule generation for computational biology, pharmaceuticals, and drug discovery, riding graph-structured representations.
• Real-world: CycleGAN-style domain translation — zebras from horses — with cycle-consistent losses; industrial cousins fix lighting and seasons in mapping imagery.
• Real-world: text style and sentiment editing via variational autoencoders — aggressive versus neutral versus submissive rewriting for customer-response drafting.
• Real-world: limited-domain GPT-like models for Indic languages, where linguistic principles diverge from European-language defaults — a realistic specialization niche for regional deployment teams.
• Real-world: agentic software maintenance — conversational agents operating on existing code bases — adjacent to this course, served by the software engineering specialization's generative AI courses.

Each entry above traces back to a model family previewed in this overview: representation tools (PCA, autoencoders) behind the feature-based systems, and the six generative families behind everything that creates new content. When you reach an industry question in interviews or project reviews, locating it on this list tells you which chapter of the course holds your answer.