Kubernetes Architecture and Deployment Strategies

14.1 From Docker to Kubernetes — Why Orchestration Is Needed

14.1.1 Docker Architecture Recap

Docker provides the foundational building block for containerized applications: the container engine (the runtime that creates, starts, and manages containers on a single machine). The Docker architecture has three tiers:

1. Client side — the CLI where you type commands like docker pull, docker build, and docker run. The client does not run containers itself; it sends instructions to the daemon.
2. Docker daemon (dockerd) — the background service on your machine that actually builds images, runs containers, manages networks, and handles storage. It listens for REST API requests from the client.
3. Docker Hub (or any container registry) — a central image registry where pre-built images are stored and versioned. Think of it as an app store for container images.

The workflow is straightforward:

1. Write a Dockerfile with layer-by-layer instructions (install OS, copy code, set entrypoint).
2. Build an image: docker build -t my-app:v1 .
3. Optionally push it to a registry: docker push my-app:v1
4. Anyone with a Docker engine can pull and run: docker pull my-app:v1 && docker run my-app:v1

This applies uniformly whether the application is a web app, a mobile backend, or an ML model serving predictions.

Limitations of Docker alone:

However, Docker alone is insufficient for running applications at scale across multiple nodes. When an application needs to serve hundreds of thousands or millions of users, a single container on a single machine cannot handle the load. Docker does not provide built-in mechanisms for:

• Distributing containers across machines — Docker runs containers on the machine where you execute the command. There is no built-in way to say "run this container on the machine with the most free resources."
• Load balancing traffic — If you manually run 5 copies of a container on 5 machines, Docker has no mechanism to spread incoming requests across them.
• Automatically restarting failed containers — While Docker can restart a single container with --restart=always, it cannot detect and recover from machine-level failures.
• Scaling instances based on demand — Docker cannot watch traffic metrics and add or remove containers in response.

This is where Kubernetes enters the picture. Kubernetes is a container orchestration tool — it automates the deployment, scaling, and management of containerized applications across clusters of machines.

14.1.2 Why Kubernetes Is Needed

The two technologies go hand in hand. Docker creates and runs individual containers; Kubernetes orchestrates those containers across many machines. The relationship is complementary, not competitive:

Every managed Kubernetes service in the cloud — AWS EKS (Elastic Kubernetes Service), Azure AKS (Azure Kubernetes Service), Google GKE (Google Kubernetes Engine) — uses the same underlying architecture. Understanding the architecture once means you can work with Kubernetes anywhere.

Real-world connection: When Instagram scaled from a small startup to serving hundreds of millions of photos, they moved from running Django on a few servers to containerized microservices orchestrated by Kubernetes. The same pattern applies to ML model serving — a single docker run works for development, but production ML inference at scale requires Kubernetes to manage dozens of model-serving replicas across multiple machines.

14.2 Kubernetes Architecture — Control Plane and Worker Nodes

Kubernetes follows a master-slave architecture (also called control plane and worker nodes). The overall structure is a three-tiered architecture:

graph TD
    A["Client Tier<br/>(kubectl CLI)"] --> B["Control Plane<br/>(Master Node)"]
    B --> C["Worker Node 1"]
    B --> D["Worker Node 2"]
    B --> E["Worker Node N"]
    B --> F["etcd<br/>(State Store)"]
    
    subgraph "Control Plane Components"
        B1["API Server"]
        B2["Scheduler"]
        B3["Controller Manager"]
    end
    
    subgraph "Worker Node Components"
        C1["Kubelet"]
        C2["Container Runtime"]
        C3["Pods"]
    end

14.2.1 The API Server

The API Server is the front door to the Kubernetes cluster. Every kubectl command is sent to the API Server first. It is essentially a REST API that listens for HTTP POST requests — the same pattern as running an inference server where you receive a request, process it, and return a response.

Commands that arrive at the API Server can be at multiple levels:

• Cluster level: Create an entire cluster, delete a cluster.
• Pod level: Create a new pod, delete pods, get the list of running pods.
• Container level: Inspect or manage individual containers.

Common operations map to HTTP methods:

The API Server processes these commands by coordinating with the other control plane components. It does not make decisions alone — it delegates to the Scheduler for placement decisions and to the Controller Manager for state enforcement.

14.2.2 etcd — The Cluster State Store

etcd (pronounced "et-see-dee") is a distributed key-value store that holds the current state of the entire cluster. Just as JSON is a key-value format used in everyday computing, etcd is a purpose-built distributed key-value database for Kubernetes. It stores information such as:

• How many pods are running
• Which pods are healthy and which are unhealthy
• Which containers are running on which nodes
• The current configuration of every resource (deployments, services, config maps)

When the API Server receives a get request (e.g., "how many pods are running?"), it queries etcd directly. etcd returns the current state, and the API Server sends the response back to the client. Every other control plane component — the Scheduler, the Controller Manager — also queries etcd whenever it needs data about the current state of the cluster.

Why distributed? etcd runs on multiple nodes (typically 3 or 5 for fault tolerance) and uses the Raft consensus algorithm to keep all copies in sync. If one etcd node fails, the others continue serving requests. This ensures the cluster state is never lost even if individual machines fail.

14.2.3 The Scheduler

When the API Server receives a command to create a new pod, it must decide which worker node should host that pod. This is the job of the Scheduler. Scheduling is not unique to Kubernetes — it is a fundamental problem in computing (operating systems schedule processes on CPUs, airlines schedule flights to gates, hospitals schedule surgeries to operating rooms).

The scheduling process works as follows:

1. A command arrives at the API Server: "create a new pod."
2. The API Server asks the Scheduler about the status of worker nodes.
3. The Scheduler checks resource availability — worker node 1 might be 80% full (memory and CPU), worker node 2 might be 40% full.
4. Based on its scheduling algorithm, the Scheduler suggests the API Server route the pod creation to the worker node with the most available capacity (e.g., worker node 2 at 40%).

The two primary resource constraints the Scheduler considers are memory and CPU. The Scheduler evaluates how much memory and CPU each worker node has available and assigns the pod accordingly. In practice, the default scheduling algorithm (called LeastRequested) prefers nodes with the most free resources, spreading the load evenly.

14.2.4 The Controller Manager

The Controller Manager enforces the desired state of the cluster. It continuously watches etcd for the current state and compares it to the desired state declared in your YAML manifests. If they differ, the Controller Manager takes action to reconcile them.

It has two sub-controllers:

Replica Set Controller: Determines how many instances (replicas) of a pod should be running. If the desired replica count is 1, only one pod runs. If the replica count is increased to 10, the Replica Set Controller ensures 10 pods are created. The default replica set is 1.

For example, consider a pod running a container. If the replica set is 1, one instance runs. If demand increases (say, on December 25th when traffic spikes), the replica set can be increased from 1 to 10 or 20. Each new pod runs its own container instance of the same image. If one pod can serve 100 users, 10 pods can serve 1,000 users.

Deployment Controller: Operates at a higher level of abstraction than the Replica Set Controller. When you issue kubectl create deployment, the command goes directly to the Deployment Controller, which then coordinates with the Replica Set Controller. The hierarchy is:

Deployment → Replica Set → Pod → Container

You cannot directly create a pod in Kubernetes — you always create a deployment, which internally manages replica sets and pods.

14.3 Pods — The Smallest Deployable Unit

A pod is the smallest deployable unit in Kubernetes. You cannot run a container directly within Kubernetes — containers must be wrapped in pods. A pod provides:

• A unique IP address (e.g., 10.10.10.1) for network access. Every pod in the cluster gets its own IP, so pods can communicate with each other directly.
• One or more containers running images.
• Optional volumes for data persistence.

14.3.1 Pod Composition Patterns

A pod can contain different combinations of containers and volumes:

The general industry guideline from AWS and Azure is one container per pod (one-to-one). The reasoning is practical: if a container fails within a one-to-one pod, the kubelet simply restarts that single container. In a multi-container pod, diagnosing which container failed and restarting only the failed one adds complexity.

However, there are valid exceptions. For a small application where a web server and a database serve very few users, placing both in a single pod makes sense — they scale together. When you increase the replica set, both the web application and database instances increase in tandem.

14.3.2 Volumes and Data Persistence

A volume in Kubernetes serves the same purpose as in Docker: it persists data across container restarts. Without a volume, whenever a container is stopped and restarted, all data within that container is lost. Volumes ensure that data survives across multiple container instances within the same pod, and across pod restarts.

14.3.3 Scaling with Pods

Scaling is straightforward: if you want 5 instances of an application, you create 5 pods. If one container serves 100 users, 5 pods serve 500 users. The Replica Set Controller manages this — it ensures the desired number of pod instances are always running. Behind the scenes, each pod runs its own container(s), each running the same image.

14.3.4 Pod Size and Abstraction

A pod is an abstraction layer — it has no inherent size. The pod's resource footprint is determined entirely by the containers it encapsulates. Pod, Replica Set, and Deployment are virtual layers of abstraction; the only physical entity actually running is the container (which reflects the image). The container's size — its memory and CPU requirements — determines the effective "size" of the pod.

14.4 The Worker Node and Kubelet — Self-Healing in Action

Each worker node contains a critical component called the kubelet (pronounced "kube-let"). The kubelet is the agent responsible for maintaining the health of all pods and containers within its worker node. It performs three key functions:

1. Container creation: When a new pod is scheduled to a worker node, the kubelet creates the containers within that pod by pulling the image and starting the runtime.
2. Container restart: If a container crashes or stops, the kubelet restarts it immediately. It does not wait for a human to intervene.
3. Health monitoring: The kubelet continuously monitors the state of every container, ensuring the desired number of healthy instances are running. It checks container status every few seconds.

14.4.1 Self-Healing at Two Levels

Self-healing operates at two levels in the Kubernetes architecture:

Pod level (managed by API Server and Scheduler): If a pod cannot be scheduled on a worker node (e.g., insufficient resources), it is rescheduled to another worker node. This is the cluster-level self-healing mechanism — the control plane detects that the desired state (N pods running) does not match the actual state (fewer than N pods running) and takes corrective action.

Container level (managed by kubelet): Within a worker node, if a container fails, the kubelet restarts it. This happens continuously — the kubelet never stops monitoring. If you specify that 2 pod instances should be running, the kubelet ensures exactly 2 instances are running at all times, even if one is stopped, deleted, or crashes.

14.4.2 Crash Loop BackOff

When a container repeatedly fails and is restarted, Kubernetes reports its status as CrashLoopBackOff. This means:

• The container is crashing (exiting with a non-zero exit code).
• Kubernetes is waiting before retrying — this is the "back off" period (exponential backoff: 10s, 20s, 40s, 80s, up to 5 minutes).
• The kubelet will attempt to restart it again after the backoff period.

14.4.3 The Docker Engine Dependency

Everything in Kubernetes runs on top of the Docker engine (or another container runtime like containerd). If the Docker engine itself fails on a worker node, all pods on that node will stop running. However, as long as the Docker engine is operational, the kubelet handles all container-level failures automatically.

14.5 Replica Sets and Scaling

The Replica Set is the mechanism for controlling how many pod instances run at any time. The default replica set is 1 — when you create a deployment, one pod is created by default.

14.5.1 Scaling Up

To handle increased load, the replica count can be increased. For example, changing the replica set from 1 to 10 creates 9 new pods (in addition to the existing one). Each pod runs its container independently. The kubelet on each worker node is responsible for physically creating these new containers.

The command to scale is kubectl edit deployment <deployment-name>. Opening the deployment YAML shows the current replica count. Changing the number and saving the file triggers the scaling — Kubernetes automatically creates or deletes pods to match the new desired count. No manual execution step is needed; the change is applied immediately.

kubectl get pods
NAME                     READY   STATUS              RESTARTS   AGE
iris-deploy-abc123       1/1     Running             0          5m
iris-deploy-def456       0/1     ContainerCreating   0          2s
iris-deploy-ghi789       0/1     Pending             0          2s
...

14.5.2 Scaling Down

Scaling down works the same way in reverse. If you change the replica count from 4 to 1, three pods enter the Terminating state and are removed. The kubelet handles the deletion.

14.5.3 Behind the Scenes

When you change the replica count, the Replica Set Controller issues the command (increase or decrease), and the kubelet on the worker node physically creates or destroys containers. The Replica Set Controller decides what to do; the kubelet does the actual work.

14.6 Deployments and YAML Manifests

A Deployment is the highest level of abstraction in Kubernetes. The hierarchy is:

graph TD
    A["Deployment<br/>(highest abstraction)"] --> B["Replica Set<br/>(how many instances)"]
    B --> C["Pod<br/>(networking + storage wrapper)"]
    C --> D["Container<br/>(runs the image)"]

When you issue kubectl create deployment, the command flows through the Deployment Controller, which creates a Replica Set, which in turn creates Pods, which contain Containers. This entire chain is triggered by a single command.

14.6.1 Two Ways to Create Deployments

Method 1 — Command line (imperative):

kubectl create deployment nginx-deploy --image=nginx

This creates a deployment named "nginx-deploy" using the nginx system image. The default replica set is 1.

Method 2 — YAML manifest file (declarative): Create a YAML file that declares the desired state, then apply it:

kubectl apply -f iris-deployment.yaml

The -f flag stands for "file" (not "force").

docker build -t iris-model:latest .

minikube image load iris-model:latest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: iris-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: iris
  template:
    metadata:
      labels:
        app: iris
    spec:
      containers:
      - name: iris-container
        image: iris-model:latest
        imagePullPolicy: Never

kubectl apply -f iris-deployment.yaml

kubectl get pods
# Should show 2 pods in Running state
kubectl get deployment
# Should show iris-deployment with 2/2 READY

14.6.2 The YAML Manifest Structure

A deployment YAML file (also called a manifest) has this structure:

14.6.3 YAML as a Configuration Language

YAML is a semi-structured language, sitting between structured data (Excel, CSV) and unstructured data (PPT, video). XML and JSON were historically the most popular semi-structured formats, but YAML has become the dominant language for configuration files, especially in Kubernetes and cloud-native tooling.

{"apiVersion": "apps/v1", "kind": "Deployment", "spec": {"replicas": 2}}

apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 2

The manifest file is written in YAML and describes the desired state of the cluster — what deployments should exist, how many replicas, which images, and so on.

14.6.4 kubectl apply vs kubectl create deployment

Both commands achieve the same result — creating a deployment — but they work differently:

14.7 Kube Proxy and Services — Making Pods Accessible

14.7.1 Static IPs, DNS, and Request Routing

All the components discussed so far — pods, replica sets, deployments — operate within the cluster. But the end goal is for users anywhere in the world to access the application. With so many layers of abstraction, how does an end user reach the right container?

The answer involves two components: Services and kube-proxy.

graph LR
    A["User Request"] --> B["Service<br/>(Static IP + DNS)"]
    B --> C["kube-proxy<br/>(Load Balancer)"]
    C --> D["Pod 1<br/>(Running)"]
    C --> E["Pod 2<br/>(Running)"]
    C --> F["Pod 3<br/>(Crashed)"]

The implication is important: all pod replicas are equal in nature. Any incoming request is routed to whichever pod is available. This means state should not be maintained within containers — if you need a database or event store, it should be in a separate container or external service. The application containers should be stateless.

14.8 Minikube — A Single-Node Cluster for Learning

14.8.1 Minikube Setup and Commands

Minikube is a tool that creates a single-node Kubernetes cluster — it includes both a control plane and one worker node on your local machine. Every command that works on a production Kubernetes cluster (AWS EKS, Azure AKS, Google GKE) also works on Minikube. It is designed for experimentation and learning purposes.

# Install Minikube (if not already installed)
# Then start the cluster
minikube start

minikube image load iris-model:latest

kubectl apply -f iris-deployment.yaml

kubectl get pods

NAME                              READY   STATUS    RESTARTS   AGE
iris-deployment-7b8f9d4c5-abc12   1/1     Running   0          30s
iris-deployment-7b8f9d4c5-def34   1/1     Running   0          30s

kubectl get deployment

NAME             READY   UP-TO-DATE   AVAILABLE   AGE
iris-deployment  2/2     2            2           45s

kubectl edit deployment iris-deployment

minikube dashboard

kubectl delete deployment iris-deployment

14.9 The Layers of Abstraction in Kubernetes

14.9.1 Container, Pod, Replica Set, Deployment

Kubernetes introduces four layers of abstraction, each building on the one below:

graph TD
    A["Deployment<br/>Manages versions and rollouts"] --> B["Replica Set<br/>Maintains N copies"]
    B --> C["Pod<br/>Networking + storage wrapper"]
    C --> D["Container<br/>Runs the image"]
    
    style A fill:#e1f5fe
    style B fill:#f3e5f5
    style C fill:#e8f5e8
    style D fill:#fff3e0

The key insight: you cannot directly create a pod in Kubernetes. There is no kubectl create pod command. You always create a deployment (kubectl create deployment), and the entire chain — Deployment → Replica Set → Pod → Container — is created automatically.

14.10 Current State vs. Desired State — The Self-Healing Loop

14.10.1 The Reconciliation Cycle

The fundamental operating principle of Kubernetes is the continuous reconciliation of two states:

Every Kubernetes component — the Scheduler, the kubelet, the Controller Manager — works continuously to match the current state to the desired state. If the desired state says "5 replicas" but only 3 are running, Kubernetes creates 2 more. If 7 are running but only 5 are desired, Kubernetes deletes 2.

graph TD
    A["YAML Manifest<br/>(Desired State)"] --> B["Control Plane<br/>(Comparison)"]
    C["etcd<br/>(Current State)"] --> B
    B -->|"Current < Desired"| D["Create Pods"]
    B -->|"Current > Desired"| E["Delete Pods"]
    B -->|"Current = Desired"| F["No Action"]
    D --> C
    E --> C

This reconciliation happens continuously. The source of the desired state is the manifest file (YAML), which is typically stored in a Git repository. A CD pipeline (Continuous Deployment tool) — such as Argo CD, Jenkins, or Flux CD — pulls changes from Git and syncs them with the cluster. The default sync interval is 3 minutes (180 seconds), though it can be configured to be nearly continuous.

The sync cycle works as follows:

1. The CD pipeline pulls the latest manifest file from Git.
2. It compares the desired state (from the manifest) with the current state (from etcd).
3. It calculates the differences.
4. The self-healing mechanism (kubelet, Scheduler, Controller Manager) creates, updates, or deletes pods to reconcile the differences.

This is the "big picture" of how changes flow from a developer's YAML file to running containers in production.

14.11 Deployment Strategies

Whenever an application is deployed to production, the goal is zero downtime deployment — end users should not experience any interruption when switching from version 1.0 to version 2.0. There are four standard deployment strategies:

14.11.1 Basic Deployment

In a basic deployment, all nodes are updated simultaneously. If there are 12 nodes running version 1.0, at the switch moment (e.g., 12:02 PM), all 12 nodes are replaced with version 2.0 at the same time.

graph LR
    subgraph "Before (12:01 PM)"
        A1["Node 1: V1"] --- A2["Node 2: V1"] --- A3["Node 3: V1"]
    end
    subgraph "After (12:02 PM)"
        B1["Node 1: V2"] --- B2["Node 2: V2"] --- B3["Node 3: V2"]
    end
    A1 -->|"All switch at once"| B1

Success scenario: Version 2.0 works perfectly, and all users seamlessly transition.

Failure scenario: Something goes wrong, and you must roll back (revert) to version 1.0 across all nodes. During the rollback, all users experience errors.

This is an "all or nothing" approach. It is suitable for small applications or services where the risk of simultaneous failure is acceptable and rollback is straightforward.

14.11.2 Ramped (Rolling Update / Incremental) Deployment

The ramped strategy — also called rolling update or incremental — is the default in Kubernetes. Instead of updating all nodes at once, the new version is introduced one node at a time:

graph TD
    subgraph "Step 1: All V1"
        A1["Node 1: V1"] --- A2["Node 2: V1"] --- A3["Node 3: V1"] --- A4["Node 4: V1"]
    end
    subgraph "Step 2: 1 node updated"
        B1["Node 1: V2"] --- B2["Node 2: V1"] --- B3["Node 3: V1"] --- B4["Node 4: V1"]
    end
    subgraph "Step 3: 2 nodes updated"
        C1["Node 1: V2"] --- C2["Node 2: V2"] --- C3["Node 3: V1"] --- C4["Node 4: V1"]
    end
    subgraph "Step 4: All V2"
        D1["Node 1: V2"] --- D2["Node 2: V2"] --- D3["Node 3: V2"] --- D4["Node 4: V2"]
    end
    A1 --> B1 --> C1 --> D1

1. Initially, all 4 nodes run V1.
2. V2 is introduced to 1 node — 3 nodes still run V1, 1 runs V2.
3. After a configurable interval (seconds, minutes), V2 spreads to 2 nodes.
4. Then 3 nodes, then all 4 nodes run V2.

The interval between updates is configurable — it can be 5 seconds, 10 seconds, one minute, or any duration. The key characteristic: at any point during the rollout, some users are on V1 and some are on V2. This creates temporary inconsistent behavior, but it is not erroneous — users simply see different versions until the rollout completes.

14.11.3 Canary Deployment

The canary strategy releases a new version to a small subset of users first, observes the results, and then rolls out to everyone if everything goes well.

Canary deployments can target users by:

• Geography: Only users in India get the feature first.
• User segments: Identified through clustering and analytics — groups of users who are most likely to respond well to the feature.
• Percentage: Typically 10–25% of users receive the new version initially.

The selection of which users get the canary version is often non-technical — it is driven by marketing analytics, user clustering, and business strategy. Data scientists and analysts identify the target segments before the rollout decision.

Multiple versions simultaneously: Canary deployment supports testing multiple versions at once. V2.1 can be released to one subset, V2.2 to another subset, and V2.3 to yet another. After observing which version performs best, a full rollout of the winning version is executed.

Gradual rollout: Canary deployments often follow a graduated pattern — 1% first, then 5%, then 10%, then 20%, and so on — increasing the percentage as confidence grows.

A/B testing connection: Canary deployment is closely related to A/B testing. When you have two versions (A and B) and want to determine which performs better, you can use canary deployment to release each version to a subset of users and measure the results. The version with better metrics gets the full rollout. In day-to-day industry practice, canary and A/B testing are often used interchangeably, though there are subtle differences — canary focuses on safe rollout, while A/B testing focuses on measurement and comparison.

14.11.4 Blue-Green Deployment

Blue-green maintains two complete, identical environments:

• Blue (Live): The current production environment running V1.1.
• Green (Staging): An exact replica of the production environment running V1.2.

graph TD
    A["Users"] -->|"100% traffic"| B["Blue Environment<br/>(V1.1 — Live)"]
    C["Green Environment<br/>(V1.2 — Staging)"]
    A -.->|"Switch after validation"| C

The staging environment must have the same number of nodes as the live environment. Users access the live (blue) environment. When the green environment is ready and validated, the users are switched from blue to green in one operation — not gradually, but all at once.

Advantages:

• Rollback is straightforward — if the green environment fails, redirect users back to the blue environment.
• Risk is reduced compared to other strategies because the new version is fully tested in an identical environment before the switch.
• Easy to implement.

Disadvantages:

• Cost and complexity: Maintaining two complete production environments is expensive. For microservices architectures with hundreds or thousands of services, replicating the entire environment doubles infrastructure costs.
• Resource-intensive — you are paying for two full environments simultaneously.

Quality assurance considerations: The staging environment can be validated using either production data or dummy data. For new applications without much production data, dummy data is used. For established applications with months or years of production data, the staging environment can be tested against real production data. Smoke tests are performed on the staging environment before the switch to ensure the new version is functioning correctly.

The switch is one-time: Users are redirected from one environment to the other in a single operation. It is not gradual like a rolling update.

14.12 Critical Services and High Availability

14.12.1 Three Principles for Zero Downtime

When designing Kubernetes deployments for critical services that require zero downtime, three principles apply:

14.13 Stateful vs. Stateless in Kubernetes

14.13.1 Stateless Design and StatefulSets

Kubernetes is designed to be stateless by default. Containers should not maintain application state internally. State — such as database contents, session data, or event queues — should be decoupled from the application containers and stored in separate, dedicated containers or external services.

This design is critical because:

• All pod replicas are treated equally by the Service and kube-proxy.
• Any request can be routed to any replica.
• If a pod dies, its replacement has no knowledge of previous requests.

For stateful requirements (databases, persistent storage), Kubernetes offers StatefulSets — a different resource type with specialized configurations for maintaining identity, stable network names, and persistent storage across pod restarts. However, maintaining stateful systems is significantly more expensive and complex than stateless ones.

Real-world consideration: In industries like banking, sessions cannot simply be killed during deployments. Banks typically schedule downtime windows (often at night) to perform deployments, blocking new sessions and waiting for active sessions to complete before shutting down. They use a "tail" strategy — stop accepting new sessions, wait a buffer period (e.g., 30 minutes) for active sessions to finish, then perform the deployment.

For stateless APIs, broken sessions are acceptable — the kubelet spawns a new pod, fresh sessions start, and whatever was lost is lost. But for databases and financial transactions, stateful configurations with careful deployment strategies are required.

14.14 Auto-Scaling

14.14.1 How Auto-Scaling Works

Auto-scaling is an independent component (provided by cloud platforms like AWS, Azure) that ties into monitoring systems (e.g., AWS CloudWatch). It continuously monitors metrics like the number of concurrent users per second and automatically adjusts the replica set based on configurable rules.

# Horizontal Pod Autoscaler (HPA) rule
minReplicas: 1
maxReplicas: 20
metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Example rule: "If concurrent users per second increases from 1 million to 10 million, create 3 additional replicas." Conversely: "If users decrease, remove the extra pods." The default is always 1 pod — additional pods are created only when demand increases and removed when demand decreases.

Auto-scaling is not managed by the kubelet. The kubelet handles container health; auto-scaling is a higher-level orchestration driven by monitoring data and configurable rules.

Exam Guidance Summary

High-priority topics:

1. Kubernetes architecture — Understand all four control plane components (API Server, Scheduler, Controller Manager, etcd) and what each does. Know the three-tier architecture (client, control plane, worker nodes).

1. Abstraction hierarchy — Deployment → Replica Set → Pod → Container. Know that you cannot create a pod directly — only a deployment. Understand that the container is the only physical entity.

1. Self-healing loop — Current state (etcd) vs. desired state (YAML manifest). How the reconciliation cycle works. What happens when a pod crashes (kubelet restarts) vs. when a node fails (control plane reschedules).

1. Deployment strategies — Be familiar with all four: Basic, Ramped (rolling update), Canary, Blue-Green. Know when each is appropriate. Canary and blue-green are the most commonly discussed in industry.

1. YAML manifests — Understand the structure: apiVersion, kind, replicas, containers. Know the difference between kubectl create deployment (imperative) and kubectl apply -f (declarative).

1. Practical commands — kubectl create deployment, kubectl get pods, kubectl get deployment, kubectl edit deployment, kubectl apply -f, kubectl delete deployment.

1. Stateless vs. Stateful — Understand why K8s favors stateless design. Know what StatefulSets are and when to use them. The banking "tail" strategy is a good exam example.

Key Industry Applications

These are the real-world tools and platforms that implement the concepts covered in this lecture. Understanding how they connect to the architecture helps you apply this knowledge in practice.

Cloud Kubernetes Services:

• AWS EKS (Elastic Kubernetes Service): Amazon's managed Kubernetes. Uses the same architecture (control plane + worker nodes), but AWS manages the control plane for you.
• Azure AKS (Azure Kubernetes Service): Microsoft's managed Kubernetes. Same architecture, same kubectl commands.
• Google GKE (Google Kubernetes Engine): Google's managed Kubernetes (Google created Kubernetes originally). Same architecture.

Monitoring and Observability:

• Prometheus: An observability/monitoring tool that runs as a container in a pod. It continuously scrapes metrics from application pods on specific ports and stores them as time-series data. Used with Grafana for dashboards.
• AWS CloudWatch: Monitoring tool used for auto-scaling decisions. Tracks CPU, memory, and custom metrics across your cluster.

Event-Driven Architecture:

• Kafka: A distributed message broker, containerized and running in pods. Used for event-driven architectures where microservices communicate through events rather than direct API calls.

Continuous Deployment:

• Argo CD: A GitOps CD tool that pulls manifest changes from Git and syncs them with the cluster. Default sync interval: 3 minutes.
• Jenkins: A general-purpose CI/CD tool that can be configured to deploy to Kubernetes.
• Flux CD: Another GitOps CD tool, lighter weight than Argo CD.

Real-World Deployment Examples:

• WhatsApp Meta AI rollout: A classic canary deployment — the Meta AI feature appeared on some devices first, then gradually expanded to all devices over about a week.
• Banking sector: Uses scheduled downtime windows for deployments due to stateful session requirements. Sessions are drained before deployment using the "tail" strategy — stop accepting new sessions, wait for active sessions to complete, then deploy.