Docker and Kubernetes for ML Deployment

# Docker and Kubernetes for ML Deployment

13.1 Software Deployment Evolution

Software deployment is the process of making an application available for use in a production environment. Unless a machine-learning model is deployed, end users receive no benefit from it. Across the industry today — for both traditional software engineering applications and machine-learning systems — the de facto standard for deployment relies on two technologies: Docker for containerization and Kubernetes for orchestration. To understand why these tools exist, it helps to see the problems that each earlier era left unsolved.

13.1.1 Traditional Deployment

In the 1970s and 1980s, organisations invested in physical servers. The model was straightforward: buy hardware, install an operating system (Windows, Linux, or another OS), write the application, and host it directly on that physical machine. Every application ran on its own dedicated server.

The one-application-per-server rule carried a visible cost. A typical web or database server spends most of its time idle — CPU utilisation of 5–15% was common — because workloads rarely sit at their peak. Yet the hardware was bought for the peak: you paid for a machine sized for the worst moment, and it sat mostly unused. Scaling meant buying another physical box, which took weeks (ordering, shipping, racking, installing an OS) and usually meant provisioning for future peak demand rather than current needs.

The era's advantage was simplicity and predictability: one server, one OS, one application, and a very clear mental model of where everything runs. Its disadvantages were cost, wasted hardware, slow scaling, and poor resource utilisation.

13.1.2 Virtualized Deployment

When cloud computing became popular in the early 2000s, virtual machines (VMs) changed the equation. A hypervisor — a software layer installed on top of the host operating system — lets you create multiple isolated virtual machines on a single physical system. Each VM gets its own guest operating system, which can differ from the host. For example, a Windows 11 host can run Ubuntu and Linux guest VMs simultaneously through a hypervisor such as VirtualBox, VMware, or Hyper-V.

Think of the hypervisor as a property manager for a building. The building (the physical server) is one piece of hardware. The property manager divides it into flats (VMs). Each flat has its own locks, plumbing, and electricity — its own guest OS and applications — so tenants cannot walk into each other's flats even though they share one building. If one tenant crashes their application, installs software, or reboots their OS, the other tenants are not disturbed.

In the cloud world, virtual machines are a core service. On AWS they are called EC2 instances. When launching an EC2 instance, the system asks you to choose an Amazon Machine Image (AMI) — essentially the guest operating system you want. On top of that guest OS, you install your runtime and your application.

The key advantage: one physical machine hosts many VMs, so hardware is better utilised. The key disadvantage: every VM carries a full guest operating system, which is heavy on resources. Each guest OS needs its own CPU, RAM, disk, and licences — a Linux guest can consume 500 MB to 1 GB of RAM before your application even starts. Spin up fifty VMs and you are running fifty full operating systems, most of them doing nothing.

13.1.3 Container-Based Deployment

Containers eliminate the guest operating system entirely. Whatever the host OS is, that is what the container uses. You install a container engine — the most popular being Docker — on the host OS, and then you run your applications inside containers on top of that engine. No guest OS layer sits between the engine and the application. This makes containers significantly lighter than VMs.

A container is not a new machine; it is an isolated process (or set of processes) that shares the host OS kernel. Because there is no guest OS to boot, containers start in seconds rather than minutes, and dozens can run on hardware that would strain under a handful of VMs. You get the isolation benefit that motivated VMs — separated applications with their own files and libraries — without paying for a full OS per application.

Real-world: On AWS, the VM counterpart is EC2; the container counterpart is AWS Lambda. When you create a Lambda function, the system asks for your runtime (Python, Java, .NET, Ruby, etc.) but does not ask for an operating system — it takes a default Linux image automatically. Behind the scenes, Lambda runs on containers.

13.1.4 VMs versus Containers — Summary

Visual: Picture a vertical stack for each option. The VM stack runs: hardware → hypervisor → guest OS → application. The container stack runs: hardware → host OS → container engine → application. The container stack has one fewer layer — and that missing layer is exactly the weight, boot time, and cost that containers save. A VM is a "computer inside a computer"; a container is a "process with its own isolated view of the world."

Visual takeaway: The single landmark on both stacks is the layer between hardware and application. Where the VM inserts a hypervisor plus a guest OS, the container inserts only an engine. One-sentence takeaway: containers remove the duplicated operating system, and everything else follows — speed, weight, portability.

Real-world: The eras describe layers you still choose between today. Engineers pick EC2 (VMs) when they need full OS control, Lambda (containers under the hood) when they want zero server management, and EKS or ECS (containers at scale) when they need orchestration. The container era's core insight — package everything, ship once, run anywhere — is the theme of the rest of this lecture.

13.2 Containerization Fundamentals

The word "container" in software comes from manufacturing and logistics. A physical shipping container is a standard-sized metal box. You can put anything inside it — vehicles, furniture, refrigerators — and then load it onto a truck or ship. The container gets sealed and shipped from point A to point B. What is inside does not matter; the container format is universal. A software container works the same way: you bundle everything your application needs into a single package and ship it between environments.

13.2.1 Environment Parity Problem

A common pain point in software engineering is the inability to replicate a production issue in the development environment. A critical bug appears in production but when the developer tries to reproduce it locally, the issue does not manifest. One major reason is environment discrepancies — different library versions, OS patches, configuration settings, or runtime versions. Containers solve this by packaging the entire environment so that code, binaries, runtime, libraries, and configuration are identical across all environments.

The two analogies reinforce each other. The singer analogy explains why containers exist (environment parity — eliminating venue vagaries). The shipping-container analogy explains how they achieve it (standard package format that any host can carry). Both break down only at the edge: a physical container is fully sealed and tamper-proof, while a software container still depends on the host OS kernel — you cannot ship a Windows container to a Linux-only host.

13.2.2 What Goes Inside a Container

For a software engineering application, a container bundles:

• Source code — the application itself
• Runtime — Python 3.10/3.11, Java JVM, .NET CLR
• System libraries — JAR files, DLLs, pip packages
• Configuration settings — database connection strings, server ports

For a machine-learning application, the container additionally includes:

• The trained model — a pickle or ONNX file
• Inference code — a FastAPI or Flask server
• ML dependencies — scikit-learn, NumPy, pandas, joblib
• Configuration files — YAML or JSON with model parameters, feature names, thresholds

Everything and anything needed for the software to run is inside the container.

Real-world: Every major cloud provider builds on this idea: AWS Lambda, Azure Functions, and Google Cloud Run all accept a container and run it for you, and Kubernetes (13.11) orchestrates thousands of containers in production. The container is the unit of shipping for the entire industry — for ML systems, it is the unit that carries a model into production intact, which the reference book (Deploying a Model) describes as the standard way to package a model inference service for deployment.

13.3 Docker

Docker is the most popular container engine. It is a piece of software you install on any operating system — Windows, Mac, or Linux. Once Docker is installed on a machine, that machine becomes a Docker host. On that host, you can pull images from a registry, build your own images, and run containers.

The key property: a Docker image can run on any machine that has Docker installed, regardless of the underlying operating system. As long as Docker is there, your containerised application will run.

13.3.1 Docker Architecture

Docker uses a three-tier architecture: Client, Docker Host (Docker Daemon), and Registry.

Client. This is where you type commands. The Docker client runs in your terminal or command prompt. Commands like docker build, docker pull, and docker run are all client commands. The client sends these commands to the Docker daemon.

Docker Host (Docker Daemon). Any system where Docker is installed becomes a Docker host. The Docker daemon runs on this host and manages two core objects: images and containers. When the client sends a command, the daemon executes it — building images, starting containers, stopping containers, and so on. The client is only a messenger; all the real work happens in the daemon.

Registry. The registry stores Docker images. Docker Hub is the default public registry, analogous to GitHub for source code. Just as GitHub lets you git push and git pull code, Docker Hub lets you docker push and docker pull images. Other registries exist: AWS has Elastic Container Registry (ECR), Azure has Azure Container Registry, and Google Cloud has its own container registry.

13.3.2 Command Flow Through the Architecture

When you run docker pull ubuntu from the client, the command flows: client → Docker daemon → checks local images → if not found, connects to Docker Hub → downloads image → stores on Docker host. When you run docker run ubuntu, the daemon creates a container from the image and starts it. If the image is not local, it pulls first.

Exam note: Understanding the flow from client → daemon → registry is fundamental. Expect conceptual questions on this three-tier architecture — including which component does what during docker pull, docker build, and docker run.

13.3.3 System Images versus Custom Images

Docker Hub contains two kinds of images. System images (official images) are pre-built and freely available for virtually any software: operating systems (Ubuntu, Alpine), databases (PostgreSQL, MongoDB, MySQL), web servers (Nginx, Apache), programming language runtimes (Python, Java, Node.js), and ML frameworks. Custom images are images you build yourself on top of system images. You push them to Docker Hub under your account (e.g., 6945/iris-model:latest). Anyone can pull and run them.

A custom image internally uses one or more system images as its foundation. When you write FROM python:3.11 in a Dockerfile, you are pulling the official Python 3.11 system image and layering your application on top.

Real-world: Docker Hub is free for public images, just like GitHub. You log in through GitHub or Gmail credentials. Account IDs are numeric (e.g., 6945), and images are pushed under <account-id>/<image-name>:<tag>.

13.4 Docker Objects: Image, Container, Volume

Docker has three core objects that every practitioner must understand.

13.4.1 Image

An image is a static, read-only blueprint that contains everything needed to create a running container: the base OS or runtime, application code, libraries, configuration, and instructions for what to execute. Think of an image as a class in object-oriented programming — it defines the structure and behaviour but is not itself running. When you run docker build, Docker executes the Dockerfile instructions layer by layer and produces an image. Docker caches these layers for performance.

13.4.2 Container

A container is a running instance of an image. When you execute docker run <image-name>, Docker creates a container from the image and starts it. The relationship between image and container is the same as between a class and an object in OOP: one image can produce many containers, each running independently.

When you run docker run -it ubuntu:latest (the -it flag means interactive terminal), Docker creates a container and drops you into a shell. You can run Linux commands, install software, and create files. However, any changes you make inside a container are ephemeral — if you exit and restart, everything is gone.

13.4.3 Volume

A volume provides persistent storage for containers. Since container filesystems are ephemeral — everything inside disappears when the container stops — volumes let you store data outside the container lifecycle. When a container restarts, the volume is mounted back, and the data is available again.

Think of the container's writable layer as a whiteboard that gets wiped whenever the container stops, and the volume as a notebook that sits beside the whiteboard: you copy important things into the notebook, and when the whiteboard is wiped and replaced (a new container from the same image), the notebook is simply placed beside the new board.

Real-world: In the BITS virtual lab environment, each student gets a containerised workspace. When a student saves a Python file and exits, the file persists because the data is stored in an AWS EBS volume. When the student logs in again, the volume is mounted, and the saved file is still there. This is volume persistence in production: container is ephemeral, volume is durable.

13.5 Docker Commands and Workflow

13.5.1 Essential Commands

The three most important commands map to the three-tier architecture:

• docker pull <image>:<tag> — download an image from a registry
• docker build -t <name>:<tag> . — build a custom image from a Dockerfile
• docker run [options] <image>:<tag> — create and start a container from an image

Additional commands: docker login (authenticate with registry), docker tag (create tag for pushing), docker push (upload image), docker ps (list running containers), docker stop (stop container), docker rm (remove container).

13.5.2 Demonstrated Workflow: Pulling and Running Ubuntu

The following sequence was demonstrated live:

13.6 Port Mapping

When you run a web application inside a container (say, on port 8000), you need to map the container's port to a port on your local machine. Without this mapping, the application is not accessible from your browser.

The -p flag handles this: docker run -p 8000:8000 <image>:<tag>. The format is -p <host-port>:<container-port>. If another application is already using port 8000 on your host, you will get a conflict — use a different host port (e.g., -p 9000:8000).

Common port conventions: web applications use port 80, 8080, or 8000; PostgreSQL uses port 5432; MySQL uses port 3306. Any unused port can be used.

13.6.1 Port Forwarding versus Kubernetes Services

Port mapping as demonstrated is called port forwarding — from the container port to the local port. In a Kubernetes deployment, port mapping is typically handled through a service definition (a YAML file) rather than direct port forwarding. This distinction will be covered in the next lecture.

13.7 Dockerfile and Building Custom Images

13.7.1 What is a Dockerfile

A Dockerfile is a plain text file containing step-by-step instructions for building a Docker image. It is not a script for your local machine — every instruction executes inside the container being built. The Dockerfile tells Docker: start with this base image, copy these files, install these dependencies, set this working directory, and run this command when the container starts. When you run docker build, Docker reads the Dockerfile from top to bottom, executes each instruction as a layer, and produces an image.

13.7.2 Dockerfile for a Python REST Application

A Python FastAPI/REST application typically has this Dockerfile structure:

FROM python:3.9
WORKDIR /code
COPY requirements.txt /code/requirements.txt
RUN pip install -r /code/requirements.txt
COPY . /code
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Line by line: FROM python:3.9 sets the base image (you do not need to specify an OS; the Python image includes one). WORKDIR /code creates and sets the working directory inside the container. COPY requirements.txt copies the requirements file. RUN pip install installs dependencies inside the container. COPY . /code copies all source code. EXPOSE 8000 declares the port. CMD sets the startup command (Uvicorn server).

The order of instructions matters. The logical sequence is: set base → set working directory → copy and install dependencies → copy source code → declare port → set startup command. Dependencies must be installed before copying source code so that Docker can cache the dependency layer — if only source code changes, dependencies are not re-installed.

13.7.3 Available Dockerfile Keywords

Dockerfiles support many keywords: FROM, WORKDIR, COPY, RUN, EXPOSE, CMD, ENV, ARG, ENTRYPOINT, ADD, VOLUME, USER, LABEL, HEALTHCHECK, SHELL, and others. The subset shown above covers the majority of real-world use cases.

13.7.4 Custom Image as Base

Instead of starting FROM python:3.11, you can start FROM <account>/<custom-image>:<tag>. This lets you layer one custom image on top of another. If your organisation maintains a base image with pre-installed security tools and monitoring agents, every team starts FROM that base image and adds their application-specific layers on top.

Real-world — professor's experience: Dockerising applications transformed training sessions. Previously, every participant's machine needed the software installed individually — a day lost to environment setup. Now the professor hands out a Docker image; participants pull and run it, and everyone starts from the identical environment in minutes. The same pattern is what makes ML demos reproducible in industry: the image carries the model, the dependencies, and the server, so "it works on my machine" disappears from the vocabulary.

13.8 Build, Run, Push — End-to-End Workflow

13.8.1 Building the REST Application Image

After creating the Dockerfile, the build command is:

docker build -t rest-app:latest .

-t rest-app:latest names the image; the . tells Docker to look for the Dockerfile in the current directory. Docker executes the Dockerfile layer by layer: FROM (pull Python image, cached if present), WORKDIR (create directory), COPY requirements, RUN pip install (longest step on first build), COPY source, EXPOSE port, CMD startup. After the build, the image appears in Docker Desktop under "Images." No container is created yet — the image is static.

13.8.2 Running the Container

docker run -p 8000:8000 rest-app:latest

This creates a container from the image, maps port 8000, and starts the Uvicorn server. The application is now accessible at http://localhost:8000.

13.8.3 Pushing to Docker Hub

To share the image or deploy it to a server:

1. docker login — authenticate with Docker Hub
2. docker tag rest-app:latest 6945/rest-app:latest — create a tag with your Docker Hub account ID
3. docker push 6945/rest-app:latest — upload the image to Docker Hub

After pushing, anyone can docker pull 6945/rest-app:latest and run the application.

Docker Hub visibility: Like GitHub, Docker Hub supports public and private repositories. Cloud registries (AWS ECR, Azure Container Registry) similarly support public and private images.

13.9 Containerizing a Machine-Learning Application

13.9.1 Application Structure

A typical ML application for containerisation contains: train.py (training script), model.pkl (trained model file), app.py (inference server with FastAPI), preprocess.py (feature preprocessing), postprocess.py (mapping predictions to labels), requirements.txt (Python dependencies), config.yaml (configuration), and Dockerfile.

13.9.2 Iris Classification Example

The demonstrated ML application uses the Iris dataset with a Random Forest classifier. The training script loads the Iris data, trains the model, and saves it as model.pkl. The inference server loads model.pkl at startup and exposes GET / (health check) and POST /predict (accepts four features: sepal length, sepal width, petal length, petal width). Post-processing maps prediction integers to class names: 0 → Setosa, 1 → Versicolor, 2 → Virginica.

13.9.3 Dockerfile for the ML Application

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
COPY . /app
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

The structure is identical to the REST application. The requirements.txt contains fastapi, uvicorn, scikit-learn, numpy, joblib, pydantic. Build and run:

docker build -t iris-model:latest .
docker run -p 8000:8000 iris-model:latest

After the container starts, a POST request to http://localhost:8000/predict with the four Iris features returns the prediction (e.g., prediction: 0, label: Setosa).

Real-world: Any application built during this course — basic ML models, agentic AI applications, RAG applications — can be dockerised using the same pattern. Create a Dockerfile, set up the environment, give the instructions, and set the startup command. This is the standard way that model inference services are packaged for deployment in industry: model file, inference code, dependencies, and configuration sealed into one portable image.

13.10 CI/CD Pipeline with Docker and Kubernetes

In industry, the end-to-end deployment lifecycle follows this pipeline:

1. Source code — developers write code and commit to a Git repository.
2. CI pipeline (GitHub Actions, Jenkins) — automated build, test, and Docker image creation. If tests pass, the image is pushed to a container registry (Docker Hub, AWS ECR, Azure Container Registry).
3. Container registry — the built image is stored with a version tag (e.g., image:1.0).
4. CD pipeline (Argo CD, Flux CD, Jenkins CD) — pulls the image and deploys to a Kubernetes cluster using a deployment manifest (YAML file specifying replicas, image version, resource limits).
5. Kubernetes cluster — runs the containers. If the image is updated (e.g., image:1.0 → image:1.1), the CD pipeline rolling-updates the pods.

13.10.1 Scaling in Production

Different microservices scale independently based on demand. Consider an OTT platform: the recommendation microservice may need only one replica when nobody is browsing, while the play microservice may need 1000 replicas when millions are streaming. Kubernetes handles this through its scaling mechanisms (Horizontal Pod Autoscaler, discussed in the next lecture).

13.11 Introduction to Kubernetes

13.11.1 Why Kubernetes

Docker alone is sufficient for running an application on a single machine. But for large-scale deployment — serving millions or billions of users across distributed infrastructure — you need container orchestration. Kubernetes (K8s) is that orchestration platform. Originally developed by Google Labs and donated to the Cloud Native Computing Foundation (CNCF) in 2015–2016, it was the first graduated project of CNCF.

Real-world: Almost every major application today — including ChatGPT — runs on Kubernetes. It provides high availability (no downtime), scalability (handle traffic spikes), high performance, and disaster recovery (backup and restore).

13.11.2 CNCF Graduated Projects

CNCF has 38 graduated projects and 38 incubating projects, all open source. Notable graduated projects: Kubernetes (container orchestration), Argo CD (GitOps CD), Flux CD (alternative CD), Helm (Kubernetes package manager), Prometheus (monitoring), Envoy (service proxy), Istio (service mesh), Kubeflow (ML workflows), OpenTelemetry (observability), etcd (distributed key-value store), gRPC (RPC framework). Incubating projects include Chaos Mesh, Litmus, OpenFeature, Thanos, Volcano.

Real-world: Helm, Prometheus, Argo CD, and Istio are among the most widely adopted CNCF tools. Several appear again in this course: Argo CD and Flux CD in the CI/CD pipeline (13.10), Prometheus for monitoring in production, and etcd as the Kubernetes control plane's state store (13.12).

13.12 Kubernetes Architecture

13.12.1 Master-Slave Pattern

Kubernetes follows a master-slave (control plane – worker node) architecture — the same pattern used by Apache Hadoop, OpenStack, and many job schedulers. A single machine cannot handle all the work; distributed compute across many machines is necessary.

Master node (control plane) receives all input and directs work to worker nodes. It contains:

• API Server — the front door for all kubectl commands
• Scheduler — decides which worker node runs a new pod
• Controller Manager — ensures desired state matches actual state
• etcd — stores cluster state

Worker nodes are machines (physical or virtual — EC2 instances, laptops with Minikube, bare-metal servers). Each contains:

• Docker engine — every worker node must have Docker installed
• Pod — smallest deployable unit
• Kubelet — agent communicating with master node
• Kube Proxy — networking and traffic routing

If you have 1000 worker nodes, all 1000 must have Docker installed. The Docker engine lives exclusively in worker nodes, not in the master node.

13.12.2 Pod

A pod is the smallest deployable unit in Kubernetes. You do not run containers directly; you run pods that contain containers. A pod can contain one container (most common — 1:1 mapping) or multiple tightly-coupled containers sharing the same network namespace and storage volumes.

Exam note: Know that a pod is the smallest deployable unit, not a container.

13.12.3 Managed Kubernetes Services

With managed services, the cloud provider manages the master node; you manage worker nodes. Version upgrades are user-driven — the ops team decides when to upgrade.

The alternative — Amazon ECS: AWS offers Elastic Container Service (ECS), its own orchestration (not Kubernetes-based). For ECS, upgrades are automatic because AWS fully controls the platform. Trade-off: EKS gives portability across clouds; ECS locks you into AWS.

13.12.4 Kubernetes Features

• High availability — no downtime. If a pod crashes, Kubernetes restarts it automatically.
• Scalability — handle traffic spikes by adding more pod replicas.
• High performance — efficient resource utilisation across the cluster.
• Disaster recovery — backup and restore capabilities.

13.13 kubectl

kubectl is the command-line tool for communicating with the Kubernetes master node. All Kubernetes commands start with kubectl: kubectl get pods (list pods), kubectl apply -f deployment.yaml (deploy application), kubectl scale deployment my-app --replicas=5 (scale replicas). The commands are identical whether running on Minikube (local), EKS (AWS), GKE (Google), or AKS (Azure).

13.13.1 Minikube

Minikube is a tool for running a single-node Kubernetes cluster on your laptop — one master node and one worker node, enough to learn and experiment. For production, use managed services (EKS, GKE, AKS) with many worker nodes.

13.14 Self-Study Assignment

Before the next class, students are expected to watch a 45-minute end-to-end CI/CD demo covering the entire lifecycle: source code → GitHub Actions CI → Docker build → push to registry → Argo CD CD → deploy to Kubernetes. The demo uses the same REST application shown in class.

13.14.1 Homework Instructions

Access the demo using your BITS e-mail ID through Microsoft Teams. This will make the next lecture on Kubernetes deployment significantly easier to follow.

Exam Guidance Summary

• The three eras of deployment (physical → VM → container): know the differences and advantages of each.
• Docker's three-tier architecture (client → daemon → registry): understand what happens during docker pull, docker build, docker run.
• The difference between an image (static blueprint) and a container (running instance) is frequently tested.
• Dockerfile instruction order matters: base image → working directory → copy dependencies → install dependencies → copy source → expose port → set command.
• Port mapping (-p host:container) is necessary to access containerised applications externally.
• Volumes provide persistence; without them, container data is ephemeral.
• Kubernetes master-slave architecture: master node (API server, scheduler, controller manager, etcd) and worker nodes (Docker engine, pod, kubelet, kube proxy).
• A pod is the smallest deployable unit in Kubernetes, not a container.
• Managed Kubernetes services (EKS, GKE, AKS) versus self-managed: know version upgrade behaviour.
• CNCF is the open-source foundation that maintains Kubernetes.
• CI/CD pipeline: source code → CI build → Docker image → registry → CD deploy → Kubernetes pods.

Key Industry Applications

• AWS Lambda — container-based serverless functions; no OS management required
• AWS EC2 — virtual machine instances; full OS control
• AWS EKS — managed Kubernetes service on AWS
• AWS ECS — Elastic Container Service, AWS's own orchestration (alternative to Kubernetes)
• AWS ECR — Elastic Container Registry (alternative to Docker Hub)
• Docker Hub — the most popular public container registry (free for public images)
• Azure Container Registry / AKS — Microsoft's container registry and managed Kubernetes
• Google Container Registry / GKE — Google's equivalents
• GitHub Actions — CI/CD pipeline tool demonstrated in the self-study material
• Argo CD — GitOps-based continuous delivery, a CNCF graduated project
• Helm — Kubernetes package manager
• Minikube — local single-node Kubernetes cluster for development and learning
• BITS Virtual Lab — uses containerised images on AWS VMs with EBS volumes for persistence
• OTT platforms (Amazon Prime, Netflix) — use Kubernetes for microservice scaling
• ChatGPT — runs on Kubernetes infrastructure