—

Stream Processing and Analytics

ZooKeeper: Configuration and Coordination in the Streaming Pipeline

Published: 2026-08-07

Level: postgraduate

Audience: Postgraduate students in stream processing and analytics

Prerequisite Knowledge

This lecture builds on the following concepts from earlier lectures. If any feel unfamiliar, review the linked notes before proceeding.

Previously Covered in This Subject

Events: the fundamental building block — the shape and role of streaming events (Lecture 2)
The layered streaming architecture — collection tier, data flow tier, analysis tier (Lecture 4)
The data flow layer — which events go to which node and which client (Lecture 4)
Replication and fault tolerance — the safety net behind the ensemble (Lecture 4)
Message delivery semantics — at-most-once, at-least-once, exactly-once (Lecture 4)
Stream data properties — high availability, low latency, and scalability (Lecture 4)
Processing frameworks — Storm, Spark, and Kafka in the streaming stack (Lecture 3)

5.1 Why Streaming Needs a Configuration and Coordination System

5.1.1 Metadata and State

Hook: Your cluster processes thousands of events every second — credit card swipes, clicks, sensor readings. Now try to answer a simple question: how many nodes are working right now, which client is reading from which stream, and is the network between the collection tier and the processing tier still alive? Nobody in the pipeline is tracking this by itself. That gap is exactly what the configuration and coordination system fills.

A streaming platform is built in tiers. Events are collected in a collection tier, then flow through a data flow layer, and finally reach a processing tier where the actual computation happens. Between the collection tier and the processing tier, the system constantly carries two very different kinds of information — and it is easy to mix them up.

The first kind is the actual data: the events themselves. A credit card swipe, a click, a reading from a sensor — these travel through the pipeline as events. The second kind is metadata, which literally means data about the data. Metadata is not captured through the events. It is captured through a separate mechanism: the configuration and coordination system. That system is the subject of this entire session.

Metadata vs state. Within the coordination system, two related concepts need to be kept straight:

State information — the operational picture of the cluster: the number of nodes, the number of partitions, how many partitions are filled, who the clients are, and how many requests are coming in.
State — once that kind of operational detail is processed and maintained, those details are called state. In the words of the lecture: "if we do this processed information, those details are called state."

So the metadata records the state of the system, and the coordination system exists to manage that state centrally. The professor's phrase for the centralized store is "data central" — the one place that holds the state of the whole distributed cluster. Without somebody managing this particular data, running a distributed cluster at all becomes very difficult.

Analogy: think of a city's traffic control room. The cars on the road are the events — the real data. The control room's screens — how many signals are green, where the congestion is, which intersections are staffed — are the metadata. A screen is not a car; it is data about the cars. If the screens go dark, the city does not stop having traffic, but nobody can manage it: signals go stale, blocked streets stay blocked, and commuters are late. The coordination system is the control room for a streaming cluster — and the analogy breaks in one place: a control room is a single point of failure, while a coordinator must itself be distributed (Section 5.8.3).

5.1.2 What Happens Without a Coordinator

The lecture is explicit about the failure mode: if no configuration and coordination system is in place, the pipeline can suffer delays in processing — or worse, miss out on the processing completely. The reasons trace back to the physical reality of distributed systems: unreliable networks, unsynchronized clocks, and nodes that die silently, all of which are covered in Section 5.3. For batch systems a single failure is a minor annoyance that can be fixed by hand; for real-time systems it is a real problem, because recovery introduces exactly those delays or drops the work outright.

The name of the system that solves this problem is Apache ZooKeeper — the configuration and coordination system used in this architecture. Everything in this session is about what ZooKeeper does and how it works: why the cluster needs it, what services it provides, how it elects a leader, how it survives failures, and what guarantees it gives.

Recap: a streaming pipeline carries two kinds of information — events (the data) and metadata (data about the data, captured through the coordination system, not through the events). The centralized store of processed state — "data central" — is what lets a distributed cluster run at all. Without it, processing is delayed or missed entirely. ZooKeeper is the coordinator used in this architecture, and the rest of the session explains it in depth.

5.2 Nodes, Clients, and the Data Flow Layer

5.2.1 Computing Nodes and Clients

Hook: The first slide of the session shows a picture of boxes and arrows — computing nodes at the top, clients at the bottom. The professor opens with a question to the class: what do these boxes represent, and are the nodes ZooKeeper? The short answer is no — in this picture there is no ZooKeeper yet. The long answer is what the session is about.

The nodes are the machines where the actual computation happens — "these nodes are nothing but machines." The clients are the entities that consume the processed data. A client is downstream of the processing: it registers for consumption and accesses the results once the calculation is done.

The web store example makes this concrete. Imagine a web store. The store generates events like who purchased something and what the volume of transactions is. Those events must be processed, and the web store application is the actor that consumes the processed data. So the web store is the client. Whatever actor consumes the processed data can be one of these clients. That is the interaction you see in the diagram between nodes and clients.

Q: I am unable to understand what the nodes and clients in that slide of yours represent. Are they representing ZooKeeper? I mean, are the nodes ZooKeeper? A: No, no. These are the computing nodes. There is no ZooKeeper so far. We are only talking about how the processing happens and why we need the configuration system. Only that we are discussing. The client is the one which is downstream. In a web store you have events like who purchased and what is the volume of transactions — events to be processed. Your web store will process that information. The consumption of the data is again your web store application. Whatever actor is consuming this processed data can potentially be these clients. That is why you see this interaction between the node and the client.

The correction to hold onto: the diagram is a picture of the processing pipeline before coordination — nodes doing computation, clients consuming results. ZooKeeper has not appeared yet; it is introduced in this session as the system that manages this whole arrangement.

Q: Why is client B not interacting directly with node 1 or node 3? Why only node 2? A: No, no. Because the client application will have some limitation about whom it has to interact with. If you have a kind of application, it is not necessary that you should interact with every node.

A client application is not obligated to reach every node — it interacts with the nodes it is set up to interact with. The data flow layer (below) decides which events a client may consume, and the client talks to the nodes that hold those events.

5.2.2 Worked Example: Credit Card Events and Mean Amount per Vendor

A student question — "are these nodes message queues?" — triggers the richest worked example of the session. The answer is a firm no: nodes are not message queues. The professor then demonstrates what a node actually does with a streaming computation.

Setup: take credit card events. Each event is generated when a card is swiped. An event looks like this:

Timestamp — when the transaction happened
Card number (or card ID) — which card was used
Amount — the amount of the transaction
Vendor — where the transaction happened

One swipe is one event. The next swipe is another event with the same shape. These events enter the system continuously.

The business question: suppose a credit card scheme has been launched and many people are using their cards. The business wants feedback — what are people spending, and where? The computation that answers this is: compute the mean amount based on vendor — in database terms, group by vendor and calculate the mean value.

How the streaming platform runs this: the events are distributed across the individual nodes, and each node is a machine. Each machine will give a result — a mean value for the vendors whose events landed on it. The nodes do a group-by-vendor and calculate the mean. This is exactly what the slide's "nodes" represent: actual computation happens in the nodes.

Formalizing the group-by mean. The lecture's plain-language description is the audit trail for the math: "Group by vendor. And then they calculate the mean value." Reconstructed with symbols:

$a ˉ_{v} = \frac{1}{∣ E _{v} ∣} e \in E_{v} \sum a_{e}$

Every symbol named:

$E_{v}$ — the set of events for vendor $v$ (the group)
$∣ E_{v} ∣$ — how many events that vendor has (the size of the group)
$a_{e}$ — the amount of event $e$
$a ˉ_{v}$ — the mean amount for vendor $v$

Two-step view of the same formula — first sum, then divide:

$a ˉ_{v} = \frac{\sum _{e \in E_{v}} a _{e}}{∣ E _{v} ∣} = \frac{total spent at v}{number of swipes at v}$

This is the standard mean-of-a-group formula; texts write it identically as $x ˉ_{v} = \frac{1}{n _{v}} \sum_{i = 1}^{n_{v}} x_{i, v}$ . Here we keep $a ˉ_{v}$ and $a_{e}$ to match the lecture's "amount" wording. The reconstruction is confirmed: the lecture states only "group by vendor and then calculate the mean value," and the formula above is exactly the standard group-by average used by streaming aggregation engines.

Worked example: three swipes at one vendor. Suppose vendor $v$ = "Coffee Cart". Three events land on the node:

Event $e_{1}$ : timestamp 09:01, card 4111, amount $a_{e_{1}} = 4.50$ , vendor Coffee Cart
Event $e_{2}$ : timestamp 09:05, card 4111, amount $a_{e_{2}} = 5.00$ , vendor Coffee Cart
Event $e_{3}$ : timestamp 09:09, card 5555, amount $a_{e_{3}} = 8.00$ , vendor Coffee Cart

So $E_{v} = {e_{1}, e_{2}, e_{3}}$ and $∣ E_{v} ∣ = 3$ . Step 1 — sum the amounts:

$e \in E_{v} \sum a_{e} = 4.50 + 5.00 + 8.00 = 17.50$

Step 2 — divide by the count:

$a ˉ_{v} = \frac{17.50}{3} \approx 5.83$

Answer: the mean amount per Coffee Cart swipe is about 5.83.

Sense-check: a mean must lie between the smallest amount (4.50) and the largest (8.00). 5.83 is inside that range, so the arithmetic is plausible.

Who decides which event goes to which node? That is the job of the data flow layer. In this course's stack, the data flow layer is Kafka: the Kafka broker decides which event goes to which node. The data flow layer also decides which event goes to which client — a client must register for consumership first. Once the calculation is done, the results go downstream and the clients access the data.

Q: Are these nodes the message queues? I am a bit confused. A: No, these are not message queues. Suppose you perform some task, like finding the average on the streaming data. Take credit card events: transaction number, amount. When you swipe the card, an event is generated. The event has a timestamp, the card, the amount, the vendor. When these events enter the system, let us compute the mean amount based on vendor. Suppose as a credit card company I have launched a scheme because of which many people are using credit cards, and I want feedback. This data is processed by individual nodes. The nodes are nothing but machines. Each machine will give a result — the mean value. The machines do the group-by-vendor and calculate the mean. This is what the slide refers to: actual computation happens in the nodes. Who decides which event goes to which node? The data flow layer decides — in our case the Kafka broker, which is the data flow layer. It also decides which event goes to which client, because the client needs to register for consumership. After the calculation is done, the clients downstream access the data.

Why the nodes are not message queues. A message queue stores messages and hands them to a consumer later; a node computes — it groups, averages, joins. Both sit between producers and consumers, which is why the confusion is natural, but the roles are different. In the same stack, Kafka plays the queue-like role (the data flow layer), while the nodes play the computing role (the processing layer).

Pitfalls:

Nodes are not queues. A queue stores and forwards; a node transforms. The node's job is computation (group, average, join), not storage.
The mean of means is not the global mean. Each node computes the mean for its own events. If a vendor's events are split across two nodes, the true global mean is (sum of local sums) ÷ (sum of local counts) — never average the per-node means. If node 1 saw 1 event of 5.00 and node 2 saw 2 events of 9.00, mean-of-means gives $(5 + 9) /2 = 7.00$ , while the correct mean is $(5 + 9 + 9) /3 \approx 7.67$ .
A client cannot read results it did not register for. The data flow layer decides which events go to which client, and only after the client registers for consumership.

Real-world: this exact pattern — a card scheme launch followed by per-vendor spending analysis — is a standard streaming analytics workload in banking: continuous events in, group-by aggregations out, with the results feeding dashboards and business feedback loops. Streaming processing engines implement precisely this group-by-mean computation over windows of events, and the same pattern generalizes to per-store sales totals, per-device error rates, and per-IP traffic averages.

Recap: nodes are the machines where computation happens; clients are the downstream consumers of processed data; the data flow layer (the Kafka broker, in this stack) decides which event goes to which node and which client. The canonical workload is the group-by-vendor mean $a ˉ_{v} = \frac{1}{∣ E _{v} ∣} \sum_{e \in E_{v}} a_{e}$ . The next section moves from the happy path to the failure modes that make a coordinator necessary.

5.3 Failure Modes the Coordinator Must Handle

5.3.1 Unreliable Network Connections

The main purpose of the coordination and configuration system is to track problems across the tiers. The professor enumerates the failure points explicitly:

Failure of the network between the collection tier and the data flow tier.
Failure of the network between the data flow layer and the processing layer.
Problems within the processing layer itself — if some node misbehaves, how do you even track that?

Scope: how unreliable networks really are. Even inside a single well-managed datacenter, a network is unreliable compared to a single machine: latency varies moment to moment, bandwidth changes over time, and connections can be lost. In a wide-area setting, a single severed cable can split previously unified networks in two — an event with the vivid industry name of a "backhoe event." Any of the three failure points above translates directly into the delays and missed processing described in Section 5.1.2.

Without a coordinator, these failures translate directly into the delays and missed processing from Section 5.1.2. The coordinator's job is to track these problems so the pipeline can react instead of silently degrading.

5.3.2 Clock Synchronization

Every node in the network carries a system clock, and the times on those clocks vary. Clock synchronization is the mechanism that brings consistency into the time space of the cluster.

Why does clock skew matter? Different systems will be having different times, and that becomes a real problem for applications that must be delivered on time — the professor's example is applications whose delivery depends on another cluster manager or cluster machines; if their clocks disagree, coordination breaks down and the delivery fails. Clock synchronization fixes the variability in times so every machine in the network works on a common timestamp. This is not optional polish; it is a prerequisite for any cross-node coordination.

What happens when clocks drift. Server hardware clocks are not perfect and drift over time. If they drift far enough, one server can observe an event as happening after the current time. Concretely: an analysis system comparing the timestamps of two types of events can start computing negative durations — nonsense output caused purely by clock skew, not by the data. The standard fix is the Network Time Protocol (NTP), which adjusts each machine's clock against a set of time servers. NTP is "close enough" in most setups, but it fails when machines cannot synchronize to the same NTP servers — for example, a secure internal domain behind a restricted gateway. There the internal NTP server can drift away from the external one, and time-based services (such as authorization APIs that use time) can break outright.

5.3.3 The Split Brain Problem and Replication

A second failure mode is the split brain problem. The professor's description is intuitive: within a certain timeframe, a chunk of information goes missing. Concretely: some amount of state becomes inaccessible — the state being computed at a particular node cannot be reached. The consequence is that the distributed state across the cluster cannot be updated. The cluster literally has two "brains" — the part that is reachable and the part that is not — and the system cannot reconcile them.

Analogy: two offices of the same company lose the phone line between them. Each office still works and each believes it has the full picture — one office approves a raise, the other approves a different raise for the same employee, and nobody can reconcile the two realities. A cluster in split brain is exactly this: two groups, two versions of the truth, and no way to join them.

The handling strategy is replication: create a copy of the machine. With a replica in place, you allow one partition to remain functional by degrading the capabilities of the other. The professor is candid that degrading a partition's capabilities may not be a good option either — but with a replication of the machine, whatever state information was lost through the failed node can be recovered from the backup. Replication is the safety net that makes state loss survivable.

How the functional side is chosen — the odd-count quorum trick. A common strategy is to require a quorum (a minimum number of servers) for a partition to stay fully functional. Size the cluster as an odd number and a beautiful property appears: split an odd number of servers into two groups, and one group always has an odd count while the other has an even count. The odd group stays functional; the even group is degraded (read-only, or with reduced capabilities). So at most one partition can ever claim to be the live "brain," and the two halves of a split brain can never both believe they have the authority to update state. ZooKeeper uses this same quorum idea, and Section 5.11.3 states its availability rule.

Recap: three failure modes make the coordinator necessary — unreliable networks between the tiers (delays and missed processing), unsynchronized clocks (skew breaks on-time delivery and can even produce negative durations), and the split brain problem (state becomes inaccessible and the cluster cannot reconcile two realities). Replication is the safety net that makes state loss survivable, and an odd-count quorum decides which partition stays functional during a split.

5.4 Leader Election: The Modified Paxos Algorithm

5.4.1 How a Leader Is Chosen

ZooKeeper does not rely on Paxos exactly — it uses a modified version of the Paxos algorithm. The professor's plain-language description of the election mechanism is the core of this section:

The whole ZooKeeper network — which is continuously monitoring the flow of events — has a leader node and worker nodes.
Each worker node pings the leader node. The leader is selected based on the number of pings: whichever node has the maximum responses from the ping operation is elected as the leader.
Once the leader is elected, the remaining nodes become executors (workers).
If a problem occurs with the leader node, another leader is elected — the election runs again.

The logic is simple and robust: the node that the network can reach best (maximum ping responses) is the node best positioned to coordinate.

Why "modified Paxos"? The election rule — "whichever node has maximum responses from the ping operation" — is quoted from the lecture, and the mechanics of how the responses are counted are not specified in detail in the lecture. The reconciliation: in the actual protocol, candidate servers exchange votes rather than raw pings, and each vote is ranked by an epoch number plus a transaction counter (the zxid); the candidate with the highest-ranked state wins, and the leader must collect acknowledgments from a quorum before committing changes. The lecture's "maximum ping responses" is the plain-language stand-in for "the servers that can reach you and agree to vote for you." The professor's version and the standard mechanism agree in spirit, so both are kept: pings = the votes and acknowledgments described in Section 5.11.2.

5.4.2 The ZooKeeper Ensemble

The diagram of a running ZooKeeper system shows ZooKeeper followers and a ZooKeeper leader (drawn in blue). Apache ZooKeeper was conceptualized at Yahoo! and is used across various distributed platforms.

The structure: there are client nodes and server nodes. From the perspective of a data architect, you configure the cluster, and the client node is somebody interacting with the cluster from outside — a client node can be the data ingestion source from which the data is coming in.

The collection of ZooKeeper server nodes — the red box in the diagram — is called the ensemble: a collection of nodes, each with specific tasks, with no overlap of tasks, connected through a distributed network. The ensemble is the unit that survives failures; a single ZooKeeper process is never the whole system.

Worked example: the election sequence.

A ZooKeeper network is monitoring the event flow.
Every worker node pings the leader node (or the candidate nodes, in the running system).
The node with the maximum number of successful ping responses becomes the leader.
All remaining nodes become executors.
If the leader fails, the election runs again and a new leader is elected.

Concretely, with a three-server ensemble A, B, C: A reaches B and C, B reaches only A, C reaches A and B. A collects two successful responses, C collects two, B collects one. A and C tie for the maximum, and the tie-breaker (in the real protocol, the highest zxid) picks one of them; the other two become executors. If the elected leader fails, the remaining servers run the election again and one of them takes over.

This is the sequence that answers the student's question about what happens when ZooKeeper itself dies — see Section 5.8.3.

Recap: ZooKeeper elects a leader with a modified version of Paxos — in the lecture's terms, the node with the maximum ping responses wins, the rest become executors, and a failed leader triggers a fresh election. The ensemble (leader + followers, each with non-overlapping tasks) is the unit that survives failures; no single ZooKeeper process is ever the whole system.

5.5 ZooKeeper Fundamentals and Services

5.5.1 Origin and the API

ZooKeeper is designed at Yahoo! and exposes a set of APIs for interaction, which is what allows it to be embedded in different platforms — the professor names Hadoop and HBase explicitly ("whether it is Hadoop or HBase platform"). Because the platform is generic, the same coordination services are available to any distributed system that plugs into it.

What "a set of APIs" means in practice. ZooKeeper deliberately does not impose a fixed set of coordination features on its users. It provides a low-level API — modeled loosely on a hierarchical file system — and leaves the higher-level patterns (leader election, distributed queues, group membership) to be built on top of it. These patterns are called recipes in ZooKeeper jargon, and client libraries such as Curator ship battle-tested implementations of them. This trade-off is why one ZooKeeper cluster can serve many platforms: the service is generic, so Hadoop, HBase, Kafka, Storm, and Samza all use it for their coordination tasks.

5.5.2 The Services ZooKeeper Provides

ZooKeeper provides multiple services:

Naming service — helps get node information, especially the node name. A node name is associated with a file name in the physical structure; logically, the system also creates names for the nodes. So each node is addressable both physically (the file) and logically (the name). This is the same idea as service discovery: applications look up "where is the service I need" instead of hard-coding addresses.
Configuration management — multiple things fall under this, starting with updating the new configuration. When configuration must change across a cluster, the coordinator is the one place that pushes the change coherently.
Cluster management — addition of nodes, deletion of nodes, and basically load factor management. Nodes join, nodes leave, and the coordinator keeps the cluster healthy and balanced.

The three services at a glance.

Service	What it answers	Example
Naming service	"Where is node X, and what is it called?"	node name mapped to a physical file; logical names for nodes
Configuration management	"What is the current configuration?"	one place to push a cluster-wide configuration change
Cluster management	"Who is in the cluster, and is it balanced?"	add and delete nodes; load factor management

In production terms: the naming service registers where things are, configuration management keeps every machine on the same settings, and cluster management keeps membership and load healthy. Together, the three services are how ZooKeeper plays the "data central" role from Section 5.1.

Recap: ZooKeeper was designed at Yahoo! as a generic coordination service — an API, not a fixed feature set — which is why Hadoop, HBase, and the rest of the streaming stack can all plug into it. Its three headline services are naming (addressable names), configuration management (coherent updates), and cluster management (membership and load factor). The next sections go one level deeper: what a Z node is, and the three node types.

5.6 Z Nodes: Names, Paths, and the 1 MB Limit

5.6.1 Every Node Is a Z Node with a Path

Every node in the ZooKeeper hierarchy is called a Z node (the lecture's term; the reference documentation calls the same object a znode). Each Z node has a name, and there is a path associated with that name. The professor's rule for path construction: whatever traversal you do, when you get the node information you just concatenate that node information at the end of the current path, and that becomes your entire path — which also refers to the name of the node. So a node at the root is /, a child of the root is /app1, a child of that is /app1/1.1, and so on; every node is identified by its name and the sequence of paths that lead to it.

Analogy: think of the filesystem on your computer. /users/docs/notes.md is not four separate facts; it is one address built by concatenating every folder from the root down to the file. A Z node's path works the same way — the path is the address, and the last component is the node's name. The analogy breaks in one place: in ZooKeeper every node can also carry a small data payload (up to 1 MB, Section 5.6.2), so nodes behave like files and directories at once.

5.6.2 Why 1 MB of Data per Node

Every Z node can store up to 1 MB of data. The professor asks the question himself: why 1 MB? Because this system is talking about events, and events are not memory heavy. Each event is of the order of a few bytes of information — the credit card transaction example from Section 5.2 is the template. A few-byte event fits comfortably in a 1 MB node. The design priority behind the limit is clear from the lecture: the main things are synchronization and fault tolerance, not data storage. ZooKeeper is a coordination store, not a data store; the 1 MB ceiling keeps nodes light so coordination stays fast.

Scope: what the 1 MB limit implies. Because ZooKeeper is not a bulk storage facility, it does not support partial reads, partial writes, or appends: reading or writing a Z node transmits the entire byte array in a single call. And because the whole dataset must stay in memory (with snapshots and change logs written to disk), the data kept in Z nodes is small and slow-changing — "which node is the leader for partition 7" — not the streaming events themselves, and not application state that changes thousands of times per second. If you are storing more than a few bytes per event, you are using the wrong tool.

5.6.3 The Namespace Tree and the Path Correction

The namespace is a hierarchical tree. The example structure from the session: a root node; under the root, a node called first node; under first node, two child nodes; under the first of those, two directory nodes. The question is how you specify a path for each node. The tree, drawn as corrected during the session:

/
└── first node
    ├── f1
    │   ├── d1
    │   └── d2
    └── f2

This triggers a student correction that the professor accepts enthusiastically — a valuable lesson in how paths work and in the care required to keep a diagram consistent:

Worked example: addressing every node in the tree. Apply the concatenation rule from the root to each node:

Root: /
Child of root: /first node
Children of first node: /first node/f1 and /first node/f2
Children of f1: /first node/f1/d1 and /first node/f1/d2
f2 has no children.

Reading the deepest example: /first node/f1/d1 = the node named d1, inside f1, inside first node, inside the root. The address of any Z node is fully determined by its ancestors — you never need anything else to find it.

Q: By the addressing, shouldn't it be /first node/f1? The way d1 and d2 are /f1/d1, /f1/d2 — so f1 should be /first node/f1. A: Correct, correct. You are right. I think there is a gap in this. You are correct. It should be /first node/f1/d1. That is what I explained on the board, but it is not reflected here. So there is a correction. I agree with you. This is a good observation. It is /first node/f1/d1. Right? And it is the same case with f2 — it should be /first node/f2. Every node, that is what it is.

The lesson of the correction: a diagram can silently drop path components — the slide showed d1 and d2 addressed under f1 but forgot the /first node prefix. The addressing rule itself never changes; only the drawing was wrong.

Q: The observation is correct, but I think it is better if we remove /f1 from the leaf node — the tree can create that automatically. Nowhere are we giving the path, so the name can just be d1. But when you are looking at the path, at that time we can specify it. A: That may actually simplify. Yes, yes. Because otherwise there is an inconsistency — some places you are giving this, some places you are not giving this. Correct? It is a minor thing, but I agree with you.

The takeaway: a Z node's full path is the concatenation of every ancestor's name from the root — /first node/f1/d1 means the node named d1 under f1 under first node under root — and a diagram or document must apply that rule uniformly or it breeds inconsistency. Whether you write the full path on every leaf or only at lookup time is a presentation choice; the addressing rule itself is fixed.

Recap: every Z node is addressable by the path formed by concatenating its ancestors' names from the root; a node holds at most 1 MB because events are a few bytes and ZooKeeper is a coordination store, not a data store; and the namespace is a tree whose diagrams must apply the concatenation rule uniformly — the session's correction: /first node/f1/d1, not /f1/d1.

5.7 The Three Types of Z Nodes

5.7.1 Persistent Nodes

ZooKeeper classifies nodes into three types, categorized by real lifetime: persistent, ephemeral, and sequential. The professor stresses that this theory matters — it is quiz material (see the Exam note below).

A persistent node is the node which is alive even after the client that created it is disconnected. Persistency means exactly this: the node exists even when the client is not accessible. The node outlives the client.

The professor's wording on the slide — "alive even after the client which created that particular Z node" — drew a sharp student question, and the correction is a useful precision about who creates nodes:

Q: Why would the client create the Z node? The client is something — I mean, is the client creating it? A: No, no. The client is not creating the Z node. The statement is like this — the wording here is improper. Who is creating this? The node is created with ZooKeeper; it has nothing to do with the client. The point you need to understand is: a persistent node is the node which is live even after the client application becomes inaccessible. Only that part you see, nothing else.

The correction then connects to a subtle language point about the slide's phrasing:

Q: So what is the value of making it passive voice — "which was created"? A: Yes, which was created — the passive construction is the right one, because it describes the node's origin without claiming that the client performed the creation. The client does not create Z nodes; the node is created with ZooKeeper.

So the correct mental model: clients connect to ZooKeeper and interact with nodes, but node creation is ZooKeeper's business. The word "created" in the slide describes the node's origin, not the client's action; the property that matters is persistence — the node survives client disconnection.

5.7.2 Ephemeral Nodes

An ephemeral node is the opposite: when the client is lost, the node is deleted or disconnected too. When the client gets disconnected, the ephemeral node is deleted automatically. The professor's phrasing: for an ephemeral node, the lifetime is equal to the client — node and client die together. (The spoken version sounds like "the top client," but the sense throughout the explanation — and the reference documentation — is the lifetime of the client: an ephemeral node is destroyed when the client session that created it loses contact with the ZooKeeper cluster or ends its session.)

Ephemeral nodes are the natural building block for membership and liveness: a node registered as ephemeral disappears by itself the moment its owner goes away, so no one has to clean up after a dead client.

Pitfall: ephemeral nodes cannot have children. Because an ephemeral node may be destroyed at any moment — the instant its client's session ends — ZooKeeper does not allow ephemeral nodes to have children. As of the 3.4 series, an ephemeral node may be a file but may not be a directory. Building a hierarchy under an ephemeral node is a design that cannot work; put the children under a persistent parent instead.

5.7.3 Sequential Nodes

A sequential node can be either persistent or ephemeral, but its representation is slightly different: its path is followed by a corresponding sequence number / ID attached at the end of the path. Where ordinary nodes have just a path, a sequential node's full name is path plus a sequence number.

The main role of sequential nodes is providing synchronization and parallelism in the network. Ordered, unique, appended identifiers are exactly what you need to serialize operations or shard work across nodes without collisions — each node gets its own numbered slot.

Worked example: unique, ordered identifiers. Suppose three servers join the same election path /ducks, and each creates an ephemeral sequential node:

Server A creates /ducks/n_0000000008
Server B creates /ducks/n_0000000009
Server C creates /ducks/n_0000000010

The counter guarantees each name is unique — no two servers can ever collide. And because the sequence is monotonic, sorting the children (n_0000000008 < n_0000000009 < n_0000000010) yields a total order: the smallest sequence number identifies the first comer. This single trick implements leader election — the node with the smallest sequence is the leader, and when it disappears the next smallest takes over. That is the actual mechanism behind the election described in Section 5.4.

5.7.4 Exam Notes on Node Types

Exam note: this theory is explicitly flagged as examinable. The professor says: "This theory, whatever I am explaining — because you will have a quiz — in the quiz, I may ask a property pertaining to these nodes." Know the three types and, for each one, the property that defines its lifetime:

Persistent node — lives on even after the client that created it disconnects; node exists without an accessible client.
Ephemeral node — deleted automatically when its client disconnects; lifetime equals the client's.
Sequential node — persistent or ephemeral, plus a sequence number appended to the path; provides synchronization and parallelism.

No matter which type, remember the vocabulary correction from this section: no client creates a Z node — every node is created with ZooKeeper, and persistence describes the node's lifetime, not the client's action.

5.8 Sessions and Heartbeats

5.8.1 Session IDs and FIFO Execution

When you use ZooKeeper you request a session, and requests within the session are executed first in, first out (FIFO). Whenever requests come in, they are put in a queue. Once the client connects to the server, a session is established and a session ID is assigned to the client. The session is the client's identity on the server, and the FIFO queue is how ZooKeeper keeps request order stable.

Analogy: a numbered ticket at a deli counter. You take a ticket (your session ID) when you walk in, and your requests are served in the order they arrived — first in, first out. The ticket is what identifies you to the counter, just as the session ID identifies the client to the server. The FIFO discipline is the foundation of the sequential consistency guarantee in Section 5.15.1: because each client's requests are executed in order, the client can rely on that order.

5.8.2 Heartbeats and Threshold Time

How does ZooKeeper know the current state of a client? Through heartbeats (a concept sometimes written as "hot bits" in noisy captions — the idea is the periodic liveness signal). The client sends heartbeat signals at regular intervals to indicate the session is valid. If the heartbeat is not received, ZooKeeper infers that the client has a problem.

All of this is configurable: you specify the threshold time — how much time is allowed before ZooKeeper really checks whether the client is alive or not. Thresholds are the tuning knobs of liveness detection, and they come back in Section 5.14 when failed nodes are handled.

How the threshold behaves in the real system. The client keeps a long-lived session with the ZooKeeper servers and exchanges heartbeats periodically. A temporary interruption does not kill the session — it stays active while heartbeats keep arriving. But if the heartbeats cease for longer than the session timeout (the threshold time from the lecture), ZooKeeper declares the session dead. That is the moment everything tied to session lifetime kicks in: ephemeral nodes created by that session are deleted automatically (Section 5.7.2), and locks held by the session are released. In the standard configuration the threshold is expressed in tick units — tickTime (for example 2000 ms) with limits such as initLimit and syncLimit — and the client also sets its own session timeout when it connects.

5.8.3 What If ZooKeeper Itself Dies?

The natural question follows: ZooKeeper receives heartbeats from clients — what happens if ZooKeeper itself is not alive? Is there another ZooKeeper?

Q: ZooKeeper receives the heartbeats from the clients, right? So what happens if ZooKeeper itself is not alive? I mean, will there be another ZooKeeper? A: It cannot happen that way, because ZooKeeper is not a single entity. It is a cluster set of nodes — that is what you are saying. It might happen that one node gets disconnected or one node goes off. But since we are talking about the ensemble, the only possibility is that when the leader node dies off, there is an election mechanism with that Paxos algorithm I mentioned. Some election mechanism will happen and a new leader will be elected.

The professor adds a practical observation: it is true — most often it can happen even when you create a local machine and run ZooKeeper locally. After some minutes, if nothing is happening in terms of data ingestion or processing, you tend to see this thing happen.

So the answer to "who monitors the monitor" is structure: ZooKeeper is never one process, it is an ensemble, and the only single point of failure — the leader — is replaceable by election. Even a local development machine will demonstrate this: an idle local ZooKeeper will eventually show leader change behavior. The ensemble does not wait for permission; it re-elects on its own.

Recap: a session is the client's identity on the server, and requests within it run FIFO. Heartbeats are the periodic liveness signal, and the threshold time you configure decides when ZooKeeper declares a session dead. And ZooKeeper itself can never "die" as a whole: it is an ensemble, and when the leader fails, the modified-Paxos election picks a new one automatically.

5.9 Watches: Change Notification

5.9.1 The Notification Mechanism

A watch is the mechanism for notification about whatever changes happen on ZooKeeper. When a client reads a particular node, ZooKeeper uses watches to tell the client about changes in the cluster of Z nodes.

The alternative to watches is polling — the client repeatedly asking "has anything changed?" Watches invert this: the client registers interest once, and ZooKeeper pushes the news the moment something relevant changes.

5.9.2 The Exception Handling Analogy

The professor's analogy makes the mechanism click: think of exception handling in programming. The purpose of exception handling is graceful termination — when an exception occurs, the code you wrote for that case gets executed, and the exception preserves the context in which it occurred, instead of the program dying abnormally.

In the same sense, a watch captures the context of the Z nodes — especially whether there is a change, and the health of the nodes (slow or fast, up or down). You configure what you want to be notified about. Watches turn ZooKeeper from a passive store into an event-driven one: interested clients are told the moment something relevant changes.

Pitfall: a watch fires once. A watch is a one-time operation: when the notification fires, the watch is used up, and the client must register it again to keep receiving notifications about the same path. If a change happens in the window between receiving a notification and re-registering the watch, that notification can be missed. ZooKeeper's designers anticipated this: setting a watch also reads the node's data, so clients can coalesce notifications — they always get the current data together with the change signal.

Recap: a watch is the notification mechanism for changes in the cluster of Z nodes — the client registers interest, ZooKeeper pushes the change. The professor's analogy: a watch is like exception handling — it captures the context of what happened (the node's state and health) instead of letting the system fail silently. Watches make ZooKeeper event-driven rather than polled.

5.10 Read and Write Operations

5.10.1 The Write Path Through the Leader

Connecting to the server is straightforward — once the ZooKeeper process starts, it waits for clients to connect; nothing special has to be done. Then there are requests for everything.

Read: if a client wants to read a particular Z node, it sends a read request for the Z node path, and the server responds with the data.
Write: if a client wants to write, it sends a write request carrying the path and the data to the server. The server forwards the request to the leader, and the leader re-issues the write request to all the followers. If the responses come back successfully, the write operation is said to be successful; otherwise, the write operation is failing.

Worked example: the write operation sequence.

Client sends a write request (path + data) to a ZooKeeper server.
The server forwards the request to the leader.
The leader re-issues the write request to all followers.
If the response is received successfully, the write is successful; if not, the write has failed.

Concretely: a client writes the value "node-7 is leader" to the path /config/leader. The write lands on server S2; S2 forwards it to the leader L; L sends the same write to every follower (S1, S3, S4); each follower applies it and answers. Only when the responses come back successfully is the write acknowledged as successful. If even one follower does not answer within the threshold, the write fails.

The write path explains why a ZooKeeper cluster is safe: no write is acknowledged until the leader has propagated it across the ensemble. Read requests, by contrast, go directly to a server.

Versions on Z nodes (supporting detail). Every Z node carries a version number that starts at creation and increments each time the node's data changes. The client can pass this version to delete and setData; if the version on the server no longer matches the one the client expected, the operation is rejected. This is the mechanism that stops two clients from silently overwriting each other's changes — a compare-and-set style guard, and one more reason ZooKeeper's operations are safe.

5.10.2 CLI and the API — Good to Know, Not Exam Material

ZooKeeper also has a command line interface (CLI). Through the CLI you can configure the system and perform tasks like setting information and removing nodes — because dynamically, one can change the node structure in the tree. On the API side, the Z node is the most important thing in ZooKeeper's API; the class details are not required.

The six fundamental operations of the Z node API are create, delete, exists, setData, getData, and getChildren — the vocabulary used by every client library, including Curator's fluent interface. (This is background vocabulary, not exam material.)

The professor is explicit about the weight of this material: "These are some of the concepts, but you may not use it in real time. It is good to know these concepts. You don't have to remember this — just to know how the whole thing works. There is no point of remembering and reproducing, because I am not going to ask any of these points in the exam perspective." So CLI and API mechanics are background knowledge; the examinable theory is the node types (Section 5.7) and the system-level concepts.

Exam note: CLI and API mechanics are "good to know" concepts — how the whole thing works — but they are not asked from the exam perspective. The examinable theory is the node types of Section 5.7 (with their lifetime properties) and the system-level concepts: metadata vs state, sessions and heartbeats, watches, the write path, and the guarantees.

5.11 The Server Cluster, Atomic Broadcast, and Fault Tolerance

5.11.1 The Ensemble Setup

The live setup, walked through with the official documentation, looks like this: ZooKeeper has multiple servers, and individual clients interact with the servers. The server leader is one of the servers coordinating with the rest of the nodes within ZooKeeper. There is deliberately no direct edge from a client to the leader — clients interact with individual server nodes, and the leader ensures the overall health of the server nodes (active, inactive, and so on) while coordinating with all of them.

The reason ZooKeeper is replicated is to enforce fault tolerance. The professor also emphasizes that "ZooKeeper is ordered": if you look at the order of transactions, the server nodes apply them in a consistent order. Ordering plus replication is what makes the ensemble behave like one coherent system.

5.11.2 Atomic Broadcast

Whenever a write request is processed, it goes through atomic broadcast. The terms are built up carefully:

Broadcast — sending data across the nodes in the network.
Atomic broadcast — broadcasting data that is atomic. Atomic data means a simple transaction: it should not have a lot of memory and size.

So a write is a small, simple transaction that gets sent to every node as one indivisible unit. That is the transport mechanism behind the write path of Section 5.10.1: the leader broadcasts, and the ensemble receives the same transaction, atomically.

The standard form: total order broadcast. The reference literature calls the same mechanism total order broadcast: every node receives the same messages in the same order — exactly the property that keeps replicas identical, since applying the same writes in the same order keeps every copy consistent. ZooKeeper's implementation is the Zab protocol — the "modified Paxos" from Section 5.4. Zab orders changes to state (transactions) rather than Paxos-style updates to the entire state, and it uses an epoch counter similar to Paxos's proposal numbers. The lecture's "atomic broadcast" and the literature's "total order broadcast" are the same concept.

5.11.3 The Availability Rule

The practical rule the professor states: as long as the majority of the servers are available, the service is available. Because the state is replicated across the ensemble, you do not need every node up — you need the majority. This is the fault-tolerance payoff of replication: a minority of dead nodes does not stop the service.

Scope: why majority, and why odd numbers. A quorum rule requires a minimum number of servers to agree before the system commits anything. With an odd-sized ensemble (3 or 5 is standard), a network split always leaves exactly one group holding the majority — that group keeps serving, the minority degrades, and the two sides can never both believe they have the quorum. This is the split-brain defense from Section 5.3.3, now stated as a concrete arithmetic rule. An even number of servers buys no extra fault tolerance over its odd neighbor: 4 servers tolerate the same one failure as 3, so production ensembles are sized 3 or 5 — more on the sizing trade-off in Section 5.12.

Recap: the ensemble is a replicated set of servers with no direct client-to-leader edge; writes travel as atomic broadcast — small, indivisible transactions delivered to every node in order (the standard literature calls it total order broadcast, implemented by Zab). Fault tolerance is bought by replication, and the availability rule falls out of it: as long as the majority of servers are available, the service is available.

5.12 Performance: More Servers Does Not Mean More Throughput

5.12.1 The Non-Linear Reality

A natural instinct is that a coordination system will get faster with more servers — start with three servers, move to five, keep adding. The professor's data point contradicts this: increasing the number of servers does not necessarily guarantee performance. Throughput varies; it is not proportional to the number of servers. When you double the nodes, the percentage of requests that are read does not increase significantly, and the performance curve is not linear.

Why? The professor pushes the class to think, then confirms the reasons:

More nodes → higher probability of failure. In the context of failure, having a lot of nodes is not a good choice — each additional node is another thing that can go wrong.
More nodes → harder coordination. The responses from all these nodes and the overall coordination become difficult. Every decision the leader makes must account for more participants, more responses, more chances for disagreement.

So the trade-off is explicit: replication buys fault tolerance (Section 5.11), but it costs coordination overhead. You size an ensemble for reliability, not for raw speed.

Q: Can you tell me the reason — when you increase the number of nodes, why the percentage of requests is not increasing significantly? Why is it not linear? A: There is a one-to-one relation with client and client number not increasing — you can think that way. But these are the reasons: if you have more nodes, the probability of occurrence of failure is also more. In the context of failure, it is not a good choice to have a lot of nodes. Also, the response from these nodes and the overall coordination become difficult. Various statistics are performed. The point to understand: just by increasing the number of nodes, it does not guarantee the performance or throughput of ZooKeeper.

The sizing rule from the reference material. The reference documentation is blunt: there is an inverse relationship between fault tolerance — more servers — and performance, because every state change must maintain consensus across the ensemble; more servers means more time per change. There is no real reason to run a ZooKeeper cluster larger than five nodes, and no reason to run fewer than three (an even count adds no fault tolerance, as shown in Section 5.11.3). Pick three nodes when the cluster will be heavily used — for example, under Kafka's metadata workload — and five when it is lightly used and you want the extra fault tolerance.

Recap: adding servers to the coordination tier does not raise throughput — the performance curve is not linear. Each extra node raises the probability of failure and makes coordination harder, and every write must still gather the quorum. Ensemble size is chosen for fault tolerance (majority availability), not for raw speed; the practical sizing is 3 or 5 nodes.

5.13 Features of Streaming Data

5.13.1 The Four Properties

The session briefly recaps the features of streaming data — discussed several times already in the course, but worth re-anchoring before the pipeline discussion:

High availability — the service stays up despite failures (the majority rule of Section 5.11.3).
Low latency — events are processed quickly as they arrive.
Fault tolerance — failures are survived via replication; state is not lost.
Horizontal scalability — the system scales by adding machines.

5.13.2 Why These Properties Need a Coordinator

These four properties are the goals that the coordination system exists to support: availability and fault tolerance through replication and election, low latency through lightweight events and fast coordination, and horizontal scalability through adding nodes — with the caveat from Section 5.12 that the coordination tier itself does not scale linearly.

Mapping the four properties to the mechanisms. Each property is delivered by a specific piece of the session's machinery:

Property	Delivered by	Session section
High availability	Majority availability rule	5.11.3
Low latency	Few-byte events, lightweight Z nodes, fast coordination	5.2, 5.6.2
Fault tolerance	Replication, split-brain defense, re-election	5.3.3, 5.4
Horizontal scalability	Adding nodes and clients	5.12 (with its caveat)

The coordination system is what makes all four possible at once — that is the payoff of the entire session's design.

Recap: high availability, low latency, fault tolerance, and horizontal scalability are the properties streaming data demands. Every one of them is engineered through the coordination mechanisms of this session: replication and election (availability, fault tolerance), lightweight few-byte events and fast coordination (low latency), and node addition (horizontal scalability) — bounded by the coordination tier's own non-linear performance ceiling.

5.14 Leader Responsibilities and Handling a Failed Node

5.14.1 What the Leader Actually Does

A student's question zeroes in on the leader's role: leader and followers are similar nodes, any follower can become the leader, and clients always connect to followers — so what does the leader do?

The answer: the leader checks the health of each of the servers. In case of any failure, it must alert the system that the particular node has failed; and if there is a replicated duplicate node, it must bring that node into the active phase. It is, in the professor's words, "kind of health coordination": if somebody is dead, the leader routes the request to somebody else.

Q: In the ZooKeeper cluster, the leader and the followers are similar nodes, so that any follower can become the leader — that is fine. But the clients are always connecting to the followers, not the leader. What is basically the responsibility of the leader? What does it do? A: Good question. It is checking the health of each of these servers. In case of any failure, it has to alert the system that the particular node has failed. If there is a replication — a duplicate node — it has to bring it to the active phase. Kind of health coordination. If somebody is dead, I will route the request to somebody else. As long as the majority of the servers are available, the service is available.

The leader is not a bigger or faster machine — it is the same hardware running the same software, with an extra job: watching everyone else and orchestrating recovery.

5.14.2 When a Client Hits a Dead Node

The follow-up question is the practical one: the client has no way to talk to the leader — what happens when the node the client is connected to fails?

The mechanism, step by step:

The leader continuously monitors state information: which nodes are active, and whether the active nodes are engaged with any client applications.
When a node fails, the leader checks that aspect and facilitates connecting the client to another node — the client is redirected because the leader has already detected the failure.
The client always sends requests directly to a node, never to the leader; the leader's job is knowing whether each node is responding.

The professor's office analogy: in your office, you are doing client work. Suppose you are not doing client work — then somebody gets escalated, somebody intervenes. The same system runs here: the leader is the escalation layer that notices a node is not doing its client work.

But there is a window: between the client's failed send and the leader's detection, events can get lost.

Q: If the client is trying to connect to a node and the node has failed or died, will ZooKeeper interact with the client, or will ZooKeeper straight away find another node? A: It will find another node, because continuously it will monitor the state information. State information means: which nodes are currently active, and are these active nodes engaged with any client applications? All that.

So the redirection is automatic: the leader's continuous monitoring of state information is what makes it possible — the client does not have to notice anything.

Q: Then how will the replicated node connect with the client? Is it the responsibility of the replicated node to get connected with the client? A: No, no, no. It is the responsibility of the leader to check that aspect and facilitate the connecting to the client application. But the client always sends requests directly to the node, not to the leader. The leader's job is whether the node is responding or not. In your office, you are doing client work; suppose you are not doing client work — somebody gets escalated, somebody will intervene. The same system here.

The division of labor is strict: the client talks only to nodes; the leader manages the ensemble; the replacement node does not chase the client — the leader arranges the handoff.

Q: But the client was trying to connect to the node and the node has failed. The client has no way to communicate with the leader. In the meantime, events could get lost because the client sent them to the node. A: Yes, that can happen. That is why you see the delivery semantics. For example, the places where you have at-most-once delivery semantics — the event is okay if you lose a couple of events; in that case it is okay. Suppose it is very sensitive and you cannot afford to lose the data — then you set the configurations for these nodes: what will be the threshold response time? It will check whether the node responded within the threshold time. That is how another node will be picked up from the ensemble.

The student's worry is real: the failed node may have accepted events that never got processed. That is why the answer moves from the mechanism to the guarantees — delivery semantics.

5.14.3 Delivery Semantics, Recovery, and Client Retry

The professor's answer introduces the crucial design concept: delivery semantics. Where at-most-once delivery is acceptable, losing a couple of events is fine. Where the data is sensitive, you configure the threshold response time per node; if a node does not respond within the threshold, the ensemble picks another node.

The three delivery semantics (standard forms). Streaming pipelines classify their data-loss tolerance into three levels:

At-most-once — an event is delivered at most once; losing a couple of events is acceptable. The cheapest option, with the best latency, used when the metric tolerates small gaps.
At-least-once — the system retries until every event is delivered at least once, but the same event can arrive twice, so duplicates must be handled downstream.
Exactly-once — each event is processed exactly once, even across failures. The most expensive option; it needs coordination between the producer, the pipeline, and the storage.

The lecture's example is the at-most-once end: "the event is okay if you lose a couple of events."

Recovery: because state information is centrally maintained, there is a way to look at state information and connection loss, and therefore a way to recover the events that got lost. But the professor is honest about the difficulty: recovery is slightly difficult, because events arrive on a regular basis — there is no finite snapshot to replay from. What you do: increase the capacity of the node (so it can absorb more) and configure the threshold response time for each node.

Can the client help? Yes — retry is the client's responsibility:

Q: I do not have much control over client nodes. But if the client knows the message might get lost or it might fail to connect to the server, it can do a retry in the configuration — for example, after one minute it sends the same message if it was not able to deliver, hoping meanwhile the leader will make someone in place to receive it. That way? A: Yes, this is the responsibility of the client to actually do the retry. That way also you can. But these are all configuration setups that are available, and by looking at those configuration settings you can customize the way you want.

So the client is not helpless: it can schedule its own retry after a configured interval, buying time for the leader to bring a replacement into place.

Q: So there is no diversion possibility for the client — the requests cannot be diverted to another node like a load balancer? A: No, there is no diversion possibility. But that can happen depending upon the threshold — how much response time you are giving to this node. It can be forwarded or it can be load balanced based on the threshold that you set as the response time.

There is no client-side load balancer in the design — the routing decision belongs to the coordination layer, expressed through the thresholds you configure.

The full picture: liveness detection (thresholds) + centralized state + client-side retry + capacity configuration together determine how much data loss a pipeline tolerates. At-most-once semantics tolerate loss; sensitive data needs threshold tuning and retry — and even then, some recovery of regularly arriving events is difficult by nature.

Recap: the leader is the health coordinator — it monitors which nodes are active and engaged, alerts on failure, and brings replicated duplicates into the active phase, while clients always talk to nodes directly. When a client's node dies, the leader detects it within the configured threshold and picks another node from the ensemble, but events sent into the failed node in the meantime can be lost — which is why delivery semantics, threshold tuning, node capacity, and client-side retry are the levers that decide how much loss a pipeline tolerates.

5.15 The Guarantees of ZooKeeper

5.15.1 The Five Guarantees

The professor walks through the guarantees ZooKeeper provides — the contract between the coordination system and everything that relies on it:

Sequential consistency — updates from a client will be applied in the order they are sent.
Atomicity — an update either succeeds or fails; there is no partial result.
Single system image — the client always sees a single system view, because the transactions are atomic; the ensemble looks like one machine, not many.
Reliability — once an update is applied, it will persist from that time forward until the client overwrites it.
Timeliness — the system responds within a certain time bound; this is where the thresholds you specify come in.

Where each guarantee comes from. Each of the five is a contract backed by machinery built earlier in the session:

Sequential consistency is the FIFO session discipline of Section 5.8.1 plus the ordered broadcast of Section 5.11.2.
Atomicity comes from atomic broadcast: a write is one indivisible transaction delivered to every node or to none.
Single system image is atomicity plus replication: because every node applies the same transactions in the same order, any server you talk to shows the same system.
Reliability is persistence on the replicated ensemble: once applied, an update survives until it is overwritten.
Timeliness is the threshold machinery of Sections 5.8.2 and 5.14: the system promises to respond within the time bound you configure.

5.15.2 Why the Guarantees Matter

These five guarantees are the answer to "why can I trust this system to coordinate my cluster": ordered, atomic, consistent, durable, and time-bounded operations — the properties built from everything earlier in the session (FIFO sessions, the write path, replication, and the threshold configuration).

Precision: what the guarantees do and do not cover. The literature adds two precision points. First, strictly speaking, ZooKeeper gives linearizable (atomic) writes; reads may be stale by default — a client that needs to see its own latest write must request it explicitly (in the reference client, a sync call before the read). Second, the guarantees hold for the coordination data — the small, slow-changing state in Z nodes — not for the high-volume event stream. The event stream itself is handled by the data flow layer (Kafka, Section 5.16), while ZooKeeper coordinates the ensemble underneath. Keep the two roles separate and the guarantees make sense.

Recap: ZooKeeper's contract with everything that relies on it is five guarantees — sequential consistency, atomicity, single system image, reliability, and timeliness. Each one is engineered from the session's building blocks: FIFO sessions, atomic broadcast, replication, persistence, and the threshold configuration. They are the reason a distributed cluster can trust a coordination service at all.

5.16 The End-to-End Pipeline and What Comes Next

5.16.1 Where ZooKeeper Sits in the Architecture

Closing the loop back to the tier diagram from Section 5.1: the collection layer, then processing. The messages coming in are where the coordination happens — ZooKeeper coordinates with the collection layer, for example with Kafka. Kafka itself is a cluster and is a combination of two things (collection plus the data flow layer); from Kafka the data goes to Spark. That is how the whole thing works: collection → Kafka → Spark, with ZooKeeper coordinating the ensemble underneath.

collection layer → data flow layer → processing layer
    (sources)         (Kafka)           (Spark)
        └─────────── ZooKeeper coordinates ───────────┘

Real-world: this is the concrete orchestration you will see in the live demo. The professor plans a demonstration after Kafka streaming is covered — watching the orchestration of the entire example live is where the actual parts become visible. Kafka itself relies on ZooKeeper for its own coordination — it uses ZooKeeper to track broker liveness and manage metadata for its servers and clients — so the coordination service sits under the whole pipeline, not just at one tier.

5.16.2 Practicalities, the Next Topic, and the Quiz

A few administrative threads were settled at the end of the session. One student asked about the documentation URL for people who were not attending live and could not see the chat:

Q: For people who are not actually attending now and would like to see the documentation — the URL is not available in the chat. Can you put it somewhere? A: What they can do: they can download the slide deck — it gets uploaded in the folder in Teams. Once they download the slides, the link is available in the slide itself. That way they can also access it.

The documentation link lives inside the slide deck, which is uploaded to the course folder — so anyone who downloads the slides gets the link with them.

The next topic was announced:

Q: What is the topic we will start next class? A: We will be starting the streaming components like the data flow layer — how you should look at it. I will directly start with Kafka streaming.

And the quiz question was settled too:

Q: What about the quiz? Are we going to have the quiz this week? A: No, no. I showed you the schedule right after a couple of lectures. We will plan.

Exam note: the quiz is not this week; it is planned after a couple of lectures, per the schedule already shown. The next session starts the data flow layer, beginning with Kafka streaming — so the natural preparation is to understand what the collection and data flow layers do, and where ZooKeeper fits underneath them in the full pipeline.

Exam Guidance Summary

Node types are the explicitly examinable theory of this session. The professor: "You will have a quiz... in the quiz, I may ask a property pertaining to these nodes." Know the three types and their lifetime properties: persistent (alive even after the client disconnects), ephemeral (deleted automatically when the client disconnects; lifetime equals the client's), sequential (persistent or ephemeral, with a sequence number appended to the path; provides synchronization and parallelism). Remember the vocabulary correction: no client creates a Z node — every node is created with ZooKeeper.
Not examinable: the CLI and API mechanics. The professor stated directly that these are "good to know" concepts — how the whole thing works — but are not asked from the exam perspective ("I am not going to ask any of these points in the exam perspective"). The Z node is the most important thing in the API, but class details are not required.
Quiz scheduling: no quiz this week; the quiz comes after a couple of lectures per the schedule shown in the session.
Know the system-level concepts even though they were not explicitly marked as examinable, because they explain how ZooKeeper works: metadata vs state, clock synchronization, split brain, leader election (modified Paxos), the ensemble, atomic broadcast, the majority availability rule, and the five guarantees (sequential consistency, atomicity, single system image, reliability, timeliness).
Next lecture: the data flow layer begins with Kafka streaming.

Key Industry Applications

Apache ZooKeeper — the configuration and coordination system of this architecture, conceptualized at Yahoo! and reused across distributed platforms including Hadoop and HBase. In production it is also the coordination backbone of Kafka (metadata for servers and clients) and of stream-processing frameworks such as Storm and Samza.
ZooKeeper's services in production — naming service (node names mapped to physical files, plus logical names), configuration management (rolling out new configuration coherently), and cluster management (node addition/deletion and load factor management).
Split brain handling via replication — the standard production answer to split brain is replication: keep a copy of each machine, keep one partition functional by degrading the other's capabilities (the odd-count quorum trick), and recover lost state from the backup.
Leader election with a modified Paxos algorithm — used by ZooKeeper to elect and re-elect leaders by ping response counts (votes and acknowledgments, in the real protocol); the same family of consensus ideas underlies many distributed coordination systems, and ZooKeeper's own implementation is the Zab protocol with epoch numbers.
Delivery semantics in real pipelines — at-most-once delivery tolerates occasional event loss; sensitive workloads require per-node threshold response time configuration, increased node capacity, and client-side retry after a configured interval. The standard taxonomy adds at-least-once and exactly-once for stricter guarantees.
The non-linear performance lesson — adding servers to the coordination tier raises failure probability and coordination cost; ensemble size is chosen for fault tolerance (majority availability), not for throughput. Production ensembles are sized 3 or 5 nodes.
The production stack — collection layer (Kafka) → processing layer (Spark), with ZooKeeper coordinating; a live demo of the full orchestration is planned after Kafka streaming is covered.
1 MB Z-node limit — a design reminder that coordination stores hold tiny events (a few bytes, like credit card transaction events), not bulk data; synchronization and fault tolerance, not storage, are the goals.

SPA Lecture 05 notes · ZooKeeper: Configuration and Coordination in the Streaming Pipeline

Stream Processing and Analytics· postgraduate· 2026-08-07

Sections Breakdown

1Why Streaming Needs a Configuration and Coordination System

A streaming pipeline carries events (the data) and metadata (data about the data, captured by the coordination system). The centralized store of processed state, 'data central', is what lets a distributed cluster run; without it, processing is delayed or missed.

2Nodes, Clients, and the Data Flow Layer

Nodes are the machines where computation happens (group-by-vendor mean over credit card events); clients are downstream consumers of processed data; the data flow layer (the Kafka broker) decides which event goes to which node and client.

3Failure Modes the Coordinator Must Handle

The coordinator exists to track failures across tiers: unreliable networks between tiers (delays, missed processing), unsynchronized clocks (skew breaks on-time delivery, can yield negative durations), and the split brain problem (state becomes inaccessible), handled by replication with an odd-count quorum.

4Leader Election: The Modified Paxos Algorithm

ZooKeeper elects its leader with a modified version of Paxos: workers ping candidates, the node with the maximum ping responses becomes leader, the rest become executors, and a failed leader triggers a new election. The ensemble (leader plus followers with non-overlapping tasks) is the failure-surviving unit.

5ZooKeeper Fundamentals and Services

ZooKeeper was designed at Yahoo! as a generic coordination service exposing a set of APIs, embeddable in platforms such as Hadoop and HBase; it provides a naming service, configuration management, and cluster management (with load factor management).

6Z Nodes: Names, Paths, and the 1 MB Limit

Every node in the ZooKeeper hierarchy is a Z node with a name and a path built by concatenating ancestor names from the root (e.g. /first node/f1/d1). Each Z node holds at most 1 MB because events are a few bytes and ZooKeeper is a coordination store, not a data store.

7The Three Types of Z Nodes

ZooKeeper classifies Z nodes by real lifetime into three types: persistent (alive even after the client disconnects; the node is created with ZooKeeper, not by the client), ephemeral (deleted automatically when its client disconnects; lifetime equals the client's), and sequential (persistent or ephemeral, with a monotonic sequence number appended to the path).

8Sessions and Heartbeats

A session is the client's identity on the server, and its requests execute FIFO. Heartbeats are periodic liveness signals, and the configurable threshold time decides when ZooKeeper declares a client dead. ZooKeeper itself cannot die as a whole: it is an ensemble, and a failed leader is replaced by re-election.

9Watches: Change Notification

A watch is the notification mechanism for changes in the cluster of Z nodes; like exception handling, it captures the context of what changed (node health, up/down, slow/fast) and makes ZooKeeper event-driven instead of polled.

10Read and Write Operations

Reads go directly to a server; writes are forwarded to the leader, which re-issues them to all followers — success only when the responses come back successfully. The CLI and API mechanics are good-to-know background, not exam material.

11The Server Cluster, Atomic Broadcast, and Fault Tolerance

The ensemble is a replicated set of servers with no direct client-to-leader edge; writes travel as atomic broadcast (small, indivisible transactions delivered to every node in a consistent order, implemented by Zab). Fault tolerance comes from replication, and the availability rule is: as long as the majority of servers are available, the service is available.

12Performance: More Servers Does Not Mean More Throughput

Increasing the number of servers does not guarantee performance: the curve is not linear, because more nodes raise the probability of failure and make coordination harder. Ensembles are sized for fault tolerance (3 or 5 nodes), not raw throughput.

13Features of Streaming Data

Streaming data has four features: high availability, low latency, fault tolerance, and horizontal scalability — each delivered by the coordination mechanisms of this session (replication, election, lightweight events, node addition).

14Leader Responsibilities and Handling a Failed Node

The leader is the health coordinator: it monitors which nodes are active and engaged, alerts on failure, brings replicated duplicates into the active phase, and facilitates reconnecting clients to another node. Events sent into a failed node can be lost; delivery semantics, threshold response time, node capacity, and client-side retry decide how much loss a pipeline tolerates.

15The Guarantees of ZooKeeper

ZooKeeper's contract is five guarantees: sequential consistency, atomicity, single system image, reliability, and timeliness — each engineered from FIFO sessions, atomic broadcast, replication, persistence, and the threshold configuration.

16The End-to-End Pipeline and What Comes Next

The pipeline runs collection → Kafka → Spark with ZooKeeper coordinating the ensemble underneath; Kafka itself relies on ZooKeeper for its metadata. The next class starts the data flow layer with Kafka streaming; the quiz is not this week.

17Exam Guidance Summary

Node types are the explicitly examinable theory (persistent, ephemeral, sequential, with lifetime properties); CLI and API mechanics are good-to-know but not asked from the exam perspective; no quiz this week; know the system-level concepts that explain how ZooKeeper works.

18Key Industry Applications

Apache ZooKeeper coordinates production distributed platforms (Hadoop, HBase, Kafka, Storm, Samza): naming, configuration and cluster management services; split brain handled by replication; leader election via modified Paxos; delivery semantics tuning; 3-or-5-node ensembles; the Kafka → Spark production stack.

Postgraduate students in stream processing and analytics

Exam Revision Notes

Below is the distilled, exam-ready core. Every entry comes from the full explanation above. Use this section for rapid review; return to the main notes when a point needs more context.

Why Streaming Needs a Configuration and Coordination System

Must-know: Events are the data; metadata is data about the data, captured separately through the configuration and coordination system. Processed metadata is called state, held centrally ('data central'). Without a coordinator, processing suffers delays or is missed.

⚠️ Top pitfall: Confusing the event stream with metadata: events travel through the pipeline as data; state information (nodes, partitions, clients, requests) lives in the coordination system, not in the events.

Self-check: What are the two kinds of information a streaming pipeline carries, and which one is captured through the coordination system?

Connects to: nodes, clients, and the data flow layer, failure modes

Nodes, Clients, and the Data Flow Layer

Must-know: Nodes are machines where the actual computation happens (not message queues, not ZooKeeper); clients are downstream consumers that register for consumption; the data flow layer (Kafka broker) decides which event goes to which node and which client.

$a ˉ_{v} = \frac{1}{∣ E _{v} ∣} e \in E_{v} \sum a_{e}$

⚠️ Top pitfall: The mean of means is not the global mean: with a vendor's events split across nodes, combine sums and counts (sum of local sums / sum of local counts), never average per-node means.

Self-check: A card swipe event has a timestamp, card number, amount, and vendor. Three swipes at one vendor are 4.50, 5.00, 8.00 — what is the mean amount per swipe?

Connects to: configuration and coordination basics, Z nodes and paths, leader responsibilities and failed nodes

Failure Modes the Coordinator Must Handle

Must-know: Failure modes the coordinator must handle: (1) unreliable networks between collection tier, data flow layer, and processing layer; (2) clock synchronization — clock skew breaks on-time delivery and can produce negative durations; (3) split brain — state becomes inaccessible, handled by replication plus a quorum rule.

⚠️ Top pitfall: Treating clocks as reliable: without synchronization, one server can observe an event as happening after the current time (negative duration), so NTP-level consistency is a prerequisite, not polish.

Self-check: Why does an odd-sized ensemble matter for split brain?

Connects to: configuration and coordination basics, atomic broadcast and fault tolerance

Leader Election: The Modified Paxos Algorithm

Must-know: Leader election uses a modified Paxos algorithm: each worker pings, the node with the maximum number of successful ping responses is elected leader, the rest become executors; if the leader fails, the election runs again. The ensemble (collection of server nodes, no overlapping tasks) is the unit that survives failures.

⚠️ Top pitfall: Thinking ZooKeeper uses stock Paxos: it uses a modified version (the reference implementation is Zab, which orders state changes and uses an epoch counter); the professor's 'maximum ping responses' is the plain-language form of vote counting.

Self-check: What happens when the leader node of a ZooKeeper ensemble fails?

Connects to: Z node types, sessions and heartbeats, atomic broadcast and fault tolerance

ZooKeeper Fundamentals and Services

Must-know: ZooKeeper is a generic platform: designed at Yahoo!, it exposes a set of APIs for interaction and is embedded in Hadoop, HBase, and other distributed platforms. Its services: naming service (node names, physically and logically), configuration management (coherent config updates), and cluster management (add/delete nodes, load factor management).

⚠️ Top pitfall: Treating ZooKeeper as an application-specific component: the API is low-level and generic, and higher-level patterns (recipes such as leader election, distributed queues) are built on top of it by libraries like Curator.

Self-check: Name the three services ZooKeeper provides.

Connects to: configuration and coordination basics, Z nodes and paths

Z Nodes: Names, Paths, and the 1 MB Limit

Must-know: Every Z node has a name and a path; the path is the concatenation of ancestor names from the root (the session's correction: /first node/f1/d1, and f2 is /first node/f2). A Z node stores up to 1 MB: events are only a few bytes, and the priorities are synchronization and fault tolerance, not storage.

⚠️ Top pitfall: Diagrams that drop path components: addressing must apply the concatenation rule uniformly, or it breeds inconsistency (the slide showed /f1/d1 instead of /first node/f1/d1).

Self-check: In the tree root → first node → f1 → d1, what is d1's full path?

Connects to: nodes, clients, and the data flow layer, Z node types

The Three Types of Z Nodes

Must-know: Three Z node types by lifetime: persistent — alive even after the client that created it disconnects (node exists without an accessible client; the client does NOT create the node, it is created with ZooKeeper); ephemeral — deleted automatically when its client disconnects, lifetime equals the client's; sequential — persistent or ephemeral plus a sequence number appended to the path, providing synchronization and parallelism.

⚠️ Top pitfall: The vocabulary correction: 'the client which created the Z node' is improper wording — no client creates a Z node; the passive 'which was created' correctly describes the node's origin. Also, ephemeral nodes cannot have children.

Self-check: Which node type survives its client's disconnection, and which is deleted automatically when the client disconnects?

Connects to: leader election (modified Paxos), Z nodes and paths, sessions and heartbeats

Sessions and Heartbeats

Must-know: Clients request a session, requests within it are executed FIFO, and a session ID is assigned to the client. Heartbeats are the periodic liveness signal; the threshold time you configure decides when ZooKeeper checks whether a client is alive. ZooKeeper is not a single entity — it is an ensemble, and when the leader dies the modified-Paxos election runs again.

⚠️ Top pitfall: Assuming one ZooKeeper process is the whole system: even on a local development machine, an idle ZooKeeper eventually shows leader change behavior; the ensemble re-elects automatically.

Self-check: What happens to the session when heartbeats cease for longer than the threshold time?

Connects to: leader election (modified Paxos), Z node types, leader responsibilities and failed nodes

Watches: Change Notification

Must-know: A watch is the mechanism for notification about changes on ZooKeeper: the client registers what it wants to be notified about, and ZooKeeper tells it the moment something relevant changes. The analogy: like exception handling, a watch captures the context of the Z nodes — whether there is a change and the health of the nodes (slow or fast, up or down).

⚠️ Top pitfall: A watch fires once: after the notification, the client must re-register to keep watching the path; setting a watch also reads the node's data so notifications can be coalesced.

Self-check: How is ZooKeeper's notification different from polling?

Connects to: Z node types, read and write operations

Read and Write Operations

Must-know: Write path: client sends a write request (path + data) to a server → the server forwards it to the leader → the leader re-issues the write to all followers → success only if the responses come back successfully. Reads go directly to a server. CLI and API mechanics are 'good to know' but not asked from the exam perspective.

⚠️ Top pitfall: Thinking a write is done when the receiving server answers: no write is acknowledged until the leader has propagated it across the ensemble.

Self-check: Trace the write operation sequence from client to followers.

Connects to: atomic broadcast and fault tolerance, ZooKeeper guarantees

The Server Cluster, Atomic Broadcast, and Fault Tolerance

Must-know: ZooKeeper is a cluster of servers with a server leader; there is no direct client-to-leader edge. ZooKeeper is replicated for fault tolerance and is ordered — server nodes apply transactions in a consistent order. Writes go through atomic broadcast: broadcasting atomic data (a small, simple transaction) to every node as one indivisible unit. Availability rule: as long as the majority of the servers are available, the service is available.

⚠️ Top pitfall: Confusing broadcast with atomic broadcast: the atomically broadcast unit must be small and simple (few-byte transactions), matching the 1 MB Z-node design.

Self-check: Why does the service stay available when a minority of servers dies?

Connects to: leader election (modified Paxos), read and write operations, ensemble performance

Performance: More Servers Does Not Mean More Throughput

Must-know: Adding servers does not guarantee throughput — performance is not proportional to the number of servers. Reasons: (1) more nodes raise the probability of failure; (2) more nodes make the responses and overall coordination difficult. Replication buys fault tolerance but costs coordination overhead; size an ensemble for reliability, not raw speed.

⚠️ Top pitfall: Assuming doubling the nodes doubles throughput: the percentage of requests does not increase significantly, and the performance curve is not linear.

Self-check: Why does adding more servers not raise ZooKeeper throughput?

Connects to: atomic broadcast and fault tolerance, features of streaming data

Features of Streaming Data

Must-know: The four features of streaming data: high availability (service stays up despite failures — the majority rule), low latency (events processed quickly as they arrive), fault tolerance (failures survived via replication; state not lost), and horizontal scalability (scale by adding machines).

⚠️ Top pitfall: Expecting the coordination tier to scale linearly: horizontal scalability applies to the pipeline, while the coordination tier itself has a non-linear performance ceiling (Section 5.12).

Self-check: Name the four features of streaming data and the mechanism behind each.

Connects to: configuration and coordination basics, atomic broadcast and fault tolerance, ensemble performance

Leader Responsibilities and Handling a Failed Node

Must-know: The leader checks the health of each server, alerts the system on failure, and brings a replicated duplicate node into the active phase — 'kind of health coordination'. The leader continuously monitors state information; when a node fails it facilitates connecting the client to another node, but events sent into the failed node in the meantime can be lost. Delivery semantics: at-most-once tolerates losing a couple of events; sensitive data needs threshold response time configuration, increased node capacity, and client-side retry.

⚠️ Top pitfall: There is no client-side load balancer: requests are never diverted to another node by the client — forwarding or load balancing happens only based on the threshold response time configured for the node.

Self-check: What does the leader do when it detects that a node has failed?

Connects to: leader election (modified Paxos), sessions and heartbeats, atomic broadcast and fault tolerance

The Guarantees of ZooKeeper

Must-know: The five guarantees of ZooKeeper: sequential consistency (updates from a client applied in the order sent), atomicity (an update either succeeds or fails, no partial result), single system image (client always sees a single system view), reliability (once applied, an update persists until overwritten), timeliness (response within a certain time bound, set by the thresholds).

⚠️ Top pitfall: Assuming all reads are linearizable: ZooKeeper gives linearizable writes, but reads may be stale by default (a client must request sync before a read to see its own latest write).

Self-check: Which guarantee is backed by the FIFO session discipline, and which by the threshold configuration?

Connects to: sessions and heartbeats, read and write operations, atomic broadcast and fault tolerance

The End-to-End Pipeline and What Comes Next

Must-know: End-to-end: ZooKeeper coordinates with the collection layer, for example Kafka, which is a combination of collection plus the data flow layer; from Kafka the data goes to Spark. Next class starts the data flow layer, beginning with Kafka streaming. The quiz is not this week; it is planned after a couple of lectures per the schedule shown.

⚠️ Top pitfall: None specific; note the documentation link for the session lives inside the slide deck uploaded to the course folder.

Self-check: What does the next class start with, and when is the quiz?

Connects to: configuration and coordination basics, nodes, clients, and the data flow layer

Exam Guidance Summary

Must-know: The quiz may ask a property pertaining to the three types of nodes: persistent (alive even after the client disconnects), ephemeral (deleted automatically when the client disconnects; lifetime equals the client's), sequential (persistent or ephemeral with a sequence number appended). CLI and API mechanics are not asked from the exam perspective. The quiz is not this week; it is planned after a couple of lectures.

⚠️ Top pitfall: Spending revision time on CLI and API mechanics: the professor explicitly said these are 'good to know' and not examinable.

Self-check: Which node type's property would the quiz most likely ask about, and what is it?

Connects to: Z node types, read and write operations, end-to-end pipeline

Key Industry Applications

Must-know: Industry placement: ZooKeeper is the configuration and coordination system behind Hadoop, HBase, Kafka (metadata for servers and clients), Storm, and Samza; split brain is handled by replication (odd-count quorum trick); leader election uses a modified Paxos (Zab, with epoch numbers); production ensembles are sized 3 or 5; the stack runs collection (Kafka) → processing (Spark) with ZooKeeper coordinating.

⚠️ Top pitfall: Sizing an ensemble for throughput: more servers raise failure probability and coordination cost; ensemble size is chosen for fault tolerance (majority availability), not throughput.

Self-check: Which production systems rely on ZooKeeper, and what is the standard ensemble size?

Connects to: leader election (modified Paxos), atomic broadcast and fault tolerance, ensemble performance, leader responsibilities and failed nodes

Previous LectureLecture 04 · Stream Processing: Architecture and Processing Strategies

Next LectureLecture 06 · Message Delivery Semantics

Was this lecture useful?

Loading comments…

🤖

BitsNotes AI Assistant

Subject Notes Assistant

Configure AI Chat

Choose how to access the chatbot

Have your own API key?

Switch to "Bring Your Own Key" tab above for unlimited access with any OpenAI-compatible provider.

AI Provider Preset

API Endpoint URL (OpenAI-Compatible)

API Key

Model Name

🔑 Enter API key above to fetch live models from provider, or enter model name manually.

Remember key for this session (until tab closes)

OpenAI-Compatible API Support

Choose any provider preset (Gemini, DeepSeek, Kimi, GLM, MiniMax, Qwen, OpenAI, Groq, Ollama, etc.) or enter a custom endpoint URL.

Security & Privacy First

Your API key is sent directly from your browser to your specified provider. BitsNotes servers never store or see your key.