Responsible ML Engineering and LLMOps — End-to-End Ops and Quality Attributes

The Four Strategies Formalized

• Basic (recreate) strategy: Stop version $v_{\text{old}}$, then start $v_{\text{new}}$. Simplest to reason about, but incurs downtime during the swap. Suitable only when brief unavailability is tolerable.

• Ramped (incremental, rolling) strategy: Gradually shift capacity from old to new. In Kubernetes this is the default RollingUpdate. You increase replicas of $v_{\text{new}}$ step by step — for example from 1 pod toward 7 pods — while proportionally decreasing replicas of $v_{\text{old}}$ until the old pods are deleted. The system overrides the default only when you explicitly configure maxUnavailable or maxSurge.

• Blue-green strategy: Keep two full environments, Blue (live) and Green (staging). Green runs $v_{\text{new}}$ with production-like data and is fully warmed. Traffic switches from Blue to Green via a router or label change. Rollback is instant because Blue stays ready. Cost is double infrastructure during the switch window.

• Canary strategy: Release $v_{\text{new}}$ to a small, explicitly chosen subset of users or traffic (say 2–5%) first. Observe error rates, latency, and user feedback. If the canary sings — no anomaly — promote to 100%. If it fails, route that small cohort back and fix without broad impact.

MLOps defined. MLOps (machine learning operations) is the end-to-end lifecycle discipline for predictive AI — the world of regression (predict a number) and classification (predict a label). Where DevOps focuses primarily on code and infrastructure, MLOps must jointly manage three interacting entities: code (training scripts, feature pipelines), data (datasets, features, labels), and models (learned parameters and their encodings). A change in any one can silently degrade the others, so design, tracking, and evolution must address all three together.

Input data in this regime is often structured data — tables with named columns and typed rows. A canonical example is a CSV file from Kaggle. The Pima Indian Diabetes dataset is a classic binary classification example (predict diabetes yes/no from tabular clinical features). Regression datasets arrive in the same shape (predict a continuous target from tabular features). This shape matters: it dictates what preprocessing (imputation, normalization, encoding), validation (schema checks, distribution skew), and monitoring (feature drift) the pipeline needs.

The ops signal for predictive models. Once a model is in production, four classification metrics and one system metric form the first dashboard layer.

For a binary classifier, let $TP,TN,FP,FN$ be true/false positives/negatives.

• Accuracy — fraction of correct predictions:

$$\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}$$

• Precision — of the items we called positive, how many were truly positive:

$$\text{Precision}=\frac{TP}{TP+FP}$$

• Recall (also sensitivity) — of the truly positive items, how many did we find:

$$\text{Recall}=\frac{TP}{TP+FN}$$

• F1 score — the harmonic mean of precision ($P$) and recall ($R$), which balances false positives against false negatives more honestly than a simple average when one of the two is small:

$$F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN}$$

Every symbol is a count; $F_1\in [0,1]$ with 1 being perfect. It punishes a model that is precise but misses many positives, or one that recalls everything but floods with false alarms.

• Latency — wall-clock time to return a prediction for a request, $\text{latency}=t_{\text{end}}-t_{\text{start}}$ in seconds or milliseconds. A correct prediction that arrives too late is not useful in a live service (think fraud scoring before the transaction clears).

Pipeline as the product. A production MLOps workflow is orchestrated as a pipeline triggered from CI. Typical stages in order: data collection (ingest from source), data transformation (clean, normalize, encode, featurize), exploratory data analysis (EDA) and visualization (understand distributions, spot leaks), training (fit and validate), evaluation (hold-out and sliced metrics), deployment (publish model artifact), and monitoring (log predictions and drift). Each stage reports its own metrics, and the whole pipeline is versioned so you can reproduce any past run.

Tooling is modular:

• Experiment tracking: MLflow — log parameters, metrics, and model artifacts per run; compare runs on a dashboard. Used in the earlier MLOps demo.
• Managed pipelines: SageMaker Pipelines — define, schedule, and execute stages on AWS with lineage tracking (shown in a webinar demo).
• Orchestration (the conductor): Apache Airflow and Prefect — define the pipeline as a directed acyclic graph (DAG) of tasks and invoke it from CI; Kubeflow Pipelines — same idea natively on Kubernetes with containerized steps. Google Ngram was named as an available resource for language-adjacent exploration but is not a pipeline orchestrator.
• LLM-adjacent toolkit: LangChain — named as the primary LLM tooling for the session; it reappears centrally in LLMOps rather than in the tabular predictive path.

These orchestrators stitch data, code, and model steps into one runnable workflow and capture metrics at each stage, so a failure can be traced to the exact stage and commit that produced it.

The input shape flips from structured to unstructured. In MLOps a typical input is a CSV with typed columns. In LLMOps the input is unstructured documents — PDFs, PowerPoint decks, Word files, images, and other free-form content — where meaning is not in column positions but in language and layout.

The leading application pattern is a RAG pipeline (retrieval-augmented generation), which accepts almost any unstructured format. Two steps appear that have no counterpart in the tabular world:

1. Vector embeddings — an embedding is a dense numerical vector, say $\mathbf{e}\in {\mathbb{R}}^d$ with $d=768$ or $1536$, that encodes the meaning of a chunk of text so that chunks with similar meaning sit near each other in vector space under cosine or dot-product similarity.
2. Vector database — a store optimized to index and retrieve those vectors by nearest-neighbour search at query time.

A full RAG workflow therefore has its own pipeline shape: ingest unstructured documents, split (chunk) them, embed each chunk, store in the vector database, retrieve the top-$k$ chunks most similar to the query, and generate the answer by calling a language model with the query plus retrieved context. That entire workflow — documents, splitter settings, embedding model version, vector index, retriever, and generator — is versioned and monitored.

Where a language model is used it appears in two places: once as the embedding model and once as the final generation model. Either can be a small language model (SLM) or a large language model (LLM). Each call adds cost, latency, and a chance of hallucination (fluent but ungrounded text), which is why the metrics and knobs in the next two subsections matter.

Hallucination rate, bias, tokens, and latency.

• **Hallucination rate — central to LLMOps.* A hallucination* is a fluent answer that is not grounded in the provided context or facts. Measuring it was posed as an open question in the session. Two directions were named: human review, including human-in-the-loop at any stage, and a relevance / faithfulness metric that checks whether the answer stays true to the retrieved context. In today’s evaluations this is often operationalized as an LLM-as-judge or entailment check, but the session left the exact metric open.

• **Bias measurement — flagged as important. Bias can enter through the dataset (what documents you indexed), the model (what it learned in pretraining), and use** (how you prompt, retrieve, and present). The session deferred the deep treatment to the responsible AI portion rather than handling it fully under LLMOps.

• **Token usage — pay-per-use cost driver.** Total tokens for a request decomposes as:

$$\text{total\_tokens}=\text{input\_tokens}+\text{context\_tokens}+\text{output\_tokens}$$

where input tokens are the user's prompt, context tokens are the retrieved document chunks supplied alongside, and output tokens are the generated answer. Example from the session: the question "Who is the Prime Minister of India?" counts as input tokens, the answer "The Prime Minister of India is Narendra Modi." counts as output tokens, and a retrieved chunk counts as context tokens. Providers meter by tokens — calls to OpenAI, Gemini, and others are pay-per-use, and model tiers labelled nano or mini versus larger tiers carry different per-token prices. Token usage therefore maps directly to cost.

• **Latency — wall-clock generation time for a prompt.** Formalized in the demo as $\text{latency}=t_{\text{end}}-t_{\text{start}}$ in seconds.

The summary slide listed more metrics beyond these, but the live demo treated tokens and latency as the entry point — "the tip of the iceberg."

Temperature — determinism versus imagination.

Temperature, denoted $T$, is a sampling hyperparameter with

$$T\in [0,1]$$

where $T=0$ is fully deterministic (always pick the most likely next token) and $T=1$ is maximally generative under this simplified $[0,1]$ framing used in the lecture. In fuller textbook treatments temperature divides the logits before the softmax, but this lecture compresses the idea to the operating range.

• $T\in [0,0.5]$ — factual, deterministic behaviour. Arithmetic such as $2+2=4$ should return $4$, never a creative alternative.
• $T\in [0.5,1]$ — imaginative, generative behaviour. A poem, essay, or marketing slogan benefits from higher $T$.

Concretely: for fact-based tasks set $T=0.1$ or $0.2$ or even $0$; for open-ended creative tasks set $T=0.8$ or $0.9$; for a RAG pipeline where the answer must stay grounded in a supplied document, keep $T=0.1$ to $0.2$ at most so the model does not hallucinate beyond the provided context; for a catchy slogan, push $T$ toward $1$.

How this is set in consumer products has evolved. Programmatically we set $T$ ourselves in code based on the use case. On portals such as ChatGPT or Claude, behaviour around 2022 when public chat products first appeared typically defaulted to $T=0.5$, the "middle path" that balanced every prompt the same way. Today the setting is effectively prompt-adaptive and auto-selected: a prompt that says "this is a math problem, please solve it" leans the system toward $T=0.1$–$0.2$; a prompt that asks for a 1,000-line poem leans toward high $T$. The product infers intent and adjusts the hyperparameter behind the scenes.

Top-p (nucleus sampling) — breadth of the answer.

Top-p, written top_p in code and also called nucleus sampling, controls how much of the token distribution the model may draw from when no explicit length constraint is given. The lecture's plain-language framing: it governs breadth — whether the same question yields a terse one-line answer or a multi-paragraph elaboration.

• If you say "give me the answer in 50 words" you have set a hard length limit (an instruction, not top-p).
• If you say "give me the answer" with no limit, top-p governs the free-form breadth. A value such as $\text{top\_p}=0.7$ or $0.8$ or even $1.0$ lets in more of the distribution; smaller values narrow it.

Together, temperature and top-p are the primary knobs for controlling hallucination: lower and narrower settings reduce hallucination on factual and RAG tasks. In the session's shorthand: temperature sets the "factual versus creative" dial, top-p sets the "how much breadth" dial.

Visual — Sampling shape: Imagine the next-token probability distribution as a hillside. Low $T$ sharpens the peak — the tallest point dominates. High $T$ flattens the hill — many tokens share probability. Low top-p cuts the hillside at a narrow contour and samples only within that summit; high top-p includes the lower slopes. Takeaway: to stay factual, sharpen and narrow; to be creative, flatten and widen.

Scope and nuance: In production APIs temperature often ranges up to 2.0 and top-p is defined strictly as a cumulative-probability cutoff, not a word-count limit. The lecture compresses both to $[0,1]$ and to "breadth" language for intuition; the textbook definition — sample from the smallest set whose cumulative probability exceeds $p$ — is the precise form to carry into exams and docs.

Measured Application Metrics — Definitions

• Latency — elapsed wall-clock time per request:

$$\text{latency}=t_{\text{end}}-t_{\text{start}}$$

where $t_{\text{start}}$ is the time just before the model call and $t_{\text{end}}$ is just after the response returns. Unit is seconds. Sample run: $2.58$s.

• Question (input) token count — in this demo, one word equals one token. In code: question.split() then len(words). Punctuation is stripped so trailing ? does not create an extra token. Example: "What are virtual machines? Answer in 50 words." counts as 8 tokens when split on whitespace in this toy tokenizer (the session's logged JSON label: question_token_count = 8).

• Context token count — same word-split rule applied to the loaded Word document. Demo: context_token_length = 548. This stays fixed for every prompt as long as the same document is loaded and no caching or accumulation is applied.

• Response (output) token count — same rule on the generated answer. Demo: response_token_length = 285; the actual answer text was 40 words for the 50-word capped request.

• Total tokens and estimated cost:

$$\text{total\_tokens}=\text{input\_tokens}+\text{context\_tokens}+\text{output\_tokens}$$

$$\text{estimated\_cost}=\text{total\_tokens}\times \text{price\_per\_token}(\text{model tier})$$

The demo multiplies total tokens by a per-token price for the illustrative tier labelled GPT-5.4 nano in the log to produce a dollar/cent figure. The absolute number is hypothetical — the session cautioned not to over-interpret it — but the mechanism (sum of all three token families times price) is exactly what a billable provider does behind the scenes for every user and every prompt.

• System metrics (not captured in the trimmed demo): CPU usage, memory usage — important when a user uploads a 3–10 GB PDF/PPT and the model must scan large context.

The AgentOps idea. Where MLOps operates predictive models (regression, classification on structured tables) and LLMOps operates generative models (RAG over unstructured documents with token/cost/latency signals), AgentOps operates agentic systems — LLMs wrapped with goals, memory, and tools that act in an environment. The defining shift is from produce an output to pursue a goal through a sequence of tool-using steps.

The pattern emphasized across paradigms is that metrics differ by paradigm: what counts as a failure for a classifier (wrong label) is not what counts for a generator (hallucinated sentence) nor for an agent (wrong tool with wrong arguments that books the wrong flight). You cannot reuse the MLOps dashboard unchanged.

Three representative — and explicitly not comprehensive — metrics were named:

• **Goal achievement — did the agent accomplish the assigned objective?** Example objective: "book a flight from Bangalore to New Delhi." Success is end-to-end: the right flight is actually booked under the stated constraints. Partial progress (found options but did not book) is not success under this metric.

• **Task success rate — proportion of tasks that succeed versus fail across runs.** If the agent attempts $N$ tasks and $S$ succeed, $\text{success rate}=S/N$. This is the aggregate view of goal achievement over a workload, analogous to accuracy but at the task level.

• **Tool call correctness — did the agent call the right tool with the right arguments among the multiple tools it is allowed to invoke?** An agent may have tools such as search_flights, book_flight, cancel_booking, check_budget. Calling search_flights with {from:"BLR", to:"DEL", date:"2026-08-23"} when that matches intent is correct; calling book_flight before searching, or with a missing date, is incorrect even if the final answer sounds plausible.

These three were flagged as representative, not exhaustive — expect additional metrics around cost per task (tokens plus tool fees), latency to goal, safety/permission violations, and reproducibility to appear as the field stabilizes.

Tooling for building and operating agents. Frameworks that were named for AgentOps orchestration all carry ops-side features (logging, tracing, evaluation):

• CrewAI — orchestrate role-based agent crews with task handoffs and tool scopes.
• AutoGen — multi-agent conversation patterns with tool use and human-in-the-loop hooks.
• LangGraph — graph-based agent orchestration (stateful, cyclic workflows) built on LangChain, with checkpointing and observability primitives.

The same frameworks that help you build an agent increasingly help you operate it — trace every tool call, replay a failed trajectory, and score task success.

Quality attributes for ML. In classical software engineering the headline quality attributes are often scalability and performance. In ML engineering a parallel set applies, and the session made the stakes explicit: these attributes can cause real harm if ignored, and harm scales with model deployment.

Each attribute in this section — explainability, fairness, safety, security, privacy, and others — could fill two full sessions with tools and deeper techniques. This course compresses them into the final sessions to give a software engineering grounding for what each term means, why it matters differently in predictive, generative, and agentic AI, and how to reason about it before adding specialist tooling. The treatment here stays at a conceptual level plus textbook grounding, with the promise that future iterations will add more hands-on demos.

The textbook grounding for this responsibility portion is Chapters 23 through 28 of textbook one, which map to: responsible engineering, versioning/provenance/reproducibility, explainability, fairness, safety, and security and privacy. Students were asked what they already knew about these terms from other courses, and that prior knowledge shaped the discussion that followed.

The set of principles presented as the pillars of responsible AI in this lecture is: versioning, provenance, reproducibility, explainability, fairness, safety, security, and privacy. The session worked through the first three (15.5.5) in detail and left the remaining five to start from the next session, noting that about five minutes remained and that explainability would open the next class.

A Harm Taxonomy for ML Systems

• **Safety — physical harm to people.** Three examples were given: a delivery robot that blocks a wheelchair user at a crosswalk, a self-driving car that misinterprets a stop sign and causes a crash, and medical device software that delivers an incorrect dosage. This connects back to the self-driving Apollo case study from the second session. In safety analysis, the object of harm travels with the human body.

• **Manipulation and addiction — exploiting how we interact with software.* The infinite scroll and autoplay patterns on Facebook and Instagram were the opening example. The scroll keeps users engaged for hours by recommending new posts based on addiction mechanisms. A design detail was stressed: when smartphones first appeared the dominant scroll was left to right, but most apps shifted to top-to-bottom, which studies cited in the session link to longer hooked sessions. Personalized ads are a second example: the system learns that a user shops frequently at a sports retailer and then exploits that vulnerability to drive impulsive buying. The mental model to keep straight: addiction is the user's hooked state; manipulation* is the intended design choice that creates it.

• **Polarization and mental health — division and distress as side effects.** Polarized news and social feeds can divide people into opposing ideological camps, feeding each side content that hardens the split. Students were reminded not to fixate on one country — the pattern is global. Excessive exposure is linked in the discussion to anxiety, depression, and low self-esteem, even when the surface metric ("more engagement") looks positive. Students were invited, without any political framing, to share examples they had seen in the chat.

• **Job loss, weapons, and surveillance — scary, fast-moving, deliberately not dwelt on. One example named was mass facial recognition in railway stations and other large crowds. Contrast was drawn: facial recognition for attendance inside a college campus is bounded and consent-scoped, while city-wide mass recognition tracks people across spaces without meaningful consent. An agentic example was also sketched: autonomous robots deployed in the field** that must make their own decisions when connectivity drops in difficult terrain. Giving a system permission to operate autonomously without a network link creates risks that are hard to bound.

• **Discrimination — given its own treatment because it is often unintentional and therefore easy to miss.* Polarization is intentional — someone wants to split opinion. Discrimination can happen even when no one intends it*, because biased data or flawed models silently produce unfair outcomes. The canonical example was the Amazon automatic hiring system (circa 2013). Training data at that time consisted mainly of male profiles with very few female profiles. When resumes were processed the model selected about 78% of male applicants and rejected many highly qualified female applicants — not because of any explicit rule, but because the model had not seen enough female examples to learn from. That is discrimination via data bias and model flaw, not via intent. More examples were promised in later slides.

Responsibility spans all levels. A definition attributed to IBM was shown and students were asked to consider: is responsible AI the job of an individual, an organization, a country, or the world? The answer emphasized was all levels at once. Whether we work as a developer, tester, product owner, or researcher, and whether we ship predictive, generative, or agentic AI, each of us is individually responsible — as is the organization, the country, and collectively the world.

In software engineering terms these responsibilities are quality attributes that sit alongside functionality. The pipeline and metrics we saw for 14 sessions deliver functionality (does it predict, generate, act correctly and quickly?); this section asks whether that functionality is also explainable, fair, safe, secure, and private when it reaches real users.

The eight pillars listed for responsible AI in this lecture are:

$$\text{versioning}|\text{provenance}|\text{reproducibility}|\text{explainability}|\text{fairness}|\text{safety}|\text{security}|\text{privacy}$$

The session worked through versioning, provenance, and reproducibility in detail (next subsection) and left explainability, fairness, safety, security, and privacy to start from the next session, noting that about five minutes remained and that explainability would open the next class.

Version four things, not one. In classical software we version code (day to day, Git). In ML we must version four things together: code, dataset, model, and pipeline. The reason is traceability under dispute. You need to reconstruct exactly what happened, not what happens today when you re-run today's artifact.

The loan decision example and the Apple Card controversy (2019) made this concrete. A person submits data to a lending model; the model returns approve or deny. Suppose a highly qualified applicant — strong income and credit history — is denied and brings a legal challenge. To investigate, you must replay the decision as it happened.

• In a classical software system there are two variables: the database content and the application code. Go to production, reproduce the fault, trace to a line of code or stored procedure.
• In an ML system there are at least three to four moving parts. In the loan example, three interacting models together scored risk and limit and then produced the final decision, each reading different inputs like income or credit history and applying its own algorithm. Data sources, model versions, and the pipeline that wired them together can all have changed since the decision. You may be on model version 20 today while the decision used model version 11.1. Re-running today's model on today's data proves nothing. You need the pipeline that pins the same data source version as it looked on that date, the same model versions as they were then, and the same pipeline wiring — all in sync.

Strategy 1 — Store the full copy per version. Version 1 is the full $500$ GB snapshot on 22 August. On 23 August $5$ GB of new data arrives so version 2 is the full $505$ GB snapshot, and so on. Any dated complaint maps directly to one full snapshot you can retrieve. The drawback is storage cost: keeping a full copy for every version is memory-intensive and only workable with abundant storage. Implementable as a naming convention (customer_profiles_v435.csv.gz, customer_profiles_v436.csv.gz) and mirrored internally by Git's copy model.

Strategy 2 — Store the delta (diff/patch). Keep one base version with $500$ GB and each day store only the change (say $+5$ GB per day via diff/patch). To reconstruct version 3, combine base plus day-one delta plus day-two delta plus day-three delta. To diagnose a problem tied to version 1, apply only the day-one delta to the base. Space-efficient when changes are small relative to the base; retrieval cost grows with the number of patches to apply. Internally this is how RCS and Mercurial work. Tooling: diff/patch, versioned object stores.

Strategy 3 — Offsets in append-only data. Suited to real-time event data such as logs, clickstreams, and IoT events including Kafka topics. Such data is append-only: records are added and rarely deleted. Think of an IoT temperature sensor in New Delhi writing every hour: 12:00, 1:00, and so on over years. Versioning is by offsets in the event stream: time zero is the start, $1,000$ events mark version 1, $2,000$ events mark version 2, and so on. To answer "what was the temperature five days ago at this hour" map the request to the offset range covering that window and read from the stream. Boundaries need not be day-based; they can be count-based depending on generation speed — fast-generating data such as Instagram posts hits $1,000$ events quickly while slow-generating data such as census data takes longer. The pairing is always offset plus timestamp so a named version maps to a concrete slice and the consumer reads until that offset. Implemented in streaming systems like Apache Kafka (offsets) and lakehouse tools like lakeFS/Dolt. This pattern does not generalize to datasets where deletion is expected; that is what event sourcing (amending by appending) handles.

Strategy 4 — Change history per individual record (row-level versioning). Suited to entity-oriented data such as products or customers. Instead of versioning the whole dataset, version each record's history. A customer record with a customer ID is illustrative: version 1 might hold {balance: 1200, status: success} on a given date; a transfer of $300$ rupees creates version 2 with {balance: 1500, status: success}; a later failed operation creates version 3 with {balance: 1500, status: failed}. Every change becomes a new version for that customer. You can then query any customer at any point in time. Implementable as per-record files under Git, per-key versioning in key-value stores (Amazon S3 buckets version per key), or database edit-history features. Mirrors the "change history per record" idea in DVC and lakehouse per-row audit logs.

Strategy 5 — Version the pipeline (holistic, preferred for ML). The pipeline itself becomes the versioned artifact and it pins the versions of everything inside it. The example pipeline shown combines raw transactions at version $3$, EDA/ETL/feature-engineering code at version $5$, configuration at version $2$, and training dataset at version $7$. The assembled pipeline is then labelled version 12. Any output of that pipeline can be regenerated by re-running pipeline version 12, which pulls exactly those pinned versions. This is why MLOps platforms, including SageMaker, show a version for the entire pipeline, not just for code. Trade-off is compute time: re-running over a large dataset may take minutes or longer. Advice: prefer pipeline versioning for analytics and ML applications while still understanding the other four patterns and when each applies.

Provenance is source trust; lineage is movement and transformation.

• **Data provenance — source and trust.** Where did this data come from, who created it, who modified it, when, and what is its history? The data could be internal or external (even public sources like Twitter data). Trust, integrity, and authenticity hinge on provenance. Key questions: What source produced this row? Who created or changed it and when? Provenance answers those, often requiring row-level edit history and authentication for every create/edit.

• **Data lineage — movement and quality drift through a pipeline.** Lineage tracks the data as it passes through systems and stages: created → extracted → transformed → loaded → staged → consumed. It answers: was the data fine when raw but broken after some transformation?

Canonical lineage example: An IoT sensor reports temperature correctly as $90^{\circ }$C as raw data. A transformation stage that should convert Celsius to Fahrenheit contains a bug. Correct conversion is

$$T_F=T_C\times \frac{9}{5}+32$$

so $90^{\circ }$C should become $90\times 1.8+32=194^{\circ }$F. The buggy code instead does $T_C+32=122^{\circ }$F or drops the scaling entirely. The raw value was correct; the Fahrenheit value downstream is wrong for every consumer table. Without lineage you do not know where quality broke. With lineage you trace stage by stage, identify the conversion code as the culprit, enumerate which tables and consumers now hold the wrong values, and fix the right place.

Reproducibility defined. Reproducibility — can we recreate an ML experiment with the same code, same data, same configuration, same dependencies, and same environment and achieve consistent results? Pipeline versioning is the mechanism: with the same data source version, same feature-engineering code version, same configuration, same training dataset, and same library pins, the same outcome should be reproducible even if we run the pipeline 100 times.

Many things break reproducibility: different Python library versions, a different dataset version as new data is appended, a different train-test split (say 70/30 in pipeline v11 versus 80/20 in v12), changed preprocessing logic, and different hyperparameters such as temperature and top-p and their interaction with growing datasets.

The cost of reproducibility is again compute time: reproducing requires re-running the pinned pipeline over large data, which may take minutes or longer and must be budgeted, just as with Strategy 5 above.

Reproducibility versus replicability (exam-useful distinction): Replicability (exact bit-for-bit identical results from the same experiment) is a stricter subset of reproducibility (comparable results under minor variations, such as the same method on fresh data). Determinism of data transforms and model inference helps replicability; controlling library versions, data versions, and seeds helps reproducibility. The textbook notes that strict bit-for-bit replicability of models is rarely the goal in production — some nondeterminism in training is accepted and versioned.

Temperature decides whether reproducibility is even desirable.

• In predictive AI reproducibility is straightforward. Given the same input, same model version, and same pipeline version, expect the same output every time. Non-determinism here is a bug.

• In generative AI the same input can legitimately produce different outputs by design when we are on the generative side of the temperature scale. If $T=0.1$–$0.2$, the output is deterministic and factual and the same prompt should give the same answer (and a RAG pipeline grounded in a fixed document should be reproducible under pinned retrieval and $T$). If $T=0.8$–$0.9$ and we ask for a catchy marketing phrase, asking twice should give two different phrases — variation is the point. In that regime insisting on identical text misunderstands the goal.

• In agentic AI the same pivot applies. "Book me a flight from Bangalore to Delhi" with no constraints run today versus five days later may choose Air India one time and Indigo the next because prices, schedules, and API state have changed and the prompt is open-ended. With pinned constraints (budget, evening only) and pinned external factors as far as possible, reproducibility returns in the sense that the same constrained request should produce the same booking. After days have passed not all external conditions can be pinned, so strict sentence-level reproducibility is not the right expectation.

The deeper point: versioning, provenance, and lineage remain important for generative and agentic systems — you must version the data source, the vector database (embedding model + index), and the agent code and tool permissions — even when strict output reproducibility is not expected. Provenance and lineage are intact even when the text varies.

Explainability asks whether a prediction is intelligible. Explainability is the quality attribute that asks: can we trace why the system predicted what it predicted, in terms that a relevant stakeholder — developer, auditor, or affected user — can understand and act on?

The textbook treats explainability and accountability together: the ability to explain is what makes it possible to hold someone accountable for a decision. The bank loan decision is again the anchor. Functionality delivers a decision — approve or deny. Explainability requires that the system can articulate why that decision follows from the inputs — which features pushed it over the threshold, what rule or learned pattern fired, and what would need to change for a different outcome.

Why this is not a side topic:

• Interpretable versus opaque models. A decision tree can be read as a path: "income > 80k and credit score > 720 → approve." A random forest aggregates many such paths and remains somewhat inspectable. A model like XGBoost (gradient-boosted trees) or a deep network can be far more accurate but is harder to explain — the trade-off between performance and intelligibility is itself a design decision that the Q&A opened.
• Domain demand. In finance, clients and regulators routinely ask "how did you reach this conclusion?" and expect traceability, not just a score. The same expectation appears in healthcare (why this diagnosis?), hiring (why was this resume rejected?), and any high-stakes allocation.
• Field maturity. Explainability is established enough to have its own global venues — the World Conference on Explainable AI was cited as evidence — and its own textbooks and tooling ecosystems. It is not a passing concern but a sub-discipline with methods ranging from inherently interpretable models to post-hoc explanations (feature attributions, counterfactuals, local surrogates).

Single-turn fixes the boundary; multi-turn grows it.

• **Single-turn interaction** — the mode of the LLMOps demo. You load one Word document as context, ask one question, get one answer, and the transaction ends. The knowledge boundary is fixed to that document load. Token accounting is simple: $\text{context\_tokens}$ is a constant (548 in the demo) added once per request. Cost and latency are predictable.

• **Multi-turn interaction — the mode typical of ChatGPT. Turn one may include a PDF; turn two says "summarize that"; turn three says "but also use this new file I just uploaded." The system carries forward prior context and prior reasoning, so the boundary grows. $\text{context\_tokens}$ increases with every turn, which grows cost and latency and pressures the model's context window** (the maximum tokens the model can attend to at once). Caching, compression, and window management become first-class concerns.

Two ways to grow the boundary, one preferred:

1. Naïve concatenation — context = context + new_document in code. Explicitly appending another upload into the same session. Works for a quick demo and makes the trade-off visible: more information improves coverage but also grows tokens and can push older relevant chunks out of the window.

1. Robust architecture: full RAG pipeline — splitter → embedding model → vector database → retriever → generator. Instead of carrying the whole concatenated history, retrieve only the top-$k$ most relevant chunks for the current query. That caps $\text{context\_tokens}$ at roughly $k\times \text{chunk\_size}$ rather than letting it grow with every turn. The same pipeline, together with its pinned versions (document version, splitter settings, embedding model, vector index, retriever policy), is what LLMOps tracks.

Cost implication directly: In single-turn with the demo numbers, a second question costs another $8+548+\approx 285=\approx 841$ tokens. In naïve multi-turn where you append a second 548-token document, the second turn costs $8+(548+548)+\approx 285=\approx 1,389$ tokens — a 65% jump just from carrying history. With RAG retrieval capped at, say, $k=4$ chunks of $128$ tokens each, the second turn costs $8+512+\approx 285=\approx 805$ tokens — actually cheaper than naïve single-turn while staying more relevant.