Intelligent Agents and Problem Solving

2.1 The Four Definitions of AI

2.1.1 The Four Quadrants and the Shift to Acting Rationally

The course continues where the previous session stopped: the four classic definitions of artificial intelligence, usually drawn as four quadrants. Thinking humanly means creating a system that mimics the way humans actually think. Acting humanly means creating a system that mimics how humans behave. Thinking rationally means doing what is logically correct. Acting rationally means doing the action that gives the best outcome. All four definitions are legitimate — different research schools each believe their quadrant is the real AI and the only AI — but in the last few years the consensus has shifted strongly toward acting rationally. The realization behind that shift: the goal of an AI system is not just to mimic humans. Acting rationally is the more promising goal, and that is the definition this course builds on.

The four quadrants come from combining two independent questions, each with two answers. First: should the AI match human performance, or match an abstract rational standard? Second: does "intelligence" live in the internal thought process, or in the external behavior we observe? Cross those two dimensions and you get exactly four cells.

Visual intuition: the quadrant grid. Picture a 2 × 2 grid. The horizontal axis runs from thought (left) to behavior (right); the vertical axis runs from human (top) to rational (bottom). The top-left cell is thinking humanly, top-right is acting humanly, bottom-left is thinking rationally, bottom-right is acting rationally. As you move around the grid, the methods change too: the two "human" cells are empirical sciences that need psychology-style observation, while the two "rational" cells are mathematics-and-engineering pursuits that connect to statistics, control theory, and economics.

Each quadrant is easier to grasp through an everyday example.

2.1.2 Thinking Humanly and Thinking Rationally

Thinking humanly. Imagine a human seeing something and saying, "this looks similar to one I saw yesterday — I think the answer is around 50." That answer came from emotions, habits, or past experiences, not from any logic. If an AI starts behaving this way — reflecting human-like thinking that is driven by emotion and memory rather than reasoning — that is thinking humanly.

In the research world, this quadrant is called the cognitive modeling approach: to make a program that thinks like a human, you first need a theory of how humans think, and you gather that theory in three ways — introspection (watching your own thoughts as they pass), psychological experiments (watching people solve problems under controlled conditions), and brain imaging (watching the brain in action). Once the theory exists, you express it as a program; if the program's input–output behavior matches human behavior, that is evidence the same mechanisms may be at work inside people. The classic example is the General Problem Solver (GPS) built by Allen Newell and Herbert Simon: they did not just check that GPS solved problems correctly — they compared the sequence and timing of its reasoning steps with those of human subjects doing the same problems. This meeting point of AI and experimental psychology is the field of cognitive science.

Thinking rationally. Suppose all assignments are due tomorrow. You deduce: "they must be completed by today, because only then can I submit." Then you add: "I also have an AI assignment due tomorrow, so I must complete it today." This is step-by-step reasoning: a chain of if-then deductions, pure logic, independent of emotions. If an AI behaves this way — holding a set of rules and reducing statements through logical reasoning — that is thinking rationally.

This is the oldest approach of all. The philosopher Aristotle tried to codify "right thinking" with syllogisms — argument patterns that always produce correct conclusions from correct premises. The canonical example: "Socrates is a man; all men are mortal; so Socrates is mortal." Nineteenth-century logicians turned such arguments into precise notation, and by 1965 programs could, in principle, solve any solvable problem expressed in logical notation. The researchers who hoped to build intelligence this way are called the logicist tradition.

2.1.3 Acting Humanly and Anthropomorphism

Acting humanly. A chatbot says, "that sounds stressful — want to talk about it?" It is not thinking like a human internally; it simply acts like one, showing empathy and initiating a conversation. This behavior is genuinely debatable, and it returns late in the course when ethics are discussed. The concern: some AI agents are deliberately tuned this way so that humans become more dependent on them, telling the AI about their day, their problems, and their joys. This is connected to anthropomorphism — giving human-like characteristics to non-humans. Pet parents name their pets and dress them; similarly, a group of people already treats AI like a human, using it as a therapist. Real-world: chatbot companions that encourage exactly this dependence are a known product pattern.

The historical anchor for this quadrant is the Turing test, proposed by Alan Turing in 1950 as a way around the vague question "Can a machine think?" Instead of debating thinking, you run an experiment: a human interrogator types questions and reads written replies, and if the interrogator cannot tell whether the replies come from a person or a computer, the computer passes the test. Passing a rigorously applied version requires a lot of machinery — natural language processing to converse, knowledge representation to store what it hears, automated reasoning to draw conclusions, and machine learning to adapt. A total Turing test goes further and also demands perception and physical manipulation — computer vision and robotics — because the interrogator can then interact with objects in the real world.

2.1.4 Acting Rationally and the Historical Arc

Acting rationally. To reach the office quickly, you check traffic on a maps app, choose the fastest route, and if walking will not do, you book a cab and go. You act to maximize your goal — reaching faster — regardless of how you think internally. Acting rationally is about doing what gives the best outcome, and it is the quadrant that has taken over.

The recap also re-anchored the historical arc: roughly 78 years of AI history from Alan Turing's question "Can machines think?" through the early workshop era to "Attention is all you need." Many fields have contributed — psychology, computer science, economics, mathematics — and applications now touch almost every field. The four-quadrant framework itself grows out of a long deliberation about what intelligence is, whether humans are always intelligent, and whether non-humans like dogs exhibit intelligence.

Real-world connection. Acting rationally is the quiet engine behind nearly every modern AI product. A navigation app chooses the route with the best expected arrival time; a chess engine searches for the move that maximizes its chance of winning; a recommendation engine picks the item that maximizes your expected engagement. None of them claim to think like a human — they are built to produce the best outcome, which is exactly the definition that now leads the field. In the next section, we turn this definition into something concrete: the agent, the entity that does the perceiving and the acting.

2.2 Agents: Perceiving and Acting

2.2.1 The Agent Definition: Sensors and Actuators

The natural next question after "act rationally" is: act rationally how? The answer starts with the agent — the entity that does the acting. An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators. Perception comes first: only if the agent knows what is around it and what is happening can it act sensibly. In simple terms, any agent has two functions: perceiving and acting. To perceive it uses sensors — the input channel. To act it uses actuators — the output channel.

Think of the agent as a worker wearing a headset: the headset's microphone is the sensor (it hears instructions), the speaker is the actuator (it replies), and the office around the worker is the environment. The analogy holds as long as we remember that the agent only knows what comes through the headset — anything not picked up by the microphone simply does not exist for the agent.

2.2.2 Human, Robotic, and AI Agents

A human agent illustrates this directly: our sensory organs — eyes, ears, and so on — are the sensors; our body parts — mouth, legs, hands — are the actuators. A robotic agent uses cameras, infrared, and range finders as sensors to know its environment, and motors of various kinds as actuators to act on it. An AI agent is the modern version: an intelligent entity powered by large language models (LLMs), capable of perceiving its environment, reasoning about goals, and executing actions. The same perceive-and-act skeleton applies — it senses its environment, processes information, makes decisions, and performs actions aligned with achieving the assigned objective. The new emphasis, especially in the recent wave of reasoning models, is on how the AI got to the goal, not just on the action itself.

Software agents fill the same skeleton without any body. Their sensors read file contents, network packets, and human input from a keyboard, mouse, touchscreen, or voice; their actuators write files, send network packets, and display text, images, or sound. The environment of a software agent is the computer's data and network — a perfectly valid environment even though it is not physical.

2.2.3 AI Agent versus Agentic AI

This course treats "agent" as a rational agent in general, not as any specific embodiment. The distinction still matters in industry, so it is worth fixing the vocabulary: the difference between an AI agent and agentic AI is the level of autonomy. An AI agent is a single bot trained on past FAQs and learned logic, following predefined scripts: it answers within its scope and nothing more — beyond its control it can only say "do you want to talk to a human agent?" When such an agent is wrapped in software — an input box, a chatbot — that packaging is an AI assistant; the agent is the core part actually powered by the scripts and the LLM. Agentic AI is a network of AI agents that not only answers FAQs but also escalates issues, analyzes customer sentiment, adapts responses, and coordinates with back-end systems — autonomously. That needs real software engineering: orchestration, pipelining, tool calling, and protocols.

The same contrast applies one step up the ladder. A typical AI agent is a single agent with predefined scripts, learned logic, its own memory and databases; it just behaves, and it does no tool calling and no talking to other agents. Agentic AI has more autonomy: a network of agents that can escalate, adapt to sentiment, coordinate with back-end systems, and contact other agents — which is why it needs orchestration and pipelining, and why technologies like tool calling, the Model Context Protocol (MCP), and agent-to-agent protocols exist.

A natural worry when you hear "the agent runs on scripts": does the agent need its own data before it can act? The answer is no — the intelligence lives in the underlying model, not in a dataset attached to the agent.

If "anything that perceives and acts is an agent," then the definition must also include people — which turns out to be exactly what the theory wants.

2.2.4 Examples of Agents

Agents come in many embodiments. Examples include physically grounded agents, intelligent buildings (which can perceive the weather and act on it), autonomous spacecraft, expert systems, and today's trending AI tools (which perceive what you type and take actions from it). Anything that perceives and acts is an agent.

2.2.5 The Agent Function and the Agent Program

Under the hood, every agent contains an agent function — the mapping from percept histories to actions. The agent perceives something in the environment, and something inside the agent translates that percept into an action; that translation is the agent function. The agent program is what runs on a physical architecture to produce that function. In the simplest equation: agent = architecture + program. There are different ways to build this internal mapping — goal-oriented, utility-oriented, and others — which the agent architectures section covers in depth.

Real-world: virtually every commercial app you chat with — customer support bots on food delivery apps and similar — is technically an AI agent: trained on past FAQs, scripted, low autonomy. The industry is now moving up the autonomy ladder with agentic AI, MCP, and agent-to-agent communication.

2.3 The Vacuum Cleaner Agent

The classic worked example of everything above is a vacuum cleaner agent with exactly two rooms, A and B. It is deliberately tiny, but it shows the full picture: what the agent can perceive, what it can do, and how percepts drive actions.

2.3.1 The Setup: Rooms, Percepts, and Actions

The setup. The agent can perceive four things: A,clean (room A is clean), A,dirty, B,clean, and B,dirty. It can perform four actions: move left, move right, suck, and no-op (no operation — doing nothing). Not every action is legal in every state: from room A the agent cannot go left, because the room ends there; from room B it cannot go right. The legal choices from A are "go right" or "suck"; from B they are "go left" or "suck"; if both rooms are clean it does no-op and keeps quiet.

2.3.2 The Percept–Action Table

The percept–action table. The heart of the example is a tiny table mapping each percept to an action:

The table is only a partial tabulation of the agent function. The full table would need a row for every percept sequence the agent could ever experience — for example, a third reading of A,clean, or any longer history — which is why a real agent program can never store the whole table but must compute the answer from rules.

2.3.3 Waiting: Repeated Percepts

The first two rows are straightforward: if A is clean, go right to inspect B; if A is dirty, suck. The interesting rows are the repeated percepts. Perceiving A,clean once does not mean "immediately move right." The agent can be designed to wait — one reading is not enough. On the first A,clean it does no-op and keeps waiting; on the second A,clean it decides the room is genuinely clean and moves right to B. The professor's verbal account: "the first time when it saw clean, what did it do? No-op — it kept waiting. Let me not quickly hurry and go to the other room. Let me wait for some more time." Likewise, if the second reading is A,dirty, the agent should not go to the other room — it should first suck the dirt. You can push this further and configure the agent to require three clean readings (clean, clean, clean → right), and if a dirty reading appears in between, suck instead. The behavior is fully configurable by the developer.

2.3.4 Student Questions and Answers

That correction fixes the vocabulary: percepts are what the agent observes, actions are what the agent does. "No-op" never appears in the percept column, only in the action column.

And what happens when the agent stops waiting and starts moving? The same rule-based logic applies on the other side of the room.

The point of the whole slide: based on the percept, an agent takes actions; some actions are legal and some are illegal in a given state; and the agent's behavior is decided by its developers. So far this is just "an agent." The next question — what makes such an agent rational — is where the real definition of AI intelligence begins.

2.4 Rational and Ideal Rational Agents

2.4.1 The Rational Agent and Its Performance Measure

Recall the fourth quadrant: the AI should act rationally. A plain agent only perceives and acts; a rational agent (also called an intelligent agent) strives to do the right thing, based on what it can perceive and the actions it can perform. The right action is the one that will cause the agent to be the most successful. For the vacuum cleaner: if there is dirt, the right thing is to clean it; if the room is clean, the right thing is not to sit idle forever — it is to go check the other room; from B the agent should not move right, because there is no room to the right. An agent consciously designed to strive for the right thing will succeed.

But how does the agent know what "the right thing" is? We have to decide first, for every agent, what its performance measure is — an objective criterion for the success of the agent's behavior. For the vacuum cleaner, candidate measures include the amount of dirt cleaned up, the amount of time taken, the amount of electricity consumed, and the amount of noise generated. Notice the judgment embedded in the first one: if there is dirt and the agent does not perceive it, or perceives it and does not clean, it is not doing the right thing — so "amount of dirt cleaned" captures that. Performance alone is not enough, though: we need a full specification of performance, environment, actuators, and sensors — the PEAS specification in the next section.

What counts as rational at any instant depends on four things: the performance measure that defines success, the agent's prior knowledge of the environment, the actions it can perform, and its percept sequence to date. In the 1980s people built agents this way: fully rule-based systems where you program everything and the agent matches certain rules. That is still an agent — just the simplest kind of internal design.

2.4.2 The Ideal Rational Agent

The ideal rational agent. The definition to learn carefully: for each possible percept sequence, an ideal rational agent does whatever action is expected to maximize its performance measure, on the basis of the evidence perceived so far and its built-in knowledge. Three words here, and they mean different things. An agent perceives and acts — simple. A rational agent strives to do the right thing, where "right" is configured either by humans or by self-learning, and comes with a specification. An ideal rational agent knows its possible percept sequences and its possible actions, and picks the one that maximizes its performance measure — possibly optimizing several measures at once (time taken, electricity consumed, and so on). Critically, the choice is based on the evidence perceived so far and the built-in knowledge given to the agent at creation. It can maximize one performance or multiple.

A useful human analogy: at any point in life, with a stable mind, we take the decision we expect to maximize our potential based on that day's knowledge. The phrase "I could have done better" makes no sense in hindsight: had I known more, yes — but I did not know that. An ideal rational agent can likewise go wrong in the future and fail to maximize its performance; you still cannot blame it, because at that moment, given what it had perceived and its built-in knowledge, it took the action it expected to maximize the measure. Whether it really maximized it or not is not the question. If it behaves in this fashion, it is an ideal rational agent.

2.4.3 Rationality versus Omniscience

Rationality is distinct from omniscience. Omniscience means all-knowing, with infinite knowledge. Humans do not have omniscience — we have finite knowledge — and neither does any real agent. Rationality only demands the best action given that finite knowledge.

2.4.4 Acting to Gather Information

Agents can act to gather information. Agents may perform actions to modify future percepts so as to get useful information — exploring before deciding, to expand the "knowledge so far." You have seen this in bot interactions: you give ChatGPT a long task, and it replies with "here is what I have perceived, here is what I can do, can you give me this more information?" or it presents a plan of action first. It is exploring, gathering a little more information so that its answer pleases you — and pleasing you is nothing but maximizing its performance. Agents can be tuned to do exactly this.

The everyday version is crossing a busy road: it would not be rational to step out without looking, because the percept sequence from not looking tells you nothing about the truck approaching. The rational action is to look first — performing an action whose purpose is to shape future percepts — and then decide. Looking is information gathering in exactly the sense defined here.

2.4.5 Autonomy and the Possibility of Ideal Agents

Autonomy. An agent is autonomous if its behavior is determined by its own experience, with the ability to learn and adapt. Are ideal rational agents possible in practice? The honest answer sits in between: you cannot claim 100 percent possibility, but it is the direction the field is heading — in real-world settings, where things are uncertain and we cannot observe everything, we usually end up with good but suboptimal solutions rather than truly optimal agents.

Real-world connection. Modern AI systems live inside this definition. Recommender systems gather information (your clicks) before acting, then pick the recommendation expected to maximize engagement — their performance measure. Self-driving cars constantly look (sensor information gathering) before acting, then choose the action with the best expected safety and legality. Even ChatGPT's clarifying questions are information gathering in the textbook sense: the model asks before answering because more percepts raise its expected performance. The next section turns the four ingredients of this definition — performance, environment, actuators, sensors — into the PEAS checklist used to design any agent.

2.5 PEAS Specification

2.5.1 The Four Components

When designing any agent, the first step is to fill in its PEAS — the standard framework for specifying an agent: P for performance measure, E for environment, A for actuators, and S for sensors. Sensors are for input — observing, like eyes, nose, and ears for humans. Actuators are for action — like hands, mouth, and legs. The framework is used constantly in agent design: for any agent you should be able to propose the performance measures, the environment it works in, the actuators it acts with, and the sensors it perceives with. There is no single exact answer — different valid answers are all correct — but you cannot be radically wrong: for a factory robot, an environment that has nothing to do with the factory, the conveyor belt, and the parts would simply be wrong.

2.5.2 Worked Example: The Automated Taxi

2.5.3 Worked Example: The Medical Diagnosis System

2.5.4 Activity: The Part-Picking Robot

Activity: the part-picking robot. On a conveyor belt, many parts move past and a robot picks them up, arranges them, and sorts them by color.

Notice how the answers share a pattern with the taxi and the diagnosis system: the performance measure is stated as observable outcomes (percentage in correct bins), the environment is the physical setting, the actuator is the moving mechanism, and the sensors are the perception devices. Once you can fill all four for a new agent, you have its task environment specified.

2.5.5 What the Developer Decides

After listing the possibilities, the developer decides: which performance measure to optimize, what type of environment it is, which actuators and sensors to use, and what each costs. For software agents this shrinks dramatically: sensors reduce to text, voice, or Braille input, and actuators reduce to whatever engine produces the output — text, image, voice, or video. The actuator is not the output itself; it is the mechanism that produces it — the robotic arm in the factory, or an LLM API or image generator in software.

A compact way to remember the whole framework: every agent uses its sensors in its environment E to optimize its performance P using its actuators A. The PEAS specification belongs in the documentation of a system — the world need not know about it in the release, but the documentation should carry it.

Real-world connection. PEAS is the standard way industry documents an AI product's contract. A food-delivery bot's PEAS, for example, is: performance — resolved issues and customer satisfaction; environment — the chat window and the order database; actuators — the reply generator; sensors — the incoming message text. RoboCup teams write PEAS for their robots before writing any code. It is also the natural bridge to the next topic: once P, E, A, and S are fixed, the environment — the E — carries its own properties that decide which agent design will work, and those properties are the subject of the next section.

2.6 Properties of Environments

PEAS tells us what the agent is and where it works; now the environment itself deserves a close look. Actuators and sensors are low-level hardware discussions — for software agents they reduce to input and output, text or multimedia — but the environment has a set of properties that determine how the agent must be designed. Six properties matter: observability, deterministic versus stochastic, episodic versus sequential, static versus semi-dynamic versus dynamic, discrete versus continuous, and single-agent versus multi-agent. The environment type largely determines the agent design.

2.6.1 Observability: Fully, Partially, and Unobservable

Can the agent's sensors give it access to the complete state of the environment at each point in time? If yes, the environment is fully observable. A chess or tic-tac-toe game is fully observable: at any point the agent knows where all the pieces are, what it has moved, and what the opponent has moved. The other extreme is completely unobservable: a completely dark room with zero visibility, where the agent has no information at all. In the middle sits partially observable: a partially lit room, or a poker game, where you know your own cards but not the randomly assigned cards of the other players. Autonomous driving is partially observable too.

An unobservable environment is an extreme and rare case, but it exists: when an agent's sensors fail, it has no percept at all and no knowledge of what is going on — a software agent that cannot take input has no information. In the AI world it happens when sensors fail, actuators are failing, or the database is crashing.

When the environment is not observable, every other property stops mattering — sequential, continuous, stochastic, all of it. An unobservable environment leaves only one action: make it observable. That is the only reactive mode the agent should enter — contact a human, escalate, do whatever restores at least partial observability.

2.6.2 Deterministic versus Stochastic

The next state of the environment is deterministic when it is completely determined by the current state and the action executed by the agent. A calculator is deterministic: the same action always produces the same result, like a function call. The environment is stochastic when there is uncertainty in the next state — the weather changes regardless of what the agent currently observes or does, and a robot trying to kick a goal in football operates in a stochastic environment because other players can intervene. In simpler words: the same action leading to the same result is deterministic; the same action leading to different results is stochastic.

One more push on the chess example, because it trips up many students: the outcome of a game depends on the opponent too, and the opponent's replies are uncertain — but that uncertainty lives in the opponent, not in the action.

2.6.3 Episodic versus Sequential

An episodic environment divides the agent's experience into atomic episodes, and the choice of action in each episode depends only on that episode. A spam email filter classifies one email as spam or not, and the story ends; give the agent one image and ask "is there a cat or not?" — that episode has nothing to do with the next one. This resembles a session in software: whatever you do is independent. A sequential environment is the opposite: current decisions affect future decisions or rely on previous ones, so long-term planning is needed. Pac-Man, an investment portfolio bot, and even chess are sequential — every decision you take now impacts your future moves.

Memory is where the operating system world inspired a classic line of research: how does an OS manage memory when RAM is limited (say 16 GB) but the disk is huge (say 500 GB)? Virtualization, paging, cache memory — the same ideas have been applied to agents: can an agent keep a short-term memory, a cache, and manage out-of-the-box context? Tools like LangChain and LangGraph now handle exactly this agent-memory management. Knowledge graphs are another interesting line in the same direction.

2.6.4 Static, Semi-Dynamic, and Dynamic

A static environment is unchanged while the agent is deliberating or thinking. You think about your next move in a Rubik's cube or a crossword puzzle; the cube does not rearrange itself while you think. A dynamic environment changes while the agent thinks — it is raining, or traffic signals change while you are driving. You wanted to move past, then the light turned red, so you had to stop: because the environment changed, your thinking and your action had to change. Semi-dynamic sits between: the environment itself does not change with time, but the performance score does. Chess with timing is the example — the board does not change while you think, but your score depends on how fast or late you make each move.

2.6.5 Discrete versus Continuous

A discrete environment has limited, countable states — clear, defined percepts and actions. Chess again: all moves are finite and concrete; the number of combinations may run into millions, but the states are discrete values, and each piece has only discrete moves. A continuous environment has infinite possibilities. Driving a taxi involves speed, position, and angle — all continuous variables with continuous values. The real world, with weather and climate, is continuous.

2.6.6 Single-Agent versus Multi-Agent

Is the agent operating by itself in the environment, or are there other agents present? Other agents are anything that changes from step to step or can sense and act. In the software world this property maps directly onto the earlier vocabulary: a single agent is most probably an AI agent — a bot trained on a bunch of things that can do one thing and not beyond that; a multi-agent setup is most probably agentic AI, because the agents work together, orchestrate, and accomplish common goals — like the robots on a car manufacturing line, all working together.

2.6.7 The Real World: Applying All Six Properties

Now the payoff question: what is the real world like? Go through the properties one by one. Is the real world fully observable? No — it is partially observable: if it were fully observable, we would all be far better informed about everything. Deterministic or stochastic? Life is never deterministic; there is a lot of probability — the real world is stochastic. Episodic or sequential? Sequential — an action taken today impacts tomorrow, and yesterday's decision impacts today; if it were episodic, we could forget the past and restart every day, which is not possible. Static or dynamic? Dynamic — while you deliberate, the world around you keeps changing: a new tool arrives, an action you take influences your whole career. Discrete or continuous? Continuous — the states you can be in are not countable. And the real world has multiple agents — you are not the only one out there.

Reading a table of environments row by row helps fix the combinations. Taxi driving: partially observable, multi-agent (other vehicles), stochastic, sequential, dynamic, continuous. A crossword puzzle (or an English tutor): fully observable, single agent, deterministic, sequential (what you fill now affects later moves), static (nothing changes while you think), and discrete — a defined number of states, one of which is the correct answer to the puzzle.

2.6.8 Activities: Deep Blue and Robot Football

2.6.9 Additional Student Questions

Exam note: interesting inferences can be drawn between these properties — for example, an environment that is stochastic cannot simultaneously be deterministic, and a fully observable, small environment behaves very differently from a large, partially observable one. Deductions of this kind between the properties are exam-relevant, so practice reasoning across the whole set rather than memorizing each property in isolation.

Real-world connection. Every AI product lives in one of these environment profiles, and the profile dictates the engineering. A self-driving system is built for a partially observable, stochastic, dynamic, multi-agent world — so it needs sensor fusion, prediction, and safety fallbacks. A chess engine gets a fully observable, deterministic, static world — so it can use exhaustive search. A spam filter gets an episodic world — so it classifies each email with no memory. The environment properties are not classroom trivia: they are the specification that selects the agent design, which is exactly the topic of the next section.

2.7 Agent Architectures: Filling the Question Mark

The agent diagram shows an environment, an agent, and a question mark between "perceived via sensors" and "act via actuators." What is the magic in between? The question mark can be filled in many ways — those are the different agent types, or agent architectures. The first attempt historically was the table lookup agent: in the 1970s, when this line of work started, people kept a table of percept sequences, and the agent simply looked up the current percept in the table and returned the mapped action. It is as simple as a search through a huge list: exact match found, take that action. The drawbacks are severe: a huge table is needed, it takes a long time to build, there is no autonomy (it is pure matching), even with learning the table entries take long to learn, and if something does not match, the agent cannot do much. Table lookup was the starting point, not the destination.

After it come the five canonical agent types: simple reflex, model-based reflex, goal-based, utility-based, and learning-based. Each adds a capability to the previous one — that is how they evolved, not a hierarchy, but a useful ladder.

2.7.1 The Table Lookup Agent

Already covered above: percept in, action out, via a lookup table. It is the simplest possible filling of the question mark, with no internal state and no learning.

2.7.2 The Simple Reflex Agent

The simple reflex agent interprets its input and matches it against condition-action rules — plain if-else logic. The vacuum cleaner rules are exactly this: if the status is dirty (this came from the percept — the agent sensed the room is dirty), then return "suck"; else if the location is A, return "right" so you can go to the B room; else if in B, go left. Interpreting the input, doing the rule matching, returning the action — done. The drawback is obvious: how many if-else conditions will you write for a real agent? This tiny agent has a few possible actions, so if-else works; for a real agent it does not.

2.7.3 The Model-Based Reflex Agent

The model-based reflex agent keeps more than rules: it also tracks how the world evolves and what my actions do — all captured in an internal model. Whatever it senses gets written back into its state: the previous state, the state before that, and so on. This gives the agent an idea of how the world is evolving and some past data, and the model guides the action. Driving with blind spots is the everyday version: whenever you go through that stretch, you explore the environment, and that knowledge goes into your model, so the next time you have information about that area.

2.7.4 The Goal-Based Agent

Add a goal, and you get the goal-based agent: it knows what it would be like if it did action A — because the model lets it simulate consequences — and it picks the action that moves toward its goal. The goal comes from the PEAS specification: the performance part defines what the agent was optimized for, and the goal is fed into "what action should I do now." If an action takes the agent toward the goal, take it; otherwise take another. Example: navigation from place A to place B — the agent has a goal, and the percepts plus the model guide it there.

The contrast with reflex behavior is fundamental. The reflex agent brakes when it sees brake lights — period; it has no idea why. The goal-based agent brakes because that is the action its model predicts will achieve the goal of not hitting anything. The goal-based design is more flexible: change the destination and the goal changes, and the same machinery plans a new route — while a reflex agent's rules for when to turn would all have to be rewritten.

2.7.5 The Utility-Based Agent

On top of the goal sits utility — how happy the agent will be in a state, how much utility it gets from doing something. The utility-based agent wants the best outcome, not just any path to the goal. Going from home to office is a goal; any route gets you there. But the utility-based agent asks which route gives the best experience: a long route with traffic and frustration makes you unhappy; a simpler, faster route makes you happy. So it optimizes the choice, not just the arrival. Utility maximization is still goal-oriented — that is why utility builds on top of the goal: you want to achieve the goal, but with the best possibility.

2.7.6 The Learning Agent

The learning agent adds a critique: it improves based on feedback. The structure has a learning element that produces learning goals, a performance element that acts, and a critic that evaluates the result; a generator can even make the performance element select suboptimal actions so that the agent has a chance to learn, and the learning element then selects a better action next time.

The braking example makes it concrete. Input percept: a red traffic signal. The performance element is triggered; the output is "press the brake." Now suppose the resulting state is bad: the car should have stopped before the zebra crossing, but it came too far. That is a negative critique — exactly like reinforcement learning. At 50 meters the agent performed a 17K Newton braking force and went way past the zebra crossing, so it got a negative critique.

The next time it faces the same percept — 50 meters away — the agent does not repeat the mistake: it adjusts the force, the model updates, and the behavior improves. That is learning.

2.7.7 Student Questions and Answers

Speed is one axis; structure is another. The ladder order tempts many students into thinking the types are ranked — they are not.

Since the types are combinable building blocks, the practical question is which ones a real product needs.

And because each rung of the ladder does more work, the response time grows as you climb.

Correlate the ladder with humans: are we rule-driven? In some cases, yes — fire will hurt, so we pull our hand back; simple reflexes are hard-coded from childhood (with research even debating whether they pass through DNA). Are we model-based? Also yes — you have not explored part of a city, you go there, details go into your mind, and the next time it guides you; people telling you the Taj Mahal is crowded in the summer is your model guiding your percepts. Goal-based: you have a goal to reach and percepts and the model guide you there. Utility-based: route optimization. Learning-based: recommendations that learn what you like.

Real-world connection. Every agent product is a mixture of these five designs. Recommendation engines (Netflix, music apps) are everyday learning agents: a critic scores your engagement, and the model improves with what you like. Navigation apps are everyday utility-based agents: they evaluate routes by expected happiness — time, traffic, tolls — not just by arrival. Simple reflex rules survive in safety-critical microcontrollers — a factory robot's emergency stop is pure condition-action logic. Agentic AI architecture in general is a much bigger software layer on top of these fundamental agent types, but this course stays at the fundamentals.

2.8 Problem-Solving Agents and Search Problems

A large share of early AI work was about problem-solving agents (PSA): formulating a problem as a search problem. A surprising number of problems can be modeled this way and then solved by search. The recipe has three steps. First, formulate: define the problem and its components. Second, search: solve the problem using search algorithms. Third, execute: run the solution as a fixed sequence of actions.

The pseudo-code of a problem-solving agent works like this. It has a problem (the problem formulation), a goal (initially null), a state (a description of the current world state), and a sequence of actions. When a percept arrives, the agent updates its state with this new information. If the action sequence is empty — nothing has been done yet, this is the first time — the agent creates the goal, formulates the problem, and searches. If a sequence is found, it sends back the next action; otherwise it reports failure.

2.8.1 What a Problem-Solving Agent Is

A problem-solving agent takes a goal and turns the world into a search problem: start at some initial state, apply actions, check for the goal. Not every problem can be modeled this way — but most can, and that is where the field started: people cared about modeling a problem as a search problem (initial state, goal state, transition, possible states, cost) and then applying search techniques on top. Every search technique has its own pros and cons.

2.8.2 The Five Components of a Search Problem

Every search problem has five components: the initial state, the possible actions (also called operators) that the agent can apply, the successor function — given an action, where will the agent reach? (also called the transition model) — the goal test, which checks whether a state is the goal, and the path cost. A path is a series of actions, each action carrying a cost that depends on the problem; the path cost is a function assigning a numeric cost to each path — the minimum duration, the money involved, or any performance metric you choose. A solution is the path plus its cost; an optimal solution reaches the goal with the least cost.

2.8.3 State, State Space, Path, and Solution

A state is all the information about the environment needed to make a decision for the task at hand — not "fully observable," but the whole set you know about the environment now (partial observability, stochasticity, dynamism included). The state space is the set of all states reachable from the initial state: I am here, where can I go from here. The state space forms a graph in which the nodes are states and the arcs between nodes are actions. In that graph, a path is a sequence of states connected by a sequence of actions — and the state-to-state transitions carry costs. The solution of a problem is a path in this graph from the initial state to the goal state. The state-space representation is the basis of most AI methods.

2.8.4 Worked Example: Arad to Bucharest

Abstraction is part of the formulation. In reality, the agent in Arad has many possible states — it might be in the middle of the city, near the airport, at a shopping center, in a hotel — and many possible actions: it can halt, take a detour, turn on the radio, and so on. All of that is irrelevant to finding the path to Bucharest, so we abstract it away. The formulation keeps only the details that matter for the goal; that is why problem formulation is a process of deciding what actions and states to consider given the goal. The abstraction must be valid — every real-world state that counts as "in Arad" must be able to reach some state that counts as "in Sibiu" by driving — and it must be useful — the abstract action "drive from Arad to Sibiu" must be easy to carry out without further search. Remove as much detail as possible while keeping both properties.

2.8.5 Worked Example: The Vacuum World Formulation

2.8.6 Worked Example: The 8-Puzzle

Search, then, is the process of looking for a fixed sequence of actions that reaches the goal, and a search algorithm is what takes the problem as input and outputs that sequence. For most problems — tic-tac-toe, games, route planning, many more — formulate them as search problems, apply a search algorithm, and the result is what people call intelligent software.

2.9 Uninformed and Informed Search

2.9.1 Uninformed Search

Search strategies come in two broad families: uninformed and informed. Uninformed search uses only the information available in the problem — there is no extra information about states beyond what the problem itself provides. It includes breadth-first search (BFS), depth-first search (DFS), uniform-cost search (UCS) — a variant of Dijkstra's algorithm, which the algorithms community calls Dijkstra's algorithm and the AI community calls uniform-cost search — and their variants. These are blind searches: they start working with no extra knowledge, and they are slow. The name "uninformed" says it all: no added guidance, just the graph.

2.9.2 Informed Search and Heuristics

Informed search has strategies that know whether one non-goal state is more promising than another — the extra knowledge is called a heuristic. If you add guidance — heuristics — the blind search becomes informed. A caution about the word "intuition": heuristics are intuition, not knowledge. Your intuition is not always correct, and informed search algorithms are not always optimal (some are, but not all). Still, an intuition is better than nothing: you are no longer searching blindly, you are guided by extra information. The informed family includes best-first search, A*, AO*, and more — A* and best-first search are the starting points in the next session.

2.9.3 Study Advice and Exam Notes

One philosophical note from the material: "Let your intuition guide you — you are what you have been looking for." The point is practical: a good heuristic is a guide, and often our intuition is exactly what helps.

Recap and bridge: search families are chosen by what information they may use — none (uninformed: BFS, DFS, UCS) versus an estimate of remaining distance (informed: best-first, A*, AO*). Heuristics are intuition, not knowledge, so informed search is usually faster but not always optimal. The next session replaces the question mark: it takes these two families and turns them into algorithms with precise rules — best-first search and A* — and shows how the heuristic quality controls how quickly the goal is found.

Real-world connection. The uninformed/informed split shows up in every route-finding product. A navigation app never runs blind BFS over the road network — it runs an informed search with a straight-line-distance heuristic, which is why directions appear in milliseconds. The same idea powers puzzle solvers, game AI, and logistics routing: the smarter the heuristic (distance, traffic prediction, delivery deadlines), the less of the search space the algorithm has to examine.

Exam Guidance Summary

Assessment structure. The first evaluation component (EC1) carries 30 marks. It has two quizzes of 5 marks each, and the best score of the two is the one that counts; each quiz is individual, runs for about 30 minutes to an hour, and is open for a window of roughly five days. There are also two group assignments worth 25 marks in total — 12 marks for the first, 13 for the second — both mandatory (no best-of); each has about fifteen days of time, and both assignments share one fixed group of exactly five people, subject-specific and self-chosen when group formation is enabled. Later components: EC2 has a regular exam and a makeup exam on different dates, and so do the final exams. Bookmark the announced dates so nothing is missed — announcements come when each item goes live.

Where the marks live in the content:

• PEAS specification — being able to produce performance measures, environment, actuators, and sensors for any given agent is worth one mark in the mid-sem assessment. Practice on the agents given in the exercises (taxi, medical diagnosis, part-picking robot, and others).
• The four definitions of AI — all four are legitimate definitions; the consensus definition the course builds on is acting rationally. Be ready to identify which quadrant a given behavior belongs to.
• Agent vocabulary — agent vs AI agent vs agentic AI differ by the level of autonomy; the reasoning emphasis of modern AI agents; and the skeleton of sensors, actuators, agent function, and agent program.
• Environment properties — for any agent, be ready to list observability, deterministic/stochastic, episodic/sequential, static/semi-dynamic/dynamic, discrete/continuous, single/multi-agent. Deductive inferences between the properties are explicitly exam-relevant — for example, an environment that is stochastic cannot be deterministic.
• Agent architectures — the five types in evolution order and what each adds; optimize (the end result) vs improve (the agent itself); the reflex agent as the fastest.
• Problem-solving agents — the five components of a search problem (initial state, actions, transition model, goal test, path cost) and the canonical worked examples: vacuum world, Arad to Bucharest, 8-puzzle.
• Uninformed vs informed search — not directly examined as data-structure algorithms (BFS and DFS come from the data structures course), but you must know the difference: uninformed works only on problem information, informed adds heuristics.
• Vocabulary precision — agent vs rational agent vs ideal rational agent; AI agent vs agentic AI (autonomy); optimize (the result) vs improve (the agent); deterministic (same action, same result) vs stochastic. These distinctions are exactly where exam questions probe understanding.

Study advice. Solve the PEAS and environment-specification exercises offline and compare answers in the discussion forum; use the textbook's interactive search animations to build BFS/DFS intuition; the textbook carries the canonical examples (vacuum world, Arad to Bucharest, 8-puzzle) used throughout this session. The exercises are practice, not marks — the point is to be able to do the two tasks cold: give a PEAS for any agent, and classify any environment.

Key Industry Applications

• AI agents in every commercial app — customer-support bots (for example, a food delivery app like Swiggy) are textbook AI agents: trained on past FAQs, scripted, low autonomy, limited to their predefined scope.
• Agentic AI and the orchestration layer — networks of agents that escalate issues, adapt responses, and coordinate with back-end systems; the ecosystem around it includes tool calling, the Model Context Protocol (MCP), and agent-to-agent protocols.
• ChatGPT, Gemini, and Claude — their "thinking" phases and planning-before-answering modes are the reasoning emphasis of modern AI agents in action; they also ask clarifying questions before long tasks, an example of agents acting to gather information.
• Agent memory management — LangChain and LangGraph manage short-term and long-term agent context out of the box, borrowing ideas from operating-system memory (virtualization, paging, caching).
• Deep Blue vs Garry Kasparov — the chess machine; its environment classification (fully observable, deterministic, static, discrete, sequential, multi-agent) is the canonical example.
• RoboCup robot football — multi-agent robotics in a stochastic, dynamic, partially observable environment.
• Autonomous taxis — the PEAS example of the session: safety, speed, legality, comfort, and profit as performance measures over roads, traffic, and pedestrians.
• Medical diagnosis systems — PEAS in healthcare: healthy patient, minimized costs, fewer lawsuits; screen display as actuator, symptom entry as sensor.
• Factory part-picking robots — conveyor-belt robots sorting parts by color; PEAS with robotic arm, camera, and joint-angle sensors.
• Intelligent buildings, autonomous spacecraft, expert systems — classic examples of agents that perceive and act in physical or information environments.
• Recommendation engines — everyday learning agents that improve from your behavior; navigation apps — everyday utility-based agents choosing the best route; financial bots — sequential decision-makers; weather prediction — the canonical stochastic environment.

The through-line across all of these: every product is an agent with a PEAS specification, living in an environment with a known property profile, built from the fundamental agent architectures, and increasingly coordinated through agentic-AI orchestration.