Agentic Architecture & Orchestration

Koko: A weak prompt gives you a weak answer. That's annoying. A weak architecture double-charges a customer, loops forever, or burns your entire compute budget overnight. That's a production incident. That gap — annoying versus catastrophic — is exactly why Agentic Architecture and Orchestration is the biggest domain on this exam, a little more than a quarter of it.

Chris: And when I mapped out the domain I kept noticing the same theme underneath every subtopic — it's all about control and recoverability, not raw capability.

Koko: That's the one instinct. Say it one more time so it sticks: make the system predictable and recoverable, not just clever. Everything else in this domain is a specific way to cash that out.

Chris: So the analogy I keep using for myself: it's like the difference between a contractor who says 'trust me, I'll figure it out on site' versus one who shows up with a project plan, named milestones, and a way to pause and resume if the lumber delivery is late.

Koko: I love that. The lumber-delivery contractor is exactly right. Named steps, checkpointed state, a clean handoff protocol. The 'trust me' contractor might be more talented, but you cannot audit him, you cannot recover when something goes sideways, and you definitely cannot bill the client accurately.

Chris: And the exam is essentially asking: which design survives real traffic? Not which design sounds most impressive in a pitch.

Koko: Exactly. So let me make this concrete. The instinct cashes out into five specific things the exam rewards. First: decompose the work into named steps — not one giant prompt, a sequence with identifiable stages. Second: checkpoint state so a run can resume if it fails halfway. Third: make every step safe to retry, meaning idempotent where you can. Fourth: compose agents into something like a clean org chart — clear roles, clear reporting lines. And fifth: bound the risky edges — guardrails that fail closed, and autonomy granted only in proportion to risk.

Chris: That last one is interesting to me. Autonomy proportional to risk. So a step that only reads data gets more latitude than a step that writes to a database or sends an external API call.

Koko: Right. Low stakes, you can let the model have more room. High stakes — money moving, external messages going out — you tighten the leash. The architecture mirrors the risk profile.

Chris: What's the recurring wrong-answer pattern I should watch for? Because I want to recognise it fast.

Koko: The trap is that 'let the model decide' and 'give it full autonomy' sound more sophisticated — like you're trusting the AI, which feels like the advanced move. The exam puts those options right next to the controlled design specifically because they're tempting. But for a stable production system, handing the model more freedom and hoping is almost always the wrong answer.

Chris: So the option that says something like 'allow the agent to determine its own workflow dynamically' is the one to be suspicious of.

Koko: Suspicious is the right word. It's not always wrong — there are contexts where dynamic planning is appropriate — but when the stem is about stability, reliability, or production readiness, predictable beats impressive. That's the one instinct. Hold onto it for every subtopic we cover, because it will be the discriminator again and again.

Koko: Okay, let's talk about the loop itself — because this is where a lot of architects go wrong before they even get to tools or memory. An agentic loop takes a big goal and walks it down a fixed, ordered list of smaller named steps until there is nothing left to do. The analogy I keep coming back to is a recipe card. You don't hand a recipe card that says 'cook dinner.' You write: chop, sauté, simmer, plate. You are done when the last line is checked off — not when it feels right.

Chris: Right, and the exam version of that is something like a nightly research assistant. Instead of 'analyze this company' in one call, you break it into gather financials, pull peer data, fetch transcripts, draft insights, synthesize — and after each step there's a pointer asking what's next. When that pointer returns nothing, the loop ends.

Koko: Exactly. The loop terminates because the plan runs out — not because the model decided it was done. That distinction is the whole instinct the exam is testing. Fixed ordered list, explicit next-step pointer, explicit stop condition.

Chris: So what's the discriminator between this and just giving the model a big prompt and letting it figure out the steps?

Koko: Observability, resumability, and budget. With a fixed list you can checkpoint each step, you can resume after a crash, and you can count steps to stay under cost limits. The trap here is 'let the model decide each step dynamically at runtime' — it sounds more agentic, and the exam will make it sound appealing, but you lose all three of those properties for a stable pipeline. Save open-ended reasoning for genuinely open-ended tasks.

Chris: That leads naturally into stop reasons, which I think of as the control signal for the loop. Every reply comes back with a label — finished its turn, wants a tool, hit a length limit, hit a stop sequence, refused, context ran out. Your loop reads that label and branches on it.

Koko: Perfect summary. I call it the printer analogy — a printer tells you 'out of paper' versus 'job complete.' You react to the status code, not by squinting at the half-printed page trying to guess whether it finished.

Chris: So the branching logic is: tool-use means run the tool and re-enter the loop; length means continue or raise the token budget; end-of-turn means you're actually done.

Koko: That's it. And the trap here is treating every response as final text. If you do that, you silently drop tool calls — the model asked for a tool, you ignored it and shipped whatever prose came back. You also ship truncated answers as if they were complete when the stop reason was a length limit. The status code is the control signal. Read it.

Chris: Okay, third piece — durable run state. If each step is a separate execution, the agent has no memory between steps unless you put it somewhere. So you keep a single notebook in external storage, one labeled slot per step, keyed by a run id, and you write to it after every finished step.

Koko: Painting a fence one plank per visit. You tick off which planks are done, you can leave and come back all day, and you never lose work. The concrete exam example is a long audio render that would blow past a platform's execution time limit — you split it into short blocks, save each finished block immediately, and an external caller re-invokes the function until every block exists, then stitches.

Chris: And the key phrase there is 'external caller re-drives the loop' — the platform isn't holding a long-lived process open.

Koko: Exactly, because platforms routinely reclaim background tasks seconds after the response is sent. The trap here is two flavors of the same mistake: 'keep results in memory and write once at the end' or 'run it as a long-lived background task.' Both lose everything on the first crash. Durable-and-incremental beats in-memory-until-the-end. Every time.

Chris: So the instinct is: externalize state to durable storage keyed by a run id, update after every step, and do a bounded slice per invocation that stops safely below the platform limit.

Koko: That's the whole thing. Fixed plan, read the stop reason, write state after every step. Those three together are what makes a loop observable, resumable, and budgetable — which is exactly what the exam means when it asks you to design a stable agentic pipeline.

Koko: So here is a question that trips up a lot of architects: what do you do when a network blips mid-step and you have no idea whether the step finished?

Chris: You retry — but you need the step to be safe to run twice, otherwise you risk charging a customer twice or writing a duplicate record.

Koko: Exactly. That property is called idempotency: the step produces the same end state whether it ran once or five times. Think of an elevator button — pressing it ten times does not summon ten elevators. One press, one elevator, done.

Chris: So the concrete mechanism is: gate the expensive work behind an already-done check, and make every write a keyed upsert — so a retry just refreshes the same record in place.

Koko: That is it exactly. And there is a phrase worth committing to memory for the exam: at-least-once delivery plus an idempotent handler equals effectively-once. The network is always going to deliver at least once, and sometimes more. Your job is to make the extra deliveries cheap no-ops.

Chris: The instinct is to make retries harmless rather than to prevent them — because preventing them is not actually possible on a real network.

Koko: Right, and the trap here is an answer that says 'add a lock so the step can never run twice' or 'disable retries entirely.' Those options sound safe but they break your system the moment the network hiccups. Retries are your resilience strategy — you just need an idempotent handler to receive them.

Koko: Now zoom out to the full multi-agent picture. When you scale from one agent to many, what does the org chart look like?

Chris: Strict hierarchy. One orchestrator scopes the job and delegates to specialized doer sub-agents. Then separate adversarial validators check the doers' output before anything ships. The orchestrator delegates; validators only inspect.

Koko: Perfect. The analogy is a general contractor: they hire subcontractors for plumbing, electrical, framing — and then an independent building inspector reviews the work. The inspector flags problems but never quietly fixes them. That independence is the point.

Chris: And the gate is an AND — you ship only when every validator passes. Not a majority, not most. All of them.

Koko: Because the author is the worst-placed party to catch its own errors. That is why the validators have to be separate agents, read-only, and told to default to fail and actively try to refute. Independent work fans out in parallel; you sequence only on real data dependencies.

Chris: What is the discriminator on which agent holds the delegation tool? Because that feels like something the exam will probe.

Koko: Only the orchestrator gets delegation. The option sitting next to it on the exam will say 'give every agent the delegation tool for flexibility' — that sounds generous but it turns a governable tree into an ungovernable web and destroys your audit trail. One node delegates; everyone else operates within their lane.

Chris: And 'let the writer validate its own output' is not adversarial — it is just self-review, which is not the same thing.

Koko: Exactly. No adversarial separation, no quality gate.

Koko: Last piece: something fails in your fleet. What is the first move?

Chris: Classify the error before you react. The category tells you what to do. Transient — timeout, retry with backoff. Validation — bad input, fix or reject, do not retry. Business — a legitimate no from a downstream system, surface it. Permission — missing access, escalate, never route around it.

Koko: That is the full taxonomy. The analogy is emergency-room triage: you sort by type before you treat, because the treatment depends entirely on what you are dealing with.

Chris: And only transient errors are retryable. A validation error will still be a validation error on the fifth retry.

Koko: The trap here is the blanket retry — wrap everything in one catch block, retry on all failures. It burns calls on errors that will never fix themselves and it obscures what actually went wrong.

Chris: The sub-agent's error response also needs to be structured — not a bare exception. What it was doing, why it failed, any partial results, a suggested alternative. That gives the orchestrator something to reason with.

Koko: Right — a confident hallucination in place of a failure signal is worse than a bare exception. Classify, return structured context, let the orchestrator degrade gracefully on the non-retryable ones. That is the whole discipline.

Koko: Alright, let's talk about what happens when things go wrong at the edges — because the exam loves to test whether your safety net is actually safe. First concept: fail closed. If a guardrail protecting a costly or irreversible action is misconfigured or uncertain, it denies the action — it does not allow it through. Think of your home's fuse box: a fault cuts the power. The fuse does not say 'I'm unsure, let's just stay on.'

Chris: Right, so the classic example is an expensive AI endpoint. If the access secret was never configured, the endpoint refuses to run — it does not fall back to running wide open.

Koko: Exactly. And the second half of this is independence. Your spend or loop kill switch has to live out of band — meaning a separate monitor, not something wired inside the agents it watches. That's the circuit breaker mounted outside the appliance it protects.

Chris: Because if the monitor is inside the agent, a bug in that agent can take down both the behavior and its own safety net at the same time.

Koko: That's it. Independence is the load-bearing word on the exam. The trap here is two-sided. One: 'let requests through if the secret isn't configured yet so setup isn't blocked' — that's fail-open, and you've just exposed the endpoint to the whole internet. Two: 'have each agent check its own spend and stop itself' — the agent fails together with the very guard it was supposed to obey. The instinct is: when in doubt, deny — and the kill switch lives somewhere the agent cannot reach.

Koko: Next: hooks versus prompts. A prompt instruction is a strong suggestion — the model usually follows it. A hook is code that fires on a lifecycle event and forces or blocks an action regardless of what the model decided. The analogy I like: a polite sign asking people to badge in versus a turnstile that will not physically turn without a badge.

Chris: So for something like 'every write is scanned for secrets before it goes through' — that has to be a hook on the write event, not a line in a system prompt.

Koko: Exactly right. The hook blocks an unscanned write. The prompt just asks nicely. What's the discriminator you'd use to decide which one to reach for?

Chris: Whether it's an invariant — an 'always' or a 'never.' If it's load-bearing, it goes in deterministic code on an event. If it's judgment — nuance, context-sensitivity — that belongs in the prompt.

Koko: Perfect. Invariants in hooks, judgment in prompts. The trap here is 'word the instruction more forcefully and repeat it.' Prompts are best-effort. No amount of bold text turns a suggestion into a guarantee.

Koko: Last one for this section: autonomy tiers. The rule is you give an action exactly as much rope as its risk justifies. Read and compute steps can run autonomously. Anything that writes to a real system is human-gated by default, and where that gate sits depends on materiality.

Chris: The example in my notes is a classifier that reads thousands of documents with a cheap model and only escalates a single case when two cheap signals disagree — or when the action crosses a materiality threshold. So you're routing cheap and fast by default.

Koko: Right, like a junior who handles routine screening and kicks anything alarming up to a senior. Cost-aware escalation — that's the whole point. And there's a rollout dimension too: a new agent goes shadow mode first, then progressive rollout, then hybrid. Never a straight flip to full autonomy.

Chris: That staged ladder makes sense. The option sitting next to it on the exam will say 'grant full autonomy once accuracy looks high in testing' — which skips the ladder entirely.

Koko: And the other trap: 'route everything to the strongest model to be safe.' That sounds responsible but it throws away cost-aware escalation entirely. The instinct is: tie autonomy to risk and materiality, escalate only on a trigger, and climb the ladder one rung at a time.

Koko: Let's put everything we've covered to work with two exam scenarios. First one: a system that assembles a research report. Several specialist agents fan out, gather information, and then a writer agent composes the final report. Sounds clean. Here's the design question — who gets to spawn helpers, and who checks the draft before it ships?

Chris: Okay, so the naive version probably hands delegation to everyone — each specialist can spin up its own sub-agents if it needs to.

Koko: And the exam will absolutely offer that. The trap is calling it 'flexible.' What it actually is: an ungovernable web with no audit trail. If every agent can delegate, you have no single point that knows what's running or why.

Chris: So the right design keeps delegation at the orchestrator only. The specialists are doers — they do their specific job, they don't spawn.

Koko: Exactly. One orchestrator, all delegation flows through it. Now, what about checking the draft?

Chris: The writer checking its own draft — that's not adversarial. You need a separate, independent validator. Ideally one that's actively trying to refute the draft, not just skim it.

Koko: That's it. Read-only, independent, defaults to fail. The report ships only when every validator passes. Think of it like a defense panel — you don't let the PhD student grade their own dissertation. You bring in people whose job is to poke holes.

Chris: And the specialist agents themselves — because their work is independent, they run in parallel. You only sequence things when there's a genuine data dependency.

Koko: Right. Parallelism is the default when work is independent. Sequencing is the exception you justify. The exam rewards knowing which is which.

Koko: Scenario two. A nightly job processes a large batch. The platform has an execution-time limit and kills the job partway through. On restart, it either loses everything or double-processes some records. What's the shape of this problem?

Chris: The job has to survive being killed. That means it needs to be resumable — it can't treat each run as starting from zero.

Koko: Exactly. And the exam puts a very attractive option right next to the correct answer: just run it as one long background task. Sidestep the time limit entirely. The trap is that you're still not resilient — if anything kills that task, you're back to square one.

Chris: So the right move is to externalize state. Checkpoint after each slice, keyed by a run ID, so a restart picks up where it left off.

Koko: Yes. And you design each invocation to do a bounded slice — stop safely before the time wall — and then let an external caller re-drive the loop. The job doesn't fight the platform; it cooperates with it.

Chris: That handles resumability. But the double-processing problem is separate — you also need each step to be idempotent. A skip-if-done check, a keyed upsert so reprocessing a record is a cheap no-op.

Koko: And that's the model you want to name out loud on the exam: at-least-once delivery plus idempotent processing equals effectively-once outcomes. The platform might deliver a record more than once — you just make sure the second delivery doesn't matter.

Chris: So the three instincts are: externalize and checkpoint, bound each invocation, and make every step idempotent. All three together.

Koko: All three. One without the others leaves a gap. If you checkpoint but your steps aren't idempotent, a restart still duplicates. If your steps are idempotent but you don't checkpoint, you restart too far back. The exam tests whether you see the whole picture.

Koko: Alright, let's lock in the instincts before you walk through that door. Seven of them — I want them to feel like reflexes, not notes you're recalling. First one: prefer the fixed, named-step pipeline over 'let the model decide' whenever the work is stable. Why? Because it's observable, resumable, and budgetable.

Chris: So the exam question might frame the dynamic approach as more flexible or more intelligent, but the fixed pipeline wins on reliability grounds.

Koko: Exactly. Flexibility is not the virtue this domain rewards. Predictability is. Second instinct: read the stop reason and branch on it. Never assume every reply is final text — the model might be pausing for a tool call, or hitting a length limit.

Chris: The trap there is treating the output uniformly regardless of why the model stopped.

Koko: Right, the exam will offer you that shortcut and it costs you. Third: state lives in durable storage, keyed by a run ID, checkpointed every step. Not in memory until the end. Not fire-and-forget in the background.

Chris: Because if the process crashes mid-run, in-memory state is just gone.

Koko: Gone. And you have no way to resume. Fourth instinct: make retries harmless with skip-if-done logic and keyed upserts. The trap here is trying to prevent retries from ever happening — you can't guarantee that, so design for it instead.

Chris: Idempotency by construction, not by hope.

Koko: I'm stealing that phrase. Fifth: many agents means a strict org chart. Only the orchestrator delegates. Independent read-only validators fail the gate on any single miss — not on a majority, not on average. One miss, gate fails.

Chris: The option sitting next to that on the exam will say validators should escalate findings to the orchestrator for a final call. But that undermines the independence.

Koko: Exactly — independence is the whole point of a separate validator. Sixth instinct: classify the error before you react. Only transient errors are retryable. Logic errors, permission errors — retrying those just burns your budget and makes noise.

Chris: So classify first, then branch — not retry by default and classify later.

Koko: Correct. And seventh, the risky-edges cluster: guards fail closed, the kill switch lives out of band, invariants go in hooks not prompts, and autonomy scales with risk on a staged ladder. Those four travel together.

Chris: The through-line across all seven is basically: the exam rewards designs that survive a crash, a retry, and a bad day in production — over designs that give the model more freedom.

Koko: That's the whole domain in one sentence. When two answers are competing, ask which one you'd trust at two in the morning when something's gone sideways. That's your answer. This is twenty-seven percent of the exam — it's the slice most worth drilling. Go hit the flashcards, run the quiz, and you've got this.

Koko: Alright, final stretch. Let's lock in the exact vocabulary the exam uses, because this is where precise wording earns or loses points.

Chris: The agentic loop control term is stop_reason. While it returns tool_use you keep iterating — run the tools, append results, loop back. When it returns end_turn, you stop.

Koko: Exactly. And the trap the exam sets is inferring completion from what the assistant *says* in its text. The text might read 'I'm done here' and stop_reason is still tool_use. You keep going.

Chris: Right, text is not a signal. And the iteration cap — that's a safety guardrail, not your primary stop condition.

Koko: Good. Cap is a backstop, not a design choice. Hold that.

Chris: On multi-agent work — the tool for spawning subagents is called Task, or Agent in current Claude Code, with Task still accepted as an alias.

Koko: And it has to appear in the coordinator's allowedTools, otherwise the spawn is not auto-approved — and in a headless run, it's denied outright. The trap there is assuming Task just works because you called it.

Chris: What about subagent context? That one surprised me when I first saw it.

Koko: Good thing to flag. Subagents get isolated context — they do not inherit the coordinator's conversation history. So you pass everything they need directly in the prompt.

Chris: And you get parallel subagents by emitting multiple Task calls in a single response. One response, several spawns, they run concurrently.

Koko: Exactly. Serial Task calls give you sequential execution. One response, multiple calls, parallel. That distinction shows up on the exam.

Chris: Hooks over prompts — that's the big reliability instinct. A PostToolUse hook normalizes tool results, like unifying timestamp formats, deterministically.

Koko: And a tool-call interception hook can block a policy violation outright — the example in the notes is a refund over five hundred dollars. A prerequisite gate blocks a downstream tool until verification has run. These are deterministic enforcements.

Chris: So when the question asks what most reliably prevents something, the answer is the hook or the gate, not a stronger prompt.

Koko: That is the instinct to carry out of this room. 'Most reliably' equals deterministic. The trap is reaching for a better-worded instruction when the exam is testing whether you recognize that prompts can be talked around and hooks cannot.

Chris: Decomposition splits into prompt chaining — fixed sequence of passes — versus adaptive decomposition, when subtasks depend on what you discover along the way.

Koko: And sessions: dash-dash-resume continues a named session — pass it what files changed so it re-analyzes. fork_session branches a divergent exploration from a shared baseline without disturbing the original thread.

Chris: So resume is continuity, fork is experimentation.

Koko: That's the one-line version and it's correct. Okay — the handful of instincts you're carrying forward: stop_reason drives the loop, not text. Hooks enforce, prompts suggest. Parallel means one response with multiple Task calls. Isolated context means you pass what subagents need explicitly. And 'most reliably' always points to the deterministic option. You've got this.

Chris: That's a clean set to take into the exam.

Koko: Keep practicing with the flashcards and the quiz at KokoAI Academy — find them at koko knows dot A I. You've put in the work. Go show them what you know.