CCA-F Final Prep — the 60-Minute Walk-In

Koko: The exam rewards the design that still holds on a bad day. That is the instinct this whole sitting is testing, and that is what this hour is about — not a re-teach, but the decision rules, the named traps, and the shape of the room, so your entire surprise budget goes on the items themselves.

Chris: Right, and the five full domain lessons already did the heavy lifting. This is the walk-in.

Koko: Exactly. So let's start with the arithmetic, because it dissolves most of the anxiety. Sixty items, one hundred and twenty minutes. Two minutes per item.

Chris: And two minutes is actually generous for this kind of item. The bottleneck isn't reading speed — it's discrimination. You get two or three options that all survive a first pass, and that's where the clock goes.

Koko: That's exactly it. So the pacing plan writes itself: bank time on the items where the answer arrives in fifteen seconds, and spend that surplus on the three or four that are genuinely close. You are not trying to go fast. You are trying to go unevenly fast in the right direction.

Chris: The items are also grouped into case-study scenarios, which changes the read strategy entirely.

Koko: It does. The sitting draws four scenarios from a pool of six, and on average about fifteen items sit under each one. So here is the rule: read the scenario brief once, properly, before the first item. You have paid the reading cost for all fifteen up front.

Chris: Because if you re-read it item by item, you pay that cost fifteen times.

Koko: Right. And when you do that first read, you are not reading for everything — you are extracting the constraints that will keep mattering across the whole scenario. What the system must never do. What has to be auditable. Volume and latency requirements. Who approves what. Where the irreversibility sits. Those are the fixed facts of that world, and every item under it is just a question about that one world.

Chris: So treat the scenario like a constitution — read it once, then govern every item by it.

Koko: I like that. Now, item format — and the exam guide is explicit here, so this one is not a reading, it is a stated rule. Items are multiple-choice and multiple-response. Each item tells you how many responses to select.

Chris: And that instruction line is not a hint. If it says select two, then two is a fact about the item. One answer or three answers is wrong, full stop, no matter how good the individual picks are.

Koko: The trap there is treating it as a hint rather than a constraint. Read that line every single time, and re-read it before you commit. Every time.

Chris: On scoring — criterion-referenced, so no curve. You're measured against a fixed standard, not ranked against whoever else sat that week.

Koko: Scale is one hundred to one thousand, cut score is seven hundred and twenty. Pass or fail, your scaled score, and — here's the part most people don't expect — your percentage correct in each of the five domains. Those domain percentages are diagnostic only. The pass decision is the total scaled score. But they mirror how you should be studying: against a per-domain readiness view.

Chris: On guessing — the current guide says nothing about a penalty either way. But under criterion-referenced scaled scoring with no stated penalty, a blank cannot possibly score better than an attempt.

Koko: That's my reading of it, and I'll own that it is a reading rather than a quoted rule. But the conclusion is clear: answer everything. Flag it, move on, come back — do not hand back an empty item at the end of two hours.

Chris: What about the logistics — delivery, cost, renewal?

Koko: Quickly, so they're not taking up working memory on the day. Delivered through Pearson VUE, online-proctored or test centre depending on your program policy. List price is one hundred and twenty-five US dollars, but partner pricing steps it down — Select and Preferred partners pay half, Global Premier partners pay nothing through the end of twenty twenty-six. Credential is valid twelve months. Renewal is a free, non-proctored assessment.

Chris: And if you let it lapse, you retake the full proctored exam at full fee. That's a purely administrative mistake.

Koko: The most expensive one available in this program. Diary the renewal the day you pass. Now — domain weights, because this is your last calibration before we go in. Agentic architecture is twenty-seven percent. Claude Code configuration and prompt engineering are each twenty. Tools and MCP is eighteen. Context and reliability is fifteen.

Chris: Which isn't a permission to skip the fifteen-percent domain — it's a tiebreaker. If you're stuck between spending thirty more seconds on an agentic item versus a context item, spend it on the agentic one.

Koko: Exactly. You cannot skip fifteen percent. But a close call is worth an extra thirty seconds in the domain that moves the needle most. Alright — that's the room. Let's go domain by domain.

Koko: Alright, here is the one sentence worth tattooing on your hand before you walk in. When a requirement must hold every time, encode it deterministically. When it is a matter of judgement, encode it in the prompt. That single rule resolves more items than anything else in the bank.

Chris: And the reason it works is mechanical, not philosophical. A prompt instruction is probabilistic — it is followed most of the time, and most of the time is a failure rate.

Koko: Exactly. Scale that across a hundred thousand runs, or across the one run that matters — a refund above threshold, a write to a system of record, a deployment rollback — and high probability is not a control. Code that fires on an event is a control.

Chris: So the instinct to carry in is: prefer the answer that makes the wrong thing structurally impossible over the answer that makes it strongly discouraged.

Koko: That is it. And here is the tell in the wording. Watch for reliably, always, must never, guarantee, consistently, enforce, cannot. Every one of those words is pointing you toward a hook, a prerequisite gate, a schema, a validator, a scoped permission — not toward better phrasing.

Chris: Right, and the adjacent option on those items will say something like 'add a strongly-worded instruction to the system prompt emphasising that the model must never do X.' Most candidates reach for that because it is the cheapest fix.

Koko: Cheap is not what is being asked. The objectives ask for enforcement, not instruction. Now flip it: when the wording is improve, tends to, more consistent, better quality — that genuinely is a prompting move. The instinct has two directions.

Chris: Can you run through what the instinct looks like across the five domains? I want to be able to recognise it fast.

Koko: Yes, five faces, same shape. Agentic: hooks and prerequisite gates over system-prompt emphasis, and reading the stop reason rather than parsing the model's prose to decide what happened. Tools: a scoped toolset with a constrained tool that validates its own inputs, over a general tool plus an instruction about when to use it. Configuration: glob-scoped rules and non-interactive mode over a convention everybody is asked to remember. Prompt engineering: a schema the platform enforces over asking for JSON in prose. Reliability: a structured error field and a persisted scratchpad over a narrative that has to survive summarisation.

Chris: That last one is interesting — the scratchpad point. Because prose is exactly what survives least well across a summarisation boundary or a subagent hop.

Koko: Right. Structure beats prose is actually one of three corollaries that spin off this instinct, and each one carries its own cluster of items. Claim-source pairs, error categories, retryability flags, confidence per field — anything a downstream system needs to decide on should arrive as a field, not as a sentence to be parsed.

Chris: What are the other two corollaries?

Koko: Independence beats effort. A system checking its own work is structurally weaker than one checked by an instance that never saw the generation. Turning up reasoning effort on self-review does not substitute for that independence — the trap there is 'increase chain-of-thought depth for the same model to review its own output.' And three: category before reaction. When something fails, name what kind of failure it is — transient, validation, business, or permission — before choosing what to do. Only transient failures are retryable. Blanket retries burn budget on failures that will never resolve themselves.

Chris: That category-first rule is clean. So the exam item will offer 'retry up to three times on any error' as the distractor, and the right answer routes on failure type first.

Koko: Exactly. Now — and I want to say this before we move on — an instinct applied without limit is its own failure mode. Not everything belongs in a hook. Hooks carry rules you can state exactly: a threshold, a required order, a forbidden action, a format normalisation. Judgement stays in the prompt. If you cannot write the rule down as a condition, you cannot put it in a hook. The over-correction distractor is the item that says 'move all the reasoning into deterministic code.' That is too far.

Chris: So the discriminator is: can you actually write this as a condition? If yes, hook. If it requires interpretation, prompt.

Koko: That is the drill. And under time pressure, here is how to run it on any item where two answers survive the first pass: ask which one still holds when the model has a bad day. The one that holds up is almost always the right answer.

Koko: Domain one is the biggest one — twenty-seven percent — and it's where production systems actually break, which is why it's worth the time. Let's start with agentic loops. The whole loop turns on one thing. Chris, what is it?

Chris: Stop reason. You check the stop reason on every response. While it reads tool use, you run the tools, append the results, and loop. When it reads end turn, you stop. That's the whole control flow.

Koko: Exactly. And the exam option sitting right next to that correct answer will say: stop when the reply sounds finished, or when the model stops requesting tools. The trap is treating completion phrasing as a signal.

Chris: Right, because completion phrasing is generated text — it varies run to run, it varies by language. You'll stop early on what's actually a polite interim summary.

Koko: That's it. Two more traps live right next door. An iteration cap — if you hit the cap, raise an error, don't exit cleanly. A cap cannot tell a finished task from a stuck one, so it is never the successful exit condition.

Chris: And text and tool use aren't exclusive. A single turn can narrate and then call a tool. So if you treat any turn containing text as final, you silently drop the tool call.

Koko: Silent drop — that's the scary one. And the subtler cousin: each iteration extends one growing conversation. If you re-send the original prompt without appending the tool results, the model has no evidence the call ever happened. It asks for the same tool again. And bills you every pass.

Koko: Multi-agent orchestration. Hub and spoke. The coordinator owns delegation, aggregation, error handling, routing — every message crosses it. What's the thing that kills the structure?

Chris: Letting subagents call each other directly. It removes a hop, sure, but you lose observability and uniform error handling — which were the entire reason for the structure.

Koko: Exactly. And the coordinator decides which subagents a query actually needs. The trap here is running every specialist on every query so nothing gets missed — that's waste dressed as thoroughness. And synthesis is an evaluation point, not a finish line. You look at what it's missing, re-delegate against the gaps, synthesise again.

Chris: On subagent invocation — the exam gets precise here. Subagents are spawned with the Task tool. The coordinator needs Task in its allowed tools, or the spawn isn't auto-approved and a headless run just denies it.

Koko: Delegation is a permission before it is an instruction. Each subagent is declared by an agent definition — a description, a system prompt, and its tool restrictions. What a subagent is allowed to do is a property of its definition, not of the conversation that spawned it.

Chris: And subagents start with isolated context. They inherit nothing. So everything they need has to go in the prompt, in full. 'Summarise the findings above' is a prompt with no findings in it.

Koko: Good example. For parallelism: emit multiple Task calls in a single coordinator response. Delegating one at a time and waiting is serial by construction. And keep content and its metadata in separate structured fields all the way down — flatten findings into prose and attribution dies.

Chris: On enforcement — where a step must always happen, the instinct is a prerequisite gate: make the next call structurally impossible until the prior step completes. Strengthening the system prompt doesn't do that.

Koko: Neither do few-shot examples that always show the correct order. Few-shot shapes judgement and format, not deterministic sequence. And a near-miss to watch: a classifier that enables only the appropriate tools per request type controls availability, not sequence. Gate the prerequisite, not the toolbox.

Chris: And when you hand off to a human, you hand over a structured summary — identifier, root cause, amount, recommended action — not the raw transcript. The human can't see the conversation and shouldn't have to reconstruct it.

Koko: Hooks next, because the exam tests the two shapes carefully. PostToolUse transforms what comes back — normalising heterogeneous timestamp and status formats so the model receives one shape and doesn't waste reasoning reconciling them. PreToolUse runs before the call goes out. If you want to prevent an action, you need PreToolUse — PostToolUse is too late by one step, the tool has already run.

Chris: And when you block, you redirect into an approval workflow — typically human escalation. Logging the violation and letting the call through is not a control.

Koko: Right. Two hooks fire around delegation itself and they're not symmetric. SubagentStart observes a spawn — only observes. SubagentStop fires when a subagent finishes and can gate completion: block with a reason and the subagent goes back to work. But neither one rewrites output. If you want to reshape what comes back, a PostToolUse hook on the Agent tool is the right instrument.

Chris: For decomposition — prompt chaining for a fixed known sequence, adaptive decomposition when subtasks depend on what you discover. And for a wide review, per-unit passes plus a separate integration pass whose only job is what crosses between units. Concatenating per-file findings is not an integration pass.

Koko: And one oversized prompt containing everything doesn't buy you a whole view — it buys attention dilution. Finally, sessions. What's the discriminator between Resume and Fork?

Chris: Resume continues a named conversation — you name the investigations you intend to return to. Fork branches divergent explorations from one shared baseline, like comparing two approaches. Fork is not for continuing one thread.

Koko: And the trap with resumption: it carries prior results forward as if they're still current. After a refactor you either say what changed and ask for targeted re-analysis, or you start fresh with a structured summary. Picking up where you left off without flagging the change gets you analysis of code that no longer exists. That's domain one.

Koko: Domain two — tool design and MCP integration — is eighteen percent of the exam and the one domain where a handful of crisp rules really does cover most of it. Let's move fast. Tool descriptions first. The framing that settles almost every question here: write the description for a reader deciding between this tool and the one next to it. Not just what the tool does — where it stops being the right one.

Chris: So the defect they test is a description that covers functionality but has no boundary — no inputs, outputs, edge cases — nothing that tells you when to stop using it.

Koko: Exactly. And the fix when two tools are routing poorly into each other — what's the move?

Chris: Remove the overlap. Rename, re-scope the description, or split the tool. The ambiguity lives in the interface, so that's where you fix it.

Koko: That's it. The trap the exam offers is: tell the model in the system prompt which one to prefer. It looks like a fix. But it patches the symptom and leaves the overlap in place. Remove the ambiguity at the source. And the reverse instinct is also a trap — one flexible tool that handles the whole job feels simpler to build, but it's harder to select correctly because the contract is vague. Purpose-specific tools with defined contracts win.

Chris: And the system prompt is part of the tool interface too, right? Keyword-sensitive wording up there can override a well-written description.

Koko: Yes — audit both together. One without the other is an incomplete review.

Koko: Structured errors. The core rule: failure is signalled with the error flag, not with a success payload whose text happens to mention something went wrong. A success shape reads as data and gets used as data — the agent doesn't know to stop.

Chris: And beyond the flag, the tool needs to return an error category and a retryable flag. The agent reads retryable first — can a retry ever work — then the category to decide how to respond: resend, self-correct, escalate, or reroute.

Koko: Exactly. The rule on retries: retry only what is marked retryable, because the tool is the thing that knows. The trap is: retry anything a few times with backoff since it's cheap and often works. A validation error and a permission error will fail identically forever.

Chris: One boundary I want to make sure I have straight — an empty result set is not an error.

Koko: Right. No matches is a result. Only a failure to execute is an error. Treating zero results as a failed search sends the coordinator hunting for a problem that doesn't exist. And a subagent should recover transient failures locally and propagate only what it genuinely can't resolve — with partial results and what was already attempted — not pass every hiccup upward. Never return raw exception text as the whole story: structured metadata for the agent, plain explanation for the person, internals for the log.

Chris: Tool choice — auto, any, named. The one that trips people is any. The exam plays on the idea that any guarantees you'll get the right tool. But it only guarantees some tool gets called.

Koko: Precisely. Any guarantees a call, not a correct call. When one specific tool must run first, name it — forced selection — then let subsequent steps happen in follow-up turns rather than expecting the whole sequence at once. Two platform notes: some current models reject forced selection, so on those you use auto with a strict tool schema and verify the request arrived. And tool choice only controls what's requested — the application still authorizes, validates, and executes it.

Chris: So scoping subagents to just the tools their role needs is part of the same instinct — selection reliability degrades as tool count grows.

Koko: That's the principle. A blocked agent is a routing question for the coordinator, not an argument for handing everyone the full toolset. For frequent cross-role needs, give a narrow scoped tool for that case. And prefer a constrained tool that validates what it's handed over a broad utility that accepts anything.

Koko: MCP. First decision is scope. Three levels: project scope for shared team tooling everyone should have, user scope for a personal server that follows you across every project, local scope for a personal server in one project. All three are active simultaneously, so the answer is rarely 'put everything in one'.

Chris: Credentials — environment variables that expand at load time, so the value never enters the config file itself.

Koko: Right. The trap that looks responsible: adding the config file containing a token to gitignore. It still writes the secret to disk. Reference it as an environment variable instead. And registering an MCP server doesn't make the agent prefer it over a built-in — describe the tool's capabilities and outputs in enough detail that it out-competes the generic option. Resources are also worth flagging: exposing a catalogue as a resource removes whole rounds of exploratory tool calls because the agent can see what's available instead of probing. For standard integrations, reach for an existing community server. Reserve custom servers for genuinely team-specific workflows.

Chris: Built-in tools — Grep versus Glob is a classic trap. Finding where a function is called is Grep, every time. Glob is path matching.

Koko: Every time. Edit versus Write is the other one. Edit is targeted, anchored on unique text. When Edit can't anchor uniquely, you extend the anchor with enough surrounding context to pin one occurrence, or replace across all matches. The documented fallback for a reliable whole-file change is Read then Write. What never works is retrying the same anchor and hoping.

Chris: And for building codebase understanding — Grep to the entry points, Read to follow the imports that matter — rather than reading the whole directory upfront for context you'll mostly never use.

Koko: Exactly. Incremental beats exhaustive. That's domain two — tight rules, clean decisions. Next up is domain three.

Koko: Domain three is configuration, and it is worth your attention because it is unusually rule-shaped — the exam is testing whether you know the rule, not whether you can reason your way to it. Twenty percent of the exam. Let's start with memory hierarchy. Instructions live at user level, project level, or directory level, and the level is the whole question.

Chris: And the symptom that keeps coming up is: it works on your machine, breaks for a new joiner.

Koko: Exactly. That symptom is the signature of a rule sitting at user level that should be at project level. The joiner doesn't have your user config, so they don't see the instruction at all.

Chris: The option next to it on the exam will say the new joiner must have misconfigured something on their end.

Koko: Right, and that is the trap — it sounds plausible, but the symptom is a placement bug. Anything the whole team must share belongs in project configuration, full stop. Think of it like environment variables: if it lives only on your laptop, it doesn't exist for the pipeline.

Chris: And the modular habit follows from that — split into focused topic files, pull them in by reference, don't grow one giant file nobody reads.

Koko: Yes. Import each package's standards into that package's own file rather than piling everything into the root. Now, when behaviour looks inconsistent — what do you reach for to diagnose it?

Chris: The context command. It lists the files that actually loaded, so you can see what the session is working with.

Koko: Correct — and that matters because it reports, it doesn't trigger. Configuration loads from its location on its own; running the diagnostic doesn't cause it to load. The trap is adding another copy of the instruction higher up to 'fix' it, which just hides the real placement bug.

Chris: Slash commands follow the same scope logic — project scope for team-wide, user scope for personal ones.

Koko: Exactly. And skills are configured through their frontmatter. Three fields do most of the work: forking the context, declaring allowed tools, and the argument hint.

Chris: Forking the context runs a verbose or exploratory skill in its own space so it doesn't pollute the session — you return the summary, not the transcript.

Koko: Right. Declared allowed tools pre-approve what the skill needs so it runs without prompts, and disallowed tools actually remove a tool from its reach. Together they make the skill's blast radius a property of the skill itself.

Chris: The trap being that you rely on the surrounding session being limited rather than controlling it inside the skill.

Koko: Exactly — that is the trap. Session context is the liability, not the safeguard. And the argument hint shows the expected parameter at autocomplete time, but it validates nothing, so the skill body's first instruction should check the value and stop without it.

Chris: What's the clean line between something that belongs in memory versus a skill?

Koko: Always-on versus on-demand. Universal standards — always in memory. Task-specific workflows you deliberately invoke — skills. And if you want a shared skill to behave differently for you personally, create a differently-named variant at user scope. Don't edit the shared one.

Chris: On plan mode — I'd say the trigger is breadth and the number of viable approaches, not how important the task feels.

Koko: That's it. A migration touching eighty files with several viable designs is plan mode even if the destination is decided. A single null guard on a known function is direct execution even if it's a production defect. Using plan mode for everything is a real cost, not just good practice.

Chris: And they're not one choice for the whole task — plan the investigation, execute the agreed approach directly.

Koko: Exactly. Discovery goes into an isolated exploration subagent so the verbose output stays out of main context. Only the summary comes back.

Chris: For path-specific rules — scope by glob pattern when a convention follows a file type rather than a location. So test conventions for co-located test files need a glob, not a memory file inside a tests directory.

Koko: Right. A memory file in the tests directory works only if those tests are confined to that directory. When test files live next to source files, the glob is the only tool that reaches them. Write the pattern, then verify it activates on one file before trusting it everywhere.

Chris: And a pattern kept deliberately broad so nothing is excluded stops meaning anything about the file being edited.

Koko: Good. Conditional activation is about what the model is being asked to hold in mind, not just token count. 'Load everything always' is not a harmless simplification — that's the trap there.

Chris: On iterative refinement — when output is inconsistent, the fix is demonstrating rather than describing. Two or three concrete input-output pairs settle a shape faster than prose.

Koko: Exactly. Write the test suite first — expected behaviour, edge cases, performance — then iterate on failures. And in an unfamiliar domain, ask for the questions before writing the specification. The interview surfaces what you didn't know to specify.

Chris: For CI — non-interactive mode is a requirement of the environment, not a preference. An interactive invocation in a pipeline just hangs.

Koko: And ask for structured output against a schema, not prose you then parse by hand. Carry project context into CI runs — existing tests, fixtures, review criteria — or you get generic suggestions you filter yourself.

Chris: Review from an independent instance, because the session that wrote the code shares exactly the context that's the liability.

Koko: And when a review re-runs on new commits, include the prior findings and ask only for what's new or still unaddressed. The trap is running a fresh review every push — you get a pile of duplicates every time. Those are your configuration rules. Rule-shaped, high-yield, worth every minute.

Koko: Domain four, prompt engineering and structured output — twenty percent of the exam, and the one domain where answering from general intuition will hurt you the most, because the documented mechanics are counterintuitive. Let's start with the through-line: precision comes from stating the condition under which something is reportable, not from asking for more certainty.

Chris: Right, and the instinct almost everyone walks in with is: tell the model to be conservative, only surface high-confidence findings. I actually thought that was reasonable until I looked at why it fails — the model's sense of its own confidence isn't calibrated to your definition of a false positive. You're not reducing errors, you're just reducing volume.

Koko: Exactly. You're turning a dial that controls quantity, not discrimination. The replacement is a categorical criterion — something like 'flag a comment when the behaviour it claims contradicts what the code does.' That's testable. 'Be accurate' is not.

Chris: And severity levels have the same problem. If each level isn't anchored to a concrete example of what belongs there, classification drifts between runs.

Koko: And when one category goes noisy, trust in the accurate ones goes with it. The trap here is leaving the noisy category running while you tune it, because some of its findings are real and you don't want to miss them. The right move is to disable the noisy category, restore trust in the rest, then re-enable once its criteria are specific.

Chris: Okay, few-shot — reach for it when detailed instructions still produce inconsistent output. More instructions is the wrong lever for a format problem; demonstrating the output is the right one.

Koko: Two to four examples do more than paragraphs of instructions. And here's the part the exam tests specifically: what actually transfers is not the example itself but the reasoning for choosing one action over a plausible alternative.

Chris: So draw examples from the awkward, disputed cases — not the clean canonical ones. A long catalogue of easy cases underperforms a few well-chosen hard ones.

Koko: Right. And when required fields keep coming back empty across varied source layouts, the option next to the right answer will say 'tighten the schema.' But that's usually a demonstration problem — show correct extraction from the specific formats that produced the blanks.

Chris: Structured output — use a schema the platform enforces, not just 'return JSON' in the prompt. Structured outputs for a direct JSON answer, strict tool schema when the result is a tool request.

Koko: And always check that the response completed before you accept it. A refusal or a truncated reply is not a result.

Chris: What's the boundary that the exam draws between schema enforcement and semantic validation? Because I want to make sure I have that crisp.

Koko: Schema enforcement removes syntax errors. Full stop. A schema-valid response can still be wrong. Semantic validation is its own separate step — the exam treats them as two distinct gates.

Chris: And on field design — mark a field optional where the document may legitimately omit it. Give a closed enum an explicit unclear value plus an other-with-detail pair.

Koko: The trap here is marking every field required so nothing gets skipped. That's precisely how you manufacture invented values. The model fills the field — it just fills it with something that isn't there.

Chris: For an extraction endpoint on a model that supports forced selection — any guarantees a call when the document type is unknown, name a specific tool when one extraction must run first.

Koko: Good. Now validation and retry — retrying with the same prompt is not a retry loop, it's a coin flip. Send the original document, the failed extraction, and the specific validation errors, and ask for a correction.

Chris: And retries fix format and structure. They cannot conjure information that isn't in the source. A required value that keeps coming back empty because the document doesn't contain it is a pipeline problem.

Koko: Exactly. Where a total should equal its parts, have the extraction report both the stated and the calculated value and flag the discrepancy as data — don't validate it downstream in application code afterwards.

Chris: Batch — match the API to the latency requirement. Batch is substantially cheaper, runs in a long processing window, no latency guarantee. A blocking pre-merge check is not batch work, however attractive the cost saving.

Koko: Size your submission schedule against the worst-case window, not the typical turnaround you happened to observe. And a multi-turn agentic loop with tool calling stays synchronous — batch handles self-contained requests.

Chris: Correlate responses by custom identifier, resubmit only the failures modified for whatever caused them — not the whole batch. And refine the prompt against a sample before committing a large volume.

Koko: Last piece — multi-pass review. Independence is the mechanism. A session reviewing its own output is a weaker check than an instance that never saw the generation.

Chris: And the exam offers a specific distractor here — raising the reasoning effort on the same session as a substitute for independence. That's the one to watch for.

Koko: It feels like a reasonable upgrade, and it does nothing for independence. Split a wide review into per-unit passes for local issues, plus a separate pass for what crosses between units. And have the verification pass self-report calibrated confidence per finding — treating every finding as equivalent just pushes the whole triage burden onto the reviewer.

Chris: So the through-line across the whole domain is: replace vague instructions with testable conditions, demonstrate rather than instruct for format, enforce at the schema level but validate semantics separately, and build independence into review rather than trying to compensate for its absence.

Koko: That's the domain in one breath. Hold onto that and you're ready for whatever they put in front of you.

Koko: Domain five, fifteen percent of the exam, and the one that determines whether your system actually works past day one. Let's start with conversation context. The key question: as a conversation grows and you need to control context size, what do you trim and what do you protect?

Chris: The narrative — the back-and-forth — you can summarise. But transactional facts, amounts, dates, order numbers, stated expectations — those have to stay verbatim in a persistent facts block that lives outside the summarised history.

Koko: Exactly. Think of it like meeting notes. You summarise the discussion, but you never paraphrase the contract number. Because the trap is that a summary is precisely where an order number quietly becomes 'the customer's recent order' — and now you've lost the order.

Chris: And the other size-control lever is trimming verbose tool output at the point it enters context — cut it to the fields the task actually needs — rather than dropping earlier turns to make room.

Koko: Right, because dropping earlier turns costs coherence. Trim the width, not the depth. And position matters too: lead with key findings, use explicit section headers for detail. Don't build to a conclusion at the end — the end is where attention degrades.

Chris: What about passing context between agents? Is that a different rule?

Koko: Yes, and it's a common exam question. Between agents you return structured key facts — the claim, the source, the relevance — not the reasoning narrative that produced them. The reasoning is for the agent that did the work. The next agent needs the map, not the journey.

Chris: Escalation. Three triggers: explicit request for a human, a policy gap, and no meaningful progress. What does the exam try to swap in instead of those?

Koko: Two big ones. Sentiment — the user sounds frustrated — and complexity — the request looks complicated. Both feel like triggers, neither is. The instinct the exam rewards is: acknowledge frustration, offer resolution, but escalate only on the stated preference or a real blocker. And on the explicit request side, the trap is attempting to resolve it first because the issue looks simple. That is a policy violation with good intentions.

Chris: So an explicit escalation request is a trigger, full stop — not an input to a judgement call.

Koko: Full stop. Same logic applies to identity ambiguity — if a lookup returns several possible matches, that's a question for the customer, not a ranking problem you solve by picking the closest match.

Chris: On error propagation — a failed subagent needs to return structured context: the failure type, what it attempted, any partial results, alternatives. Not just 'search unavailable.'

Koko: And there are two symmetrical traps here. One: returning an empty result when something failed — the pipeline keeps moving and the failure disappears. Two: aborting the entire workflow because one subagent failed. The right answer is recover locally where you can, continue with the gap explicitly annotated. And in the synthesis, say which areas are well-supported and where a source was unavailable. A hole in the evidence must not be read as an absence of risk.

Chris: Large codebase exploration has a specific failure signature — answers become inconsistent, the model starts describing what typically happens rather than what it found. That's context degradation.

Koko: Memorise that signature, because the exam describes it rather than naming it. The remedy is threefold: write findings to a scratchpad that outlives the window, delegate specific investigations to subagents while the main agent holds the map, and summarise each phase and inject the summary as the next phase's starting context. And crash recovery is designed, not improvised — each agent exports state to a known location, the coordinator reloads the manifest on resume.

Chris: And compaction is not a substitute for writing things down — persist first, then compact.

Koko: Good. Now confidence calibration. The efficient-sounding option the exam will offer is: review the low-confidence outputs, let the high-confidence ones through automatically.

Chris: The option next to it will say 'sample the high-confidence stream continuously' — and that's the right answer, because that's the only thing that catches novel errors.

Koko: Exactly. An aggregate accuracy figure hides poor performance on one document type or one field. Calibrate field-level confidence against a labelled validation set before it routes anything, and stratify the sample — a random sample is unbiased and still tells you nothing about your smallest, riskiest segment.

Chris: Provenance — carry claim-source pairs as structure through every hop, because citations can't be reconstructed from summarised prose after the fact.

Koko: And when two sources disagree, report both attributed — let the reader see the conflict. Don't pick the more credible one silently. Pass conflicts forward annotated so they get reconciled where the whole picture is visible. Require dates in structured output so a temporal difference is distinguishable from a genuine contradiction. And render by content type — tables for figures, prose for narrative, structured lists for technical findings. Flattening everything into uniform prose for a clean-looking report is where the structure dies.

Chris: So the through-line for the whole domain is: keep the structure visible at every stage — in context, in errors, in evidence, in output.

Koko: That's it. Visibility is reliability. Anything that hides a gap — a silent summary, an empty result, a blended synthesis — looks clean until the system fails, and by then you can't tell why.

Koko: So before we go any further, a quick line about what this section actually is — because the exam's own scenarios are confidential, and we have no business describing them. What we can do is walk six synthetic case studies from the KokoAI Academy practice pool. Fictional companies, invented for teaching. Their value is that they cover the recurring architectural shapes the exam works in, so when you read an unfamiliar brief on the day you can place it in the first thirty seconds.

Chris: Thirty seconds — so it's about pattern recognition before you even get to the options.

Koko: Exactly. Shape one: a regulated consumer-support agent. Telecommunications provider, coordinator plus specialists — identity, order, billing, policy — over tools that both read and take real action. Quoting a refund, executing one, changing a plan.

Chris: So the controls are doing a lot of work here. Monetary threshold, identity verification, explicit customer confirmation before the action fires.

Koko: Right, and an explicit request for a human gets honoured — hard rule. The failure modes the exam loves here: order numbers lost to summarisation, a gateway timeout producing a duplicate action, and an escalation prompt keyed to how emotional a request sounds instead of a stated trigger.

Chris: The option next to it on escalation will say something like 'detect distress in the message.' The instinct to load is escalate on stated triggers, not sentiment.

Koko: That's it. Plus idempotency keys for the timeout problem, scoped toolsets so specialists only hold what they need, and a persistent facts block so the order number survives summarisation. Shape two: multi-source intelligence network. Scouts and analysts in isolated sessions across regulatory, market, supplier and internal sources, producing an evidence-grounded briefing with an approval gate before anything urgent goes out.

Chris: The provenance rules are the whole game there. Every material claim traceable to a precise source version, retrieval date carried, contradictory evidence surfaced rather than resolved away.

Koko: And missing data stays null or labelled unavailable — never inferred. The trap here is treating a source outage as an absence of risk. Also: three syndicated reports all derived from one original are not independent corroboration.

Chris: Shape three is the monorepo modernization — and the tell is that the root memory file is nearly empty while all the real conventions live in one person's user config.

Koko: Memory hierarchy problem, yes. Glob-scoped rules for co-located test files, a forked read-and-search-only skill, MCP server configured with expanded environment variables rather than a committed token. And the execution mode question: plan mode for the wide migration, direct for the one-line defect.

Chris: Shape four — document-extraction at scale. A million-plus contracts, OCR input, two service classes: a fast synchronous path with a latency requirement and a large async backlog.

Koko: The schema design is where marks get lost. Optional fields for values documents legitimately omit, an explicit unclear value rather than a null that looks the same as absent, and per-field calibrated confidence — not one global threshold. The trap here is treating a retryable format failure the same as information that is simply not in the document.

Chris: And retries should carry the specific validation errors, not just retry blind.

Koko: Shape five: legacy migration workflow. The whole shape is built around weak habits — sessions drifting into unrelated subsystems, large references pasted into memory files, one long conversation used as the migration record.

Chris: So the instinct is decomposition, plan mode, scratchpads, session hygiene, resume by name, test-led iteration.

Koko: Shape six: internal MCP platform. Shared remote server, fronting an issue tracker, CI, deployment and a service catalogue for hundreds of engineers. The prototype mirrors upstream endpoints and returns raw payloads — that's the tell.

Chris: Task-oriented tools over endpoint mirrors, resources for the catalogues, structured errors with retryability flagged. And consequential writes get gated.

Koko: Plus per-developer authorization boundaries, and the decision about what belongs in deterministic orchestration versus an agent loop. That's all six.

Chris: What if the brief on the day doesn't look like any of these?

Koko: These are archetypes, not a key. Read the brief for its controls — the thresholds, the approvals, the audit requirements, the latency and volume numbers — and the domain instincts attach themselves. You're not looking for the matching shape, you're reading what the scenario is trying to govern. The shape just gets you there faster.

Koko: Alright, rapid-fire round. I call the frame, you read the exam's tempting option, I snap back. These are the distractors, not your views — just say what the option on the page claims. If something doesn't land instantly, flag it for your last ten minutes. Ready?

Chris: Ready. Starting with agentic loop logic. The exam says: the reply contains text, so the turn is final.

Koko: No — text and tool use coexist. A turn can have both. Branch on stop reason, nothing else.

Chris: The option next to it says: the iteration cap was reached, so the task is complete.

Koko: No — hitting the cap raises an error. End of turn is the only successful exit.

Chris: And: the reply says it is done, so the loop ends.

Koko: No — branch on stop reason. The content of the reply is not the signal.

Chris: Subagents. The exam says: the subagent can see the conversation that spawned it.

Koko: No — inherited context is something you pass explicitly. It does not travel automatically.

Chris: The subagents are defined, so delegation will start.

Koko: No — delegation is a tool permission first. Define it, then grant it.

Chris: Delegate to each subagent in turn.

Koko: No — multiple Task calls in one response is what makes them parallel. Sequential is just slow.

Chris: Let subagents call each other to cut a hop.

Koko: No — every message crosses the coordinator. Subagents do not have side channels.

Chris: And: synthesis has run, so the work is done.

Koko: No — synthesis is an evaluation point. Re-delegate against the gaps.

Chris: Okay, orchestration. The exam says: strengthen the system prompt so the required step is unmistakable.

Koko: No — make the next call structurally impossible until the step has happened. Structure beats instruction.

Chris: Add few-shot examples that always show the right order.

Koko: No — few-shot shapes judgement, not sequence. You want a structural gate.

Chris: Enable only the appropriate tools per request type.

Koko: No — availability is not sequence. The model can still call them in the wrong order.

Chris: Use a PostToolUse hook to stop a policy violation.

Koko: No — prevention intercepts the outgoing call. Post is already too late. And the flip side: don't move every rule into hooks — hooks carry rules you can state exactly. Judgement stays in the prompt.

Chris: Escalate by handing over the transcript.

Koko: No — hand over a structured summary that stands on its own. The transcript is noise.

Chris: Tool design. The exam says: keep the description short, the name says it.

Koko: No — write it for a reader choosing between this tool and its neighbour. The name is never enough.

Chris: Fix misrouting in the system prompt.

Koko: No — remove the overlap at the source. Prompt patches don't fix a design problem.

Chris: Ship one flexible tool so there is less to choose between.

Koko: No — purpose-specific contracts select more reliably. Flexibility is ambiguity.

Chris: Return one clear failure message.

Koko: No — category, retryability, explanation. Three things, every time.

Chris: Retry any failure a few times.

Koko: No — retry only what is marked retryable. And treat an empty result as a valid result, not a failure.

Chris: Set tool choice to any so it picks the right tool.

Koko: No — any guarantees a call, not a correct one.

Chris: Put the token in the config and gitignore the file.

Koko: No — reference the variable and expand at load time. The secret never touches the file.

Chris: Model everything as tools — resources are just an optimisation.

Koko: No — a resource catalogue removes the exploratory calls entirely. That's not an optimisation, that's the design.

Chris: Last one. Use Glob to find where a function is called.

Koko: No — Grep searches inside files. Glob matches filenames. Different tools, different jobs. That's part one. Anything that blurred, mark it now.

Koko: Alright, same pace, domains three through five. I'll snap the frame, you read the exam's tempting option. First one: something works locally but breaks for other users.

Chris: The option sitting next to the right answer says the joiner misconfigured something.

Koko: Right, and the frame is: works locally, breaks elsewhere is the signature of a user-level rule that should live at project level. Move it up. Next — universal standards, like always enforce citation format.

Chris: Most candidates reach for a skill, because skills feel like the right container for reusable behavior.

Koko: The trap exactly. Always-on belongs in memory. Skills are for optional, invocable capability. If it never turns off, it's not a skill, it's a standing instruction. Now, tool access on a skill.

Chris: Declare it on the skill itself — allowed tools to pre-approve, disallowed tools to explicitly remove. Don't leave it to whatever the session happens to allow.

Koko: Exactly. Session defaults are not a declaration. Next one's a prompting trap. You have a convention that applies to all test files.

Chris: Scope by glob, because the convention follows a file type, not a directory. The option next to it says put it in a memory file inside the tests folder.

Koko: That's it. Glob matches the pattern; a directory file matches the location. Different thing. Plan mode next.

Chris: The trigger is breadth — multiple viable approaches on the table. Not complexity alone, not just because the task is long.

Koko: Good. The trap is using plan mode for everything as a safety blanket. It's a branching tool, not a caution tool. Now CI versus local runs.

Chris: Non-interactive is a hard requirement in CI. The exam option says run it the way you run it locally, but local runs assume a human is present.

Koko: Right. And code review — the session that wrote the code can't review it.

Chris: Independence is the mechanism. Same session means shared context and shared blind spots.

Koko: Good. Okay, reliability traps. Format varies in your output — what's the move?

Chris: Demonstrate the output. The option next to it says add more instructions, but demonstration beats description when format is the issue.

Koko: Exactly. Schema design: you want nothing skipped.

Chris: Mark fields optional where the source document might legitimately omit them. Marking everything required just generates hallucinated filler.

Koko: And schema-valid does not mean correct — schema removes syntax errors only. When validation fails and you retry —

Chris: Retry with the specific errors attached. Not just retry harder.

Koko: And if a value keeps coming back empty?

Chris: That's a pipeline problem, not a prompting problem. Absent information isn't in there.

Koko: Good. Batch processing — when do you use it?

Chris: Work nobody is waiting on. The trap is moving a blocking check to batch just to save cost. If something is in the critical path, batch is the wrong tool.

Koko: And turning up reasoning effort for self-review?

Chris: Effort doesn't substitute for independence. You still need a separate reviewing step.

Koko: Agentic loop traps now — fast ones. Conversation grows long, what do you protect?

Chris: Keep the facts block outside the summary. Summarise older turns, fine, but the facts block does not get compressed.

Koko: Customer escalation — the trigger is?

Chris: The stated request or a real blocker. Not the customer's tone. Frustration is not an escalation trigger.

Koko: And if they explicitly ask for a human?

Chris: That's a trigger, not an input to negotiate with. The option next to it says try to resolve it first — that's the trap.

Koko: Subagent fails, what does the orchestrator return?

Chris: The failure stays visible. Never swallow it with an empty result.

Koko: Crash recovery — start the exploration again?

Chris: No. Reload the exported state manifest. Starting over throws away work and risks re-doing side effects.

Koko: Two sources, two different figures. Most candidates pick the more credible one.

Chris: Report both, attributed. And if they're from different dates, that's temporal difference, not contradiction — carry the dates and let the reader decide.

Koko: Last one. Overall accuracy looks high, so you reduce review frequency.

Chris: Verify by document type and by field. And sample the high-confidence stream continuously — that's the one that can drift without anyone noticing.

Koko: That's the whole set. If anything didn't snap into place immediately, that's your re-read list for the last ten minutes.

Koko: Ten minutes out. Nothing new goes in now. This is just about what you hold and what you do with it.

Chris: Okay. So what are the five sentences — one per domain?

Koko: Agentic: branch on the protocol signal, keep the coordinator as the only delegator, and make the must-happen step structurally impossible to skip. That is the whole domain in one breath.

Chris: Tools: write the description for someone choosing between two tools, return errors as fields not sentences, and scope the toolset to the role.

Koko: Exactly. Configuration: anything the team shares lives at project level, scope by glob when the convention follows the file type, and CI runs non-interactive with structured output reviewed by an independent instance.

Chris: Prompting: state the condition rather than asking for confidence, put the schema in the tool input, and schema-valid is not the same as semantically correct.

Koko: That last one is the one people forget under pressure. And reliability: protect the transactional facts from summarisation, escalate on stated triggers not tone, and keep the failure visible even when the answer degrades.

Chris: Five sentences. That actually fits in working memory.

Koko: Now above all of those, hold the tie-breaker. When two options both look reasonable, choose the one that still holds on a bad day — a crash, a retry, a timeout, an ambiguous input, a model having an off run.

Chris: Deterministic over probabilistic. Structural over instructional. Independent over self-checked. Visible over tidy.

Koko: Say it to yourself before you commit to any answer where you are torn. It resolves more items than anything else on this list.

Chris: Honest question — what if I go blank on a stem? Like, nothing fires.

Koko: There is a recovery routine and it works. Re-read the stem and find the operative verb. Reliably. Always. Must never. Guarantee. Consistently. Prevent. If one of those is in there, eliminate every option that is a prompt-wording change, a stronger instruction, a reminder, or a request for more care. You are usually down to one or two after that.

Chris: And if none of those words are present — if the stem says improve or more consistent — then the heavy deterministic-sounding option is probably the over-correction distractor.

Koko: Right. The machinery is the answer when the stem demands a guarantee. It is the trap when the stem is just asking for a nudge. And one mechanical thing: check the response-count line before you commit, every single time. Misreading that line is how you lose points you knew.

Chris: On pacing — ninety seconds and move, flag it, come back?

Koko: Yes. A second pass over four flagged items with forty minutes left beats grinding one item at minute twenty. Answer everything before the clock runs out, including the flagged ones. A blank cannot score better than an attempt.

Chris: And when first instinct and second-guess disagree — ask the tie-breaker rather than alternating.

Koko: Exactly. One other thing about the sitting as a physical event: most late-exam errors are attention errors, not knowledge errors. Misreading the response-count line, missing a negation, answering the question you expected instead of the one written. Slow down for the last ten items rather than speeding up.

Chris: If the option you like requires the scenario to be slightly different from how it was written, it is the wrong option.

Koko: That sentence right there has probably saved more scores than any single concept in these modules. Write it on your scratch paper.

Chris: What about afterwards — what does the report actually tell you?

Koko: You get the scaled score against the seven-twenty cut and the percent correct by domain. If it does not go your way, that is a genuine study map, not just a verdict. If it does go your way — put the twelve-month renewal in your calendar that same day. It is a free, non-proctored assessment while the credential is live. If you let it lapse, it is a full retake at full fee.

Chris: The cheapest ten seconds of admin you will ever do.

Koko: The instinct this exam rewards is the one that would survive a Tuesday afternoon incident: predictable, recoverable, observable, and honest about what it does not know. That is the whole thing. You have done the work. Go and get it. And keep sharpening it with the flashcards, the drills, and the simulated exam over at KokoAI Academy — koko knows dot A I.