Skip to main content
    All shows

    Saturday, August 15

    CCA-F Final Prep — the 60-Minute Walk-In

    0:00-:--
    Speed

    Transcript

    Koko: The exam rewards the design that still holds on a bad day. That is the one instinct this entire sitting is testing — not whether you know the happy path, but whether you know what survives when something goes wrong. Keep that in your head for the next hour.

    Sam: Okay. And what exactly is this hour? Because I have done the five domain lessons. Is this a repeat?

    Koko: No, and that is worth saying plainly. The five domain lessons do the teaching. This hour is the decision rules, the named traps, and the shape of the sitting itself — so that nothing about the room surprises you, and your whole surprise budget is available for the actual items.

    Sam: I like that framing. Use the surprise on the hard questions, not on figuring out how the test works.

    Koko: Exactly. So let's start with the arithmetic, because it kills most of the anxiety. Sixty items, one hundred and twenty minutes. That is two minutes per item, and two minutes is genuinely generous for the kind of item this exam throws at you.

    Sam: It feels tight though. When I do practice items I sometimes stall out.

    Koko: And that is the thing — you are stalling on the hard ones, which is correct behavior. The work is almost never reading speed. It is discriminating between options that both survive a first pass. In practice, most people blow their time budget on three or four genuinely close calls, not on sixty items being too long to read.

    Sam: So the pacing plan is what — just go faster?

    Koko: More specific than that. Bank time on the items where the right answer shows up in about fifteen seconds. And spend that banked time on the three or four that are actually close. That is the whole pacing strategy.

    Sam: Got it. Now the items are grouped into scenarios, right? How does that work?

    Koko: A sitting draws four case-study scenarios from a pool of six. On average you are looking at roughly fifteen items sitting under each scenario, though nothing published promises an even split. The key move is: read the scenario brief once, properly, before you touch the first item under it.

    Sam: So I'd just re-read the brief for each item as I go, right? That way I have it fresh every time.

    Koko: That is the trap. If you read it in fragments, item by item, you pay the reading cost fifteen times over. Read it once and extract the constraints that will keep mattering — what the system must never do, what has to be auditable, where the money or the irreversibility sits, who has to approve what. Then treat every item under that scenario as a question about that one fixed world.

    Sam: Oh, that is a real time save. Read once, extract the durable constraints, and those constraints answer almost everything that follows.

    Koko: Right. Now item format — this sounds basic but the guide is explicit about it, so we are going to be explicit too. Items are multiple-choice and multiple-response, and each item states how many responses to select.

    Sam: And that number is a hint, basically.

    Koko: No — it is a fact about the item. If an item says select two, then two is the answer set size, full stop. An answer set of one or of three is wrong regardless of how good your individual picks were. Read that instruction line every single time, and re-read it before you commit.

    Sam: That is the kind of thing that costs you points for a reason that has nothing to do with knowing the content. Painful.

    Koko: Genuinely avoidable, which makes it worse. Okay — scoring. The exam is criterion-referenced, which means you are measured against a fixed standard set by subject matter experts. Not against whoever else sat it that week.

    Sam: So no curve.

    Koko: No curve. The scale runs one hundred to one thousand, and the cut score is seven hundred and twenty. The report comes back with pass or fail, that scaled score, and — the part most people do not expect — the percentage you answered correctly in each of the five domains.

    Sam: Wait, domain percentages? Those drive the pass decision?

    Koko: They are diagnostic only. The pass decision is the total scaled score. But they mirror exactly the kind of per-domain readiness view you should have been studying against — so when you see your report, it speaks the same language as your prep.

    Sam: Okay, and what about blanks? If I am not sure on something, is it safer to leave it blank so I do not lose points?

    Koko: So — and I want to be honest about the limit of this — the current exam guide says nothing about penalties or guessing. It does not state a rule either way. My reading, and it is a reading rather than a quoted rule, is that under criterion-referenced scaled scoring with no stated penalty, a blank cannot possibly score better than an attempt. Answer everything. Flag it, move on, come back — but do not hand back an empty item.

    Sam: Fair enough. Attempt everything, flag the uncertain ones, revisit at the end.

    Koko: That is the move. Logistics quickly, just so they are not taking up working memory on the day. Delivery is through Pearson VUE. The fee is one hundred and twenty-five US dollars. The credential is valid for twelve months, and renewal is a free, non-proctored assessment.

    Sam: Free renewal, that seems easy to stay on top of.

    Koko: Until you let it lapse. Let it lapse and you retake the entire exam at full fee. That is the most expensive purely administrative mistake available in this program. Diary the renewal the day you pass.

    Sam: Day one. Noted. And the domain weights — how should I be thinking about those going in?

    Koko: Agentic architecture is the heaviest at twenty-seven percent. Tools and MCP at eighteen. Claude Code configuration at twenty. Prompt engineering at twenty. Context and reliability at fifteen. You cannot skip fifteen percent — there is no safe domain to abandon. But when a question is genuinely close, the weights tell you where an extra thirty seconds of scrutiny pays the most.

    Sam: So agentic architecture is where a close call costs the most.

    Koko: And also where the exam concentrates its hardest design problems, which is not a coincidence. Alright — that is the room. Now let's get into the content.

    Koko: Alright, here is the single highest-leverage sentence in this entire hour. If you walk in with nothing else, walk in with this: when a requirement must hold every time, encode it deterministically — when it is a matter of judgement, encode it in the prompt. That is it. That one sentence resolves more exam items than anything else we will cover.

    Sam: Okay, but is that really it? Because my first instinct when something is going wrong is just — write a clearer instruction. Be more forceful in the prompt.

    Koko: That is exactly the trap, and I mean that as a compliment — you are carrying the same instinct most candidates carry in. And yes, a clearer instruction is the cheapest fix. I will agree with you on that.

    Sam: So — why not do the cheap thing?

    Koko: Because cheap is not the criterion the exam is asking about. Here is the mechanical reason. A prompt instruction is probabilistic. It is followed most of the time. And most of the time is a failure rate. Scale that across a hundred thousand runs, and a high probability is not a control. Code that fires on an event is a control.

    Sam: Right, so if it is a refund above a threshold, or a write to a system of record — even one miss matters.

    Koko: Exactly. So the instinct to carry is: prefer the answer that makes the wrong thing structurally impossible over the answer that makes the wrong thing strongly discouraged. Those two things sound similar. They are not.

    Sam: How do I spot which kind of item I am looking at quickly?

    Koko: The wording is the tell. Watch for: reliably, always, must never, guarantee, consistently, enforce, cannot. Every one of those is a signal that the answer lives in a hook, a gate, a schema, a validator, a scoped permission — not in better wording. Conversely, if you see improve, tends to, more consistent, higher precision — that is a prompting move.

    Sam: So the exam is basically hiding the answer in the verb.

    Koko: Pretty much. And this instinct has the same shape across all five domains — it just wears a different costume. In the agentic domain it is hooks and prerequisite gates over system-prompt emphasis, and stop reason over reading the model's prose. In tools, it is a scoped toolset with a constrained tool that validates its own inputs, not a general tool plus an instruction about when to use it.

    Sam: And in prompt engineering I am guessing it is — supply a schema rather than asking for JSON in prose?

    Koko: Yes. Schema supplied as a tool input beats asking nicely. In configuration, glob-scoped rules and non-interactive mode beat a convention everybody is asked to remember. And in reliability, a structured error field and a persisted scratchpad beat a narrative that has to survive summarisation.

    Sam: Okay, but can you over-apply this? Like, shove everything into deterministic code?

    Koko: Great question, and yes — an instinct without a limit is its own failure mode. Hooks carry rules you can state exactly: a threshold, a required order, a forbidden action, a format normalisation. Judgement stays in the prompt. If you cannot write the rule down as a condition, you cannot put it in a hook. An item that says move all the reasoning into deterministic code is the over-correction distractor.

    Sam: So the line is — can I actually write this as a condition, yes or no.

    Koko: Exactly. Now three corollaries fall out of the same instinct, and each one has its own cluster of items. First: independence beats effort. A system checking its own work is structurally weaker than one checked by an instance that never saw the generation. Turning up the reasoning effort on self-review does not substitute for that independence.

    Sam: Because the same blind spot that caused the error is still there when the model checks itself.

    Koko: Right. Second corollary: structure beats prose. Claim-source pairs, error categories, retryability flags, confidence per field — anything a downstream system needs to decide on should arrive as a field, not a sentence to be parsed. Prose is what survives least well across a summarisation boundary or a subagent hop.

    Sam: And the third?

    Koko: Category before reaction. When something fails, name what kind of failure it is — transient, validation, business, or permission — before deciding what to do. Only transient failures are retryable. Blanket retries burn budget on failures that will never resolve themselves.

    Sam: So retrying a permission error just means failing four times instead of once.

    Koko: Exactly. Now — time pressure. When two answers survive your first pass, ask one question: which one still holds when the model has a bad day? That is usually the right answer. The wrong answer depends on the model cooperating.

    Sam: The answer that does not need the model to behave. Got it.

    Koko: Domain one is the biggest one — twenty-seven percent — and it's the domain where production systems actually fall apart. So let's move through it fast but not sloppy. Start with agentic loops. The rule is simple: the loop is driven by stop reason and nothing else. While the stop reason reads tool use, you run the tools, append the results to the conversation, and go again. When it reads end turn, you stop. Full stop.

    Sam: Okay but — if the model's reply sounds done, like it's wrapping up nicely, can't I just exit there? It seems wasteful to ignore that.

    Koko: That's the classic trap and it's very tempting. Here's why it breaks: completion phrasing is generated text. It varies run to run, it varies by language, and a polite interim summary sounds exactly like a final answer. You will end early. The model is not signaling done — it's just being conversational.

    Sam: So the stop reason is a structured signal, but the text is just... vibes.

    Koko: Exactly. Now two more traps live right next to this. First: iteration caps. A cap is a runaway backstop — it should raise an error when hit, never serve as the normal exit condition. A cap cannot tell a finished task from a stuck one.

    Sam: Right, hitting the cap just means something went wrong, not that the work is done.

    Koko: Correct. Second trap: a single turn can contain both text and a tool call. If you treat any turn that has text in it as final, you silently drop the tool call. Nothing errors. It just never happens.

    Sam: Oh that's nasty. And there's one more you mentioned — about re-sending the prompt?

    Koko: Yes. Each iteration extends one growing conversation. If you re-send the original prompt each time without appending the tool results, the model has no evidence the call ever happened. So it asks for the same tool again. And again. And you're billed for every pass.

    Koko: Multi-agent orchestration. Hub and spoke. The coordinator owns delegation, aggregation, error handling, and routing. Every message crosses it. That's the entire design.

    Sam: So if I let subagents talk to each other directly, I save a hop — that seems more efficient.

    Koko: You save a hop and you destroy the observability and the uniform error handling that were the whole reason for the structure. The hop is not overhead — it's the control plane.

    Sam: And the coordinator picks which subagents to actually use — it doesn't just fan out to everyone?

    Koko: Right. Running everything through every specialist so nothing gets missed is waste dressed as thoroughness. The coordinator decides what each query actually needs. And synthesis is not the finish line — it's an evaluation point. You look at what the synthesis is missing, re-delegate against the gaps, synthesise again.

    Sam: Okay. Subagent invocation — this is where it gets precise, right?

    Koko: Very precise. Subagents are spawned with the Task tool, and the coordinator needs Task in its allowed tools or nothing spawns at all. Delegation is a permission before it is an instruction.

    Sam: And each subagent has a definition — description, system prompt, tool restrictions.

    Koko: Exactly. What a subagent is allowed to do is a property of its definition, not of the conversation that spawned it. And subagents start with isolated context — they inherit nothing. So everything they need goes in the prompt you hand them, in full.

    Sam: So 'summarise the findings above' is a prompt with no findings in it.

    Koko: That's the whole trap, stated perfectly. Two more: to get parallelism, emit multiple Task calls in a single coordinator response. Delegating one at a time and waiting is serial by construction. And keep content and its metadata in separate structured fields all the way down the chain — flattening findings into readable prose is where attribution dies.

    Sam: Moving to enforcement — if a step must always happen, how do you actually guarantee it?

    Koko: Make the next call structurally impossible until the prerequisite step has run. A gate in the structure. Strengthening the system prompt does not do that. Neither do few-shot examples that always show the correct order — few-shot shapes judgement and format, not deterministic sequence.

    Sam: What about a classifier that enables only certain tools depending on request type — does that count as gating the sequence?

    Koko: Watch out for that one on the exam. A classifier controlling tool availability manages what's available, not the order things happen in. Gate the prerequisite step, not the toolbox. And when you hand off to a human, hand over a structured summary — identifier, root cause, amount, recommended action — never the raw transcript. The human cannot see the conversation and should not have to reconstruct it.

    Sam: Hooks next — PreToolUse versus PostToolUse.

    Koko: Two shapes, and the exam tests the difference hard. PostToolUse transforms what comes back — normalising heterogeneous timestamp and status formats so the model receives one consistent shape. PreToolUse runs before the call goes out. Interception on the outgoing call is the only thing that prevents an action.

    Sam: So if I want to block a policy violation, PostToolUse is too late.

    Koko: One step too late — the tool has already run. Block with PreToolUse, and redirect into the workflow that can approve it, usually human escalation. Logging the violation and letting the call through is not a control at all. Then for delegation hooks: SubagentStart observes a subagent being spawned — it only observes. SubagentStop fires when one finishes and can gate completion. Block with a reason and the subagent goes back to work. Neither one rewrites output. If you want to change the result, a lifecycle hook is the wrong instrument.

    Sam: Decomposition and sessions — quickly?

    Koko: Prompt chaining for a fixed known sequence; adaptive decomposition when subtasks depend on what you discover. For a wide review, run per-unit passes plus a separate integration pass whose only job is what crosses between units. Concatenating per-file findings is not an integration pass. And one oversized prompt with everything doesn't give you a whole view — it gives you attention dilution.

    Sam: And sessions — Resume versus Fork?

    Koko: Resume continues a named conversation. Name the investigations you intend to return to — don't rely on whatever ran last. And resumption carries prior results forward as current, so after a refactor you either say what changed and ask for targeted re-analysis, or you start fresh with a structured summary. Fork is for branching divergent explorations from one shared baseline — comparing two approaches — not for continuing one thread.

    Sam: Stop reason drives the loop, the coordinator is the control plane, Task is a permission before it's an instruction, PreToolUse blocks and PostToolUse transforms, and decomposition and sessions both have a wrong-tool trap. Got it.

    Koko: That's the whole domain in one breath. On to the next one.

    Koko: Domain two, tool design and MCP integration, eighteen percent of the exam. Second-smallest domain by weight, but the rules are crisp — nail them and this is free points. Let's start with tool interfaces. Here is the framing that unlocks most of this task area: write the description for a reader deciding between this tool and the one next to it. Not just what it does — where it stops being the right choice.

    Sam: So the defect in a bad description isn't that it's wrong, it's that it's incomplete.

    Koko: Exactly. No inputs, no outputs, no edge cases, no boundary — the model cannot tell when to pick this one versus the neighbor. And the trap the exam loves: two tools overlap, the model misroutes, and your fix is a line in the system prompt saying prefer tool A.

    Sam: That does seem like the quick fix though.

    Koko: It patches the symptom. The overlap is still there. The right move is to remove the ambiguity at the source — rename, re-scope the description, or split the tool. Preferences in the system prompt are a band-aid over a design flaw.

    Sam: What about the opposite — one big flexible tool that handles the whole job? Simpler to maintain, right?

    Koko: That's the other trap. A vague contract is harder to select correctly, not easier. Purpose-specific tools with defined contracts win on selection reliability every time. And one more: the system prompt is part of the tool interface. Keyword-sensitive wording up there can override a well-written description, so audit both together.

    Sam: Okay, structured errors. I feel like I'd just return a message saying what went wrong.

    Koko: That's the trap. A success payload whose text says something failed still reads as data and gets used as data. Failure gets the error flag, full stop. Then beyond the flag you return three things: error category, retryable flag, and a human-readable explanation.

    Sam: The retryable flag — so the agent checks that before deciding whether to retry?

    Koko: Read retryable first: can a retry ever work? Then read category: how do I respond — resend, self-correct, escalate, take another route? A bare 'operation failed' with neither field leaves the agent unable to tell a transient blip from a permanent refusal.

    Sam: So I'd just retry a few times with backoff, right? It's cheap and usually works.

    Koko: Only retry what is marked retryable — and the tool is the thing that knows. A validation error and a permission error will fail identically forever. Retrying them costs nothing except sending the agent in circles.

    Sam: What about an empty result set? That feels like a failure.

    Koko: Classic trap. An empty result is a result. Only a failure to execute is an error. If you surface no-matches as an error, the coordinator goes hunting for a problem that does not exist. And subagents: recover transient failures locally, propagate only what you genuinely cannot resolve — with partial results and what was already attempted. Never send the raw exception text upward. Structured metadata for the agent, plain explanation for the person, internals for the log.

    Sam: Got it. Tool distribution — I'd think more tools available means more capable agent.

    Koko: Selection reliability degrades as tool count grows. That's why scoping exists. Each subagent gets only the tools its role needs. A blocked agent is a routing question for the coordinator, not a reason to hand everyone the full toolset.

    Sam: And for tool choice — auto, any, forced. I'd set it to any so it's guaranteed to pick the right one.

    Koko: Any guarantees a call, not a correct call. When one specific tool must run first, force that tool explicitly. Then let the remaining steps happen in follow-up turns rather than expecting the whole sequence in one shot.

    Sam: MCP scope — project versus user. When do I reach for which?

    Koko: Project scope for shared team tooling everyone should have. User scope for personal and experimental servers. Both are active at once, so the answer is almost never 'put everything in one'. And credentials — reference them as environment variables that expand at load time. The value never enters the file.

    Sam: I'd add the config file to gitignore and call it handled.

    Koko: Looks responsible, still writes the secret to disk. The variable reference is the pattern. One more: registering an MCP server does not make the agent prefer it over a built-in. You have to describe the MCP tool's capabilities and outputs in enough detail that it out-competes the generic option.

    Sam: And resources — is that just a nice-to-have?

    Koko: Not optional. Exposing a catalogue as a resource removes whole rounds of exploratory tool calls because the agent can see what is available instead of probing. For a standard integration, reach for an existing community server first. Custom servers are for genuinely team-specific workflows.

    Sam: Last piece — built-in tools. Grep versus Glob, I always mix those up.

    Koko: Grep searches inside files. Glob matches file paths. Finding where a function is called is Grep, every time. For edits: Read and Write are whole-file operations. Edit is a targeted change anchored on unique text.

    Sam: And if Edit can't anchor uniquely, I retry it with more surrounding context?

    Koko: No — the move is Read then Write deliberately. And build codebase understanding incrementally: Grep to the entry points, Read to follow the imports that matter. Do not read a whole directory up front for context you will mostly not use. That is the instinct the exam rewards in this domain — precision over breadth, at every level.

    Koko: Domain three is configuration and workflows — twenty percent of the exam, and it is unusually rule-shaped. That means it rewards pattern recognition more than judgment calls, which makes this the highest-yield section in the last hour. First concept: the memory hierarchy. Instructions live at user level, project level, or directory level, and the level is the whole question.

    Sam: Okay, so what does that actually break when you get it wrong?

    Koko: There is one symptom that appears on this exam in several forms: it works on your machine and not on a new joiner's. When you see that, what is your instinct?

    Sam: The joiner must have missed a setup step. Configured something wrong on their end.

    Koko: That is the trap. That symptom is the signature of a rule sitting at user level that should be at project level. Think of it like a sticky note on your own monitor — you see it, nobody else does. Anything the whole team must share belongs in project configuration.

    Sam: So the fix is move it, not tell the joiner to add it themselves.

    Koko: Exactly. And two habits follow from that. Keep configuration modular — split into focused topic files and pull them in by reference, rather than growing one enormous file nobody reads. And import each package's standards into that package's own file instead of piling everything into the root.

    Sam: What about debugging? If behaviour looks weird, how do you even know what loaded?

    Koko: The memory command reports what actually loaded. Check that before you theorize. One important distinction: it reports, it does not trigger. Configuration loads from its location on its own — running the diagnostic does not cause it to load.

    Sam: And the trap there would be adding another copy of the instruction somewhere higher up to force it.

    Koko: Right — that just hides the placement bug. Now you have two copies and you still do not know which one is actually doing the work.

    Koko: Slash commands follow the same scope logic: project scope for team-wide commands, user scope for personal ones. Skills are configured through their frontmatter, and three fields carry most of the exam items.

    Sam: What are those three?

    Koko: Forking the context, declared allowed tools, and the argument hint. Forking runs a verbose or exploratory skill in its own space so its output does not pollute the main session — you return the summary, not the transcript. Declared allowed tools make the skill's blast radius a property of the skill itself.

    Sam: As opposed to relying on whatever the surrounding session restricts.

    Koko: Exactly. Leaving access unrestricted because the session is already limited is the trap. And the argument hint is what prompts for a parameter at invocation time — documenting parameters in the body does not do that.

    Sam: What is the line between a skill and memory? I feel like some things could go either way.

    Koko: On-demand versus always-on. Universal standards that should always be present belong in memory. Task-specific workflows you deliberately invoke belong in skills. And if you want a shared skill to behave differently for you personally, create a differently-named variant at user scope — do not edit the shared one.

    Sam: Got it. What about path-specific rules? When do those come in?

    Koko: When a convention follows a file type rather than a location. Say your test conventions apply to test files that are co-located with source code — not tucked into a single tests directory. What would you do?

    Sam: Put the conventions in a memory file inside the tests directory?

    Koko: That works only if the convention is confined to that directory. When the files are scattered next to their source, you write a glob pattern that matches the files themselves, not a directory-scoped rule. And verify it activates on one actual file before calling it done.

    Sam: What about keeping patterns deliberately broad so nothing gets excluded?

    Koko: A pattern that is so broad nothing is excluded stops meaning anything about the file being edited. And conditional activation is about what the model holds in mind, not just token count — so loading everything always is not a harmless simplification, it is a real design decision.

    Koko: Plan mode versus direct execution. The trigger is breadth and the number of viable approaches — not how important the task feels.

    Sam: So a migration touching eighty files — plan mode, even if we already know the destination?

    Koko: Yes, because there are several viable designs. But a single null guard on a known function is direct execution even though it is a production defect. Importance does not flip the decision.

    Sam: And using plan mode for everything, just to be safe — that is a real cost, not just extra caution.

    Koko: Real cost. Discovery goes into an isolated exploration subagent so the verbose output stays out of the main context. And the two modes are not one choice for the whole task: plan the investigation, then execute the agreed approach directly.

    Sam: On iterative refinement — when output is inconsistent, what is the fix?

    Koko: Stop describing and start demonstrating. Two or three concrete input-output pairs settle a shape faster than any prose. Work test-first — write the suite covering expected behaviour, edge cases, and performance, then iterate on its failures. And when reporting a defect, give the failing input and the expected output, not a description of the misbehaviour.

    Sam: What about an unfamiliar domain where you are not sure what to specify?

    Koko: Ask for the questions before writing the specification. The interview surfaces what you did not know to specify. And batch issues whose fixes interact into one message — sequence the ones that do not.

    Sam: Last piece — CI and CD. What is the must-know there?

    Koko: Non-interactive mode is a requirement of the environment, not a preference. An interactive invocation in a pipeline hangs. Ask for structured output against a schema and parse that — do not parse findings out of prose.

    Sam: And carry project context into the CI run, or you get generic suggestions.

    Koko: Exactly — existing tests, fixtures, review criteria. Otherwise you are filtering generic output by hand. And review from an independent instance, because the session that wrote the code has the shared context that is precisely the liability. When a review re-runs on new commits, include the prior findings and ask only for what is new or still unaddressed, or every push produces a fresh pile of duplicates.

    Sam: So the instinct across this whole domain is: scope is a deliberate design decision, not a default you inherit.

    Koko: That is exactly it. Level, scope, activation, blast radius — every one of those is a choice, and the exam tests whether you made it on purpose.

    Koko: Domain four is prompt engineering and structured output — another twenty percent of the exam, and the one where general intuition gets people killed. Because everyone has opinions about prompting. The exam tests the documented mechanics, not your opinions.

    Sam: Okay, what's the first trap?

    Koko: Explicit criteria. Say you're flagging suspicious code comments — comments that don't match what the code actually does. What's your instinct for reducing false positives?

    Sam: Tell the model to be conservative. Only flag high-confidence findings.

    Koko: That is exactly the trap. It reduces volume, but it does not improve discrimination. The model's internal sense of its own confidence is not calibrated to your definition of a false positive.

    Sam: So I'm just turning the dial down, not sharpening the line.

    Koko: Exactly. Replace it with a categorical criterion. Flag a comment when the behaviour it claims contradicts what the code does. That is testable. 'Be accurate' is not.

    Sam: And severity levels — I'd anchor those to examples of what belongs in each tier?

    Koko: Yes, because without concrete anchors, classification drifts between runs. And here's the companion trap: if one category goes noisy, you might be tempted to leave it running while you tune the criteria.

    Sam: Because some of its findings are real.

    Koko: Right, but a noisy category poisons trust in the accurate ones. Disable it, restore trust in the rest, re-enable once the criteria are specific. Don't leave it running.

    Sam: Okay. What about when instructions aren't enough and the format keeps coming out wrong?

    Koko: That's the signal to reach for few-shot. Adding more instructions to a format problem is the wrong lever. Demonstrating the output is the right one.

    Sam: So two to four examples beat another paragraph of instructions.

    Koko: Two to four well-chosen examples, yes. And this is the part people miss — what transfers is not the example itself, it's the reasoning for choosing one action over a plausible alternative. So pull examples from the awkward, disputed cases.

    Sam: Not the clean canonical ones.

    Koko: A long catalogue of easy cases underperforms a few hard ones. And if required fields keep coming back empty across varied layouts, that's a demonstration problem, not a schema problem. Show correct extraction from the formats that caused the blanks.

    Sam: Alright, structured output. I'd just ask for JSON in the prompt and parse the reply, right?

    Koko: That's the trap. Define the schema as a tool input and read the structured call. Schema enforcement then handles syntax errors automatically.

    Sam: But that doesn't mean the answer is actually correct.

    Koko: That's the boundary that carries exam items. Schema enforcement removes syntax errors and does nothing about semantic correctness. Semantic validation is its own separate step.

    Sam: And to avoid invented values — I'd mark every field required so nothing gets skipped.

    Koko: That is precisely how you manufacture invented values. Mark a field optional where the document may legitimately omit it. Give a closed enum an explicit unclear value and an other-with-detail pair so it can extend honestly.

    Sam: Got it. And for tool selection — any versus a specific tool?

    Koko: Any is correct when the document type is unknown. Force a specific tool when one extraction must run before another. That ordering dependency changes the call.

    Sam: Validation and retry — if the extraction fails, I retry with the same prompt.

    Koko: That's a coin flip, not a retry loop. Send the document, the failed extraction, and the specific validation errors. Ask for a correction against those errors.

    Sam: But retries can fix anything if you're persistent enough.

    Koko: No — retries fix format and structure. They cannot conjure information that isn't in the source. A required value that keeps coming back empty because the document doesn't contain it is a pipeline problem.

    Sam: And for totals that should equal their parts — validate that in application code?

    Koko: Have the extraction report both the stated value and the calculated value and flag the discrepancy as data. Catching it earlier is cleaner than downstream validation.

    Sam: Batch processing — cheaper, so use it broadly. I'd use it for a blocking pre-merge check to save cost.

    Koko: That's the trap. Batch runs in a long processing window with no latency guarantee. A blocking check is not work nobody is waiting on. Match the API to the latency requirement.

    Sam: And size the schedule against the worst-case window, not typical turnaround.

    Koko: Yes. Also — batch handles self-contained requests. A multi-turn agentic loop with tool calling stays synchronous. And when a batch partially fails, resubmit only the failures, modified for the cause, not the whole batch.

    Sam: Last one — multi-pass review. If I need a stronger check, I'd raise the reasoning effort on the same session.

    Koko: That is the distractor the exam is counting on you to choose. A session reviewing its own output is a weaker check than a fresh instance that never saw the generation. Independence is the mechanism. Raising effort is not a substitute.

    Sam: Because the same session has all the same blind spots.

    Koko: Exactly. Split a wide review into per-unit passes for local issues plus a separate pass for what crosses between units. And have the verification pass report calibrated confidence per finding — not every finding equivalent — so you can route instead of dumping the whole triage burden on the reviewer.

    Sam: Domain four in a sentence: precision comes from stated conditions, not stated confidence — and independence beats effort every time.

    Koko: That's the one to carry into the exam.

    Koko: Domain five. Smallest domain on the exam, fifteen percent of the weight, and the one that decides whether a system survives its second week. Let's start with conversation context. When a conversation grows long, what's your instinct?

    Sam: Summarise the older turns. Compress the history, keep moving.

    Koko: Right for the narrative, catastrophic for the numbers. A summary is exactly where an order number quietly becomes 'the customer's recent order.' The transactional facts — amounts, dates, identifiers, stated expectations — need to live in a persistent facts block outside the summarised history. You summarise around it, not through it.

    Sam: So the summary handles the story of the conversation, and the facts block holds the specifics that can't be paraphrased.

    Koko: Exactly. Think of it like a legal file: you can write a case summary, but you keep the original contract in a separate folder because the summary will eventually say 'agreed amount' instead of the actual figure. Position matters too — lead with key findings, give detail explicit section headers. Don't bury the conclusion at the end.

    Sam: What about trimming verbose tool output? I'd think about dropping earlier turns to save space.

    Koko: That's the trap. Dropping turns costs coherence. Instead, trim the tool output to only the fields the task actually needs, at the point it enters context. And when passing work between agents, return structured key facts with citations and relevance — not the reasoning narrative that produced them. The narrative is dead weight at that handoff point.

    Sam: Okay, escalation. Three triggers: explicit request for a human, a policy gap, and no meaningful progress. That feels clean. What's the main way people get it wrong?

    Koko: Treating an explicit request as an input to a judgement call. Someone says 'I want to speak to a person' and the system thinks — this looks like a simple issue, let me just resolve it first. That's a policy violation with good intentions. An explicit request is a trigger, not a negotiation.

    Sam: What about sentiment? If someone's really frustrated, that seems like a signal to escalate.

    Koko: Sentiment is not a trigger. Acknowledge the frustration, offer resolution — but you escalate on stated preference or a real blocker, not on tone. And complexity isn't the criterion either. You escalate where policy is ambiguous or silent, even if the request looks simple. And don't use model self-reported confidence to route — route on explicit criteria with worked examples of escalate versus resolve.

    Sam: One more — if a lookup returns a few possible matches, I'd just pick the closest one and move on.

    Koko: That's a ranking problem masquerading as a resolution. Ambiguous identity is a question for the customer, not a problem to be solved by the system. Ask.

    Sam: Right. Now error propagation — a subagent fails. What does it return?

    Koko: Structured context: failure type, what it attempted, any partial results, and alternatives. Not just 'search unavailable.' The coordinator needs enough to make a real recovery decision. Two symmetrical traps here: returning an empty result when something failed keeps the pipeline moving and destroys the record that anything went wrong. But aborting the whole workflow because one subagent failed is the opposite over-correction.

    Sam: So recover locally where you can, and otherwise continue with the gap annotated.

    Koko: And say so in the synthesis. Call out which areas are well supported and where a source was unavailable. A hole in the evidence should not read as an absence of risk.

    Sam: Large codebase exploration. The exam describes a failure mode rather than naming it — what does that failure look like?

    Koko: Answers become inconsistent, and the model starts appealing to what typically happens rather than what it actually found. That's context degradation. The fix: write findings to a scratchpad that outlives the window, delegate specific investigations to subagents while the main agent holds the map rather than the terrain, and summarise each phase before starting the next rather than carrying raw exploration output forward.

    Sam: And crash recovery — you design it in advance?

    Koko: Each agent exports state to a known location, coordinator reloads the manifest on resume. Not improvised. And compaction is not a substitute for writing things down — persist what must survive first, then compact.

    Sam: Human review and confidence. I'd review the low-confidence outputs and let the high-confidence ones through automatically. That's efficient.

    Koko: It's efficient and it will eventually fail silently. Sampling the high-confidence stream continuously is what detects novel errors — that's the only mechanism that will catch them. Also, an aggregate accuracy figure hides poor performance on one document type or one field. Verify by type and by field before you reduce review anywhere. Calibrate field-level confidence against a labelled validation set, and stratify your sample so each segment is actually measured.

    Sam: A random sample isn't enough?

    Koko: A random sample is unbiased and still tells you nothing about your smallest, riskiest segment. Stratify.

    Sam: Last one — provenance. Carry claim-source pairs as structure through every hop.

    Koko: Because citations cannot be reconstructed after the fact from summarised prose. When two sources disagree, report both attributed — don't silently pick the more credible one. Pass conflicts forward annotated to be reconciled where the whole picture is visible. Require dates in the structured output so a temporal difference between two figures is distinguishable from a genuine contradiction. And render by content type: tables for figures, prose for narrative, structured lists for technical findings. Don't flatten everything into uniform prose just because it looks cleaner.

    Sam: So the throughline for this whole domain is: keep the record intact, make failures visible, and never let the system silently paper over a gap.

    Koko: That's exactly the instinct the exam rewards. If something went wrong or something is unknown, say so explicitly and carry it forward. A clean-looking output that hides a problem is worse than a messy one that names it.

    Koko: Okay, quick honest framing before we go anywhere. The actual exam scenarios are confidential — nobody here is going to describe them or pretend to. What we can do is walk six synthetic cases that KokoAI Academy built for its own practice pool. Fictional companies, invented for teaching. Their value is coverage: between them they hit the recurring architectural shapes, so when an unfamiliar brief lands in front of you on the day, you can place it in the first thirty seconds and know which instincts to have loaded.

    Sam: So these aren't the exam questions — they're more like... shape templates?

    Koko: Exactly. Archetypes, not a key. Shape one: the regulated consumer-support agent. Think a telecoms provider. A coordinator routes to identity, order, billing and policy specialists, and those specialists hold tools that both read and take real action — quoting a refund, executing one, changing a plan.

    Sam: So the controls are doing serious work there.

    Koko: The controls are the whole content. Monetary threshold for autonomous action, identity verification, an age limit on eligible transactions, explicit customer confirmation before anything executes, and a hard rule that an explicit request for a human is always honoured. The traps are predictable: order numbers lost when a summary replaces the conversation, specialists holding tools they have no business touching, a gateway timeout producing a duplicate action, and escalation logic keyed to sentiment instead of stated triggers.

    Sam: The sentiment one is sneaky. You'd think flagging an upset customer is the safe thing.

    Koko: It feels safe, but it's fragile. Load instead: idempotency keys, scoped toolsets, a persistent facts block, and escalation on explicit stated triggers. Shape two: the multi-source intelligence network. A manufacturer running scouts and analysts in isolated sessions across regulatory, market, supplier and internal sources, producing an evidence-grounded briefing on a deadline.

    Sam: The word 'evidence-grounded' is doing a lot there.

    Koko: Every material claim must trace to the precise source version, with publication and retrieval dates carried. Contradictory credible evidence gets surfaced — not resolved away. Missing data stays null or labelled unavailable, it is never inferred. And here's the trap: a source outage must never be represented as an absence of risk.

    Sam: Oh that's a good one. Silence isn't safety.

    Koko: Exactly. Also: three syndicated reports all derived from one original do not count as independent corroboration. Load: claim-source structure, coverage annotation, and the approval gate as deterministic code. Shape three: monorepo modernization. A large engineering organisation using Claude Code, with a nearly empty root memory file and all the real conventions living in one person's user-level config.

    Sam: So the memory hierarchy is basically broken before they start.

    Koko: That's the shape. Load: fix the hierarchy, glob-scoped rules for co-located test files, a forked read-and-search-only skill for exploration, MCP server configured with expanded environment variables rather than a committed token, plan mode for the wide migration and direct execution for the one-line defect, non-interactive CI with machine-readable output, an independent review instance, and finding deduplication across pushes.

    Sam: Plan mode for broad, direct for narrow. Got it. What's shape four?

    Koko: Document-extraction pipeline at scale. A manufacturer consolidating over a million supplier contracts and invoices from OCR text, with a strict target schema and two service classes: a fast synchronous path with a latency requirement, and a very large asynchronous backlog.

    Sam: And limited review capacity, so you can't just flag everything uncertain.

    Koko: Right. Load: schema as tool input, optional fields where documents legitimately omit values, an explicit unclear value — not a guess — self-checking extractions that report both stated and calculated totals, retries that carry the specific validation errors, and the key distinction: a retryable format failure is not the same as information that is simply absent. Batch for the backlog, synchronous for the deadline. Per-field calibrated confidence, not one global threshold.

    Sam: That per-field confidence thing — I'd have instinctively gone for a single score across the whole document.

    Koko: That's the trap. Shape five: legacy migration workflow. A very large mixed-language platform being modernized without changing externally visible behaviour. The scenario is really about weak habits — modernizing broad areas with no acceptance criteria, sessions drifting into unrelated subsystems, large references pasted into memory files, a single long conversation used as the migration record.

    Sam: So the architecture problem is the workflow itself.

    Koko: Exactly. Load: decomposition, plan mode, scratchpads and session hygiene, resume by name, and test-led iteration. Shape six: the internal MCP platform. A shared remote server fronting an issue tracker, CI, deployment and a service catalogue for hundreds of engineers, whose first prototype just mirrors upstream endpoints and returns raw payloads.

    Sam: Raw payloads for hundreds of engineers sounds like a support ticket waiting to happen.

    Koko: It is. Load: task-oriented tools over endpoint mirrors, resources for catalogues, structured errors with retryability signals, gating consequential writes, per-developer authorization boundaries, and deciding what belongs in deterministic orchestration versus an agent loop.

    Sam: Okay. Six shapes. But what if the brief on the day looks like none of them?

    Koko: Then you read for the controls, not the industry. Look for the thresholds, the approval requirements, the audit obligations, the latency and volume numbers. Those are the load-bearing elements. Once you spot them, the domain instincts attach themselves — the shapes are a warm-up for that reflex, not a lookup table.

    Koko: Alright, shifting gears. This is rapid-fire. I call the trap, you give me the tempting instinct, I snap back the fix. No deep dives — if it doesn't land instantly, flag it and re-read those notes in your last ten minutes. Ready?

    Sam: Ready. Go.

    Koko: The reply says it is done, so the loop ends.

    Sam: Sure, that's the exit condition — the model said it finished.

    Koko: No. Branch on stop reason. Text content is not a signal. Next.

    Sam: Got it. Stop reason only.

    Koko: The iteration cap was reached, so the task is complete.

    Sam: Cap hit means stop — wrap it up.

    Koko: No. The cap raises an error. End of turn is the only successful exit. Hitting the cap is not done, it is interrupted.

    Sam: Okay, end of turn equals success, cap equals interruption.

    Koko: The turn contains text, so it must be final.

    Sam: Text means the model is responding, not using a tool.

    Koko: No. Text and tool use coexist in one turn. Don't conflate them.

    Sam: Right, same turn can have both.

    Koko: The subagent can see the conversation that spawned it.

    Sam: It was created by the coordinator, so it inherits that context.

    Koko: No. Inherited context is something you explicitly pass. Subagents start blank.

    Sam: Pass it deliberately. Got it.

    Koko: The subagents are defined, so delegation will start.

    Sam: They exist in the architecture, so the coordinator can hand off to them.

    Koko: No. Delegation is a tool permission first. Defining agents is not enough. They need the permission wired in.

    Sam: Permission before delegation. Okay.

    Koko: Delegate to each subagent in turn.

    Sam: Sequential handoffs, one after another.

    Koko: No. Multiple Task calls in one response is what makes them parallel. Sequential is just slower.

    Sam: Parallel means multiple Task calls in the same response. Good one.

    Koko: Let subagents call each other to remove a hop.

    Sam: Fewer hops, more efficient.

    Koko: No. Every message crosses the coordinator. Direct subagent-to-subagent calls break the control plane.

    Sam: Everything routes through the coordinator. Always.

    Koko: Synthesis has run, so the work is done.

    Sam: You synthesized the outputs, that's the final step.

    Koko: No. Synthesis is an evaluation point. Check for gaps, then re-delegate against them.

    Sam: Synthesis triggers another pass if needed. Not an ending.

    Koko: Okay, now tools. Strengthen the system prompt so the required step is unmistakable.

    Sam: Clear instructions should get the model to do the right thing in order.

    Koko: No. Make the next call structurally impossible until the required step has happened. Instructions are hints. Structure is enforcement.

    Sam: Structure over instruction for sequencing. Okay.

    Koko: Add few-shot examples that always show the right order.

    Sam: Examples teach the model what sequence looks like.

    Koko: Few-shot shapes judgement, not sequence. You still need structural enforcement for guaranteed ordering.

    Sam: Few-shot is for judgement, structure is for sequence.

    Koko: Use a PostToolUse hook to stop a policy violation.

    Sam: Hook fires after the tool runs, blocks if it violated policy.

    Koko: No. Prevention intercepts the outgoing call. Post-tool is after the damage. Hook before, not after.

    Sam: Pre-call hook for prevention. Post is already too late.

    Koko: Move every rule into hooks.

    Sam: Hooks handle enforcement, so centralise everything there.

    Koko: No. Hooks carry rules you can state exactly. Judgement stays in the prompt.

    Sam: Exact rules to hooks, nuanced judgement to the prompt.

    Koko: Return one clear failure message.

    Sam: Simple, unambiguous, easy to parse.

    Koko: No. Return category, retryability, and explanation. One message loses the signal the caller needs.

    Sam: Three-part error: category, retryability, explanation.

    Koko: Retry any failure a few times.

    Sam: Transient errors clear up on retry, so retry everything.

    Koko: Retry only what is marked retryable. Retrying a non-retryable failure just burns budget.

    Sam: Check the retryability flag first. Got it.

    Koko: Treat an empty result as a failure.

    Sam: Nothing came back, so something went wrong.

    Koko: No. Empty is a valid result. The search ran, nothing matched. That is data.

    Sam: Empty result is not an error. It is information.

    Koko: Keep the tool description short, the name says it.

    Sam: Good names are self-documenting.

    Koko: No. Write the description for a reader choosing between this tool and its neighbour. The name is not enough when two tools look similar.

    Sam: Descriptions differentiate. Names alone don't.

    Koko: Fix misrouting in the system prompt.

    Sam: Tell the model more clearly which tool to pick.

    Koko: No. Remove the overlap at the source. If two tools look the same, no amount of prompting fully fixes it.

    Sam: Design out the ambiguity. Don't patch it with instructions.

    Koko: Put the token in the config and gitignore the file.

    Sam: It is not committed, so it is safe.

    Koko: No. Reference the environment variable and expand at load time. The file is one mistake away from getting committed.

    Sam: Environment variable at load time. Never the file.

    Koko: Last one. Use Glob to find where a function is called.

    Sam: Glob finds files, so it can locate the function.

    Koko: No. Glob matches file names. Grep searches inside files. Wrong tool, wrong result.

    Sam: Glob for names, Grep for contents. Clear.

    Koko: That is part one done. Anything that felt blurry, that is your re-read list. Part two coming right up.

    Koko: Alright, domains three, four, and five — same drill. I say the tempting answer, you tell me why it's wrong. Actually, let's flip it. You float the instinct, I'll snap the frame. Ready?

    Sam: Ready. Okay — something works fine locally but breaks for a specific user. So that user misconfigured something, right?

    Koko: Wrong instinct. That's the signature of a rule that belongs at project level but got left at user level. Works locally because your setup has it. Theirs doesn't. Move it up.

    Sam: Got it. What about a standard that applies everywhere — put it in a skill?

    Koko: No. Always-on belongs in memory, not a skill. Skills get invoked. Memory is just always there. Universal standards live in memory.

    Sam: And when I create a skill, I just leave tool access open and let it use whatever it needs?

    Koko: Declare it on the skill. Don't leave it unrestricted. The exam rewards explicit scope, not convenience.

    Sam: Okay, file conventions — say test conventions, I'd put those in a memory file inside the tests directory?

    Koko: Scope by glob instead. If the convention follows a file type, the glob targets that file type. Directory location is too blunt.

    Sam: Plan mode — I'd use it for everything, just to be safe.

    Koko: Only when there's real breadth or multiple viable approaches on the table. Using it for everything is noise. The trigger is genuine ambiguity, not habit.

    Sam: CI — run it the same way I run things locally?

    Koko: Non-interactive is a hard requirement for CI. Local habits don't transfer. Different environment, different contract.

    Sam: And for code review — have the session that wrote the code review it. Fresh eyes on fresh code.

    Koko: Independence is the mechanism. A different session reviews. Same session reviewing its own output is not a review, it's a reread.

    Sam: Format problems — I'd add more instructions to cover the variation.

    Koko: Demonstrate the output. Instructions about format accumulate. An example lands immediately. Demonstrate, don't describe.

    Sam: To make sure nothing gets skipped, mark every field required.

    Koko: Mark fields optional where the source document might not contain them. Required on everything means the model invents values just to satisfy the schema.

    Sam: If the response passes schema validation, it's correct.

    Koko: Schema removes syntax errors only. Correctness is a separate question. Valid structure, wrong content — still wrong.

    Sam: Validation fails — retry the extraction.

    Koko: Retry with the specific errors attached. A bare retry gets you the same output. The model needs to know what broke.

    Sam: A value keeps coming back empty — push harder, retry more.

    Koko: Absent information is a pipeline problem, not a model problem. If the data isn't in the source, no amount of retrying surfaces it.

    Sam: Slow blocking check — move it to batch to save time?

    Koko: Batch is for work nobody is waiting on. If something is blocking the user, batch doesn't help. Batch is about deferral, not acceleration.

    Sam: Self-review misses things — turn up reasoning effort so it catches more.

    Koko: Effort does not substitute for independence. More thinking by the same session still finds the same blind spots. You need a separate pass.

    Sam: Long conversation — summarise the older turns as things grow.

    Koko: Keep the facts block outside the summary. Summarise the narrative, preserve the data. Facts that get summarised get distorted.

    Sam: Customer sounds frustrated — escalate.

    Koko: Escalate on the stated request or a real blocker, not on tone. Frustration is a signal, not a trigger. You can sound annoyed and still want resolution.

    Sam: They ask for a human — try to resolve it first, right? One more shot.

    Koko: That request is a trigger, not an input to work with. Route immediately. Don't treat it as an invitation to keep going.

    Sam: Subagent fails — return an empty result and move on.

    Koko: The failure must stay visible. Empty result hides the problem. Propagate the failure so the system — and the human — can see it.

    Sam: Agent crashes mid-task — start the exploration over.

    Koko: Reload the exported state manifest. You already did that work. Starting over is waste. The manifest exists so you don't have to.

    Sam: Context fills up — compact aggressively.

    Koko: Persist first, then compact. Compact before persisting and you've lost state permanently. Order matters.

    Sam: Overall accuracy looks high — ease up on review.

    Koko: Verify by document type and by field. Overall accuracy averages over your easy cases. The hard ones are where errors hide.

    Sam: So I focus my sampling on the low-confidence outputs.

    Koko: Sample the high-confidence stream continuously. Low-confidence you already expect errors. High-confidence is where surprises live — and where they do the most damage because you weren't watching.

    Sam: Two sources disagree on a figure — I pick the more credible one.

    Koko: Report both, attributed. You don't get to decide which source wins. Surface the disagreement.

    Sam: But if two figures just differ, that's a contradiction.

    Koko: Carry dates. A figure from three years ago and a figure from last month aren't contradicting each other — they're showing change. Temporal difference is visible when you track when each number was recorded.

    Sam: That last one is subtle. Same number, different timestamps — totally different meaning.

    Koko: Exactly. And that's the whole set. Anything that didn't land just now — that's your ten-minute reread list before you walk in.

    Koko: Okay. Ten minutes out. Nothing new goes in. This is not the moment for one more concept — this is the moment to know what you are carrying and where to reach for it.

    Sam: I feel like I should be reviewing something. My hands want to flip through notes.

    Koko: That is a perfectly normal feeling and you should ignore it. What you hold right now is five sentences — one per domain. Let me give them to you and then we will talk about what to do with them.

    Sam: One sentence each. Okay, I am ready.

    Koko: Agentic: branch on the protocol signal, keep the coordinator as the only delegator, and make the must-happen step structurally impossible to skip.

    Sam: Structurally impossible to skip — not just strongly encouraged. Got it.

    Koko: Tools: write the description for someone choosing between two tools, return errors as fields rather than sentences, and scope the toolset to the role. Configuration: anything the team must share lives at project level, scope by glob when the convention follows the file type, and CI runs non-interactive with structured output reviewed by an independent instance.

    Sam: Independent instance — not the same model checking its own work.

    Koko: Exactly. Prompting: state the condition rather than asking for confidence, put the schema in the tool input, and remember that schema-valid is not the same as semantically correct. And reliability: protect the transactional facts from summarisation, escalate on stated triggers rather than tone, and keep the failure visible even when the answer degrades.

    Sam: That last one is the one I keep wanting to flip. Tidy feels like better UX.

    Koko: And that is exactly the trap. Visible beats tidy. A hidden failure is a worse user experience than an honest one, even if it looks worse in the moment.

    Sam: Okay. Five sentences. What is the one thing that sits above all of them?

    Koko: The tie-breaker. When two options both look reasonable, choose the one that still holds on a bad day — a crash, a retry, a timeout, an ambiguous input, a model having an off run. Deterministic over probabilistic. Structural over instructional. Independent over self-checked. Visible over tidy.

    Sam: So if I am stuck between two answers, I ask which one survives the bad day.

    Koko: That is the question. Every time.

    Sam: Alright. Honest question — what if I go blank? Like, stem in front of me and nothing comes.

    Koko: Re-read the stem and find the operative verb. Reliably. Always. Must never. Guarantee. Consistently. Prevent. If one of those is in the stem, eliminate every option that is a prompt-wording change, a stronger instruction, a reminder, or a request for more care. You are usually down to one or two after that.

    Sam: And if none of those words are there?

    Koko: If the stem says improve or more consistent, do the reverse — the heavy deterministic machinery is probably the over-correction distractor. And then, before you commit to any answer, check the response-count line. Every single time.

    Sam: Missing that line is such a punishing mistake.

    Koko: It is an attention error, not a knowledge error. And most late-exam mistakes are exactly that — misreading the response count, missing a negation, answering the question you expected instead of the one that is actually there.

    Sam: So slow down at the end rather than speeding up.

    Koko: Deliberately slow down for the last ten items. And here is the other practical rule: if an item takes more than ninety seconds, flag it and move. Coming back to four flagged items with forty minutes left is far more productive than grinding one item at minute twenty.

    Sam: But do not leave blanks.

    Koko: Never. A blank cannot score better than an attempt. Answer everything before the clock runs out, including the flagged ones.

    Sam: And when my first instinct and my second-guess are fighting each other?

    Koko: Take a breath and ask the tie-breaker question. Do not alternate. And one final check — if the option you like requires the scenario to be slightly different from how it was written, it is the wrong option.

    Sam: That is a good one. I have definitely talked myself into answers that way.

    Koko: Everyone has. The scenario is fixed. Your job is to answer the world as written.

    Sam: What about after — what does the result actually tell me?

    Koko: Your report gives you the scaled score against the seven-twenty cut and the percent correct in each domain. So if it does not go your way, you get a genuine study map — the domain percentages show you exactly where the next round of work is. Not a verdict. A map.

    Sam: And if it does go my way?

    Koko: Put the twelve-month renewal in your calendar that same day. It is a free, non-proctored assessment while the credential is live. If you let it lapse, it is a full retake at full fee. That calendar entry is the cheapest ten seconds of admin available to you.

    Sam: That is the kind of thing I would absolutely forget.

    Koko: Which is why I am telling you now, not then.

    Sam: Okay. So to pull it all together — what is the instinct this whole exam is testing?

    Koko: The one that would survive a Tuesday afternoon incident. Predictable, recoverable, observable, and honest about what it does not know. That is the thread through every domain, every decision rule, and every tie-breaker we have talked about today.

    Sam: Predictable, recoverable, observable, honest about what it does not know. I like that as the thing to carry in.

    Koko: Carry it in. You have done the work. Keep sharpening it with the flashcards, the drills, and the simulated exam over at KokoAI Academy on koko knows dot A I. Now go get it.