Skip to main content
    All shows

    Sunday, August 16

    CCA-F Signals & Moves — Part Two

    0:00-:--
    Speed

    Transcript

    Sam: So between sessions I went back through the bootcamp notes and drilled all five domains the way we mapped them in part one. And one thing kept showing up — scenarios where something has to happen every single time, or can never happen under any circumstances. I want that nailed down cold before we go anywhere else.

    Koko: You picked exactly the right one to open on. This is the single highest-yield tell on the exam, and here is the thing — it is actually the same instinct part one closed on. We just got there at the domain level last time. Now we are going to apply it at the level of individual items, which is where it really earns its money.

    Sam: Right, so the signal is the word 'always', the word 'never', or anything that reads as guaranteed compliance.

    Koko: Exactly. The moment a scenario uses one of those words, or any equivalent — 'must run before', 'may not under any conditions', 'requires approval in every case' — it is telling you the acceptable failure rate is zero. And that changes everything about what the answer can be.

    Sam: Because the move that follows from zero failure rate is deterministic enforcement, not better instructions.

    Koko: Yes. Think of it like a seatbelt interlock on a car. The manufacturer does not put a very strongly worded note on the dashboard that says 'please fasten your seatbelt before driving'. They wire the ignition so the car does not start. The mechanism makes the wrong outcome structurally impossible. That is the instinct. A hook, a prerequisite gate, a code-level check — something that the call cannot route around.

    Sam: So give me a cue. 'Every refund over a certain threshold needs approval.' Same move?

    Koko: Same move. You build a gate — the refund request cannot proceed past that threshold without an approval token. The model does not decide whether to ask; the architecture does not let it skip the step.

    Sam: How about: 'The disclosure statement must run before the document is published.' That reads like an ordering constraint.

    Koko: It is an ordering constraint, and the move is still the same. A prerequisite check at publish time — the pipeline confirms disclosure has fired before the publish step is reachable. Not 'remind the model to do the disclosure first.'

    Sam: And 'no agent may touch production without a ticket' — same answer, prerequisite gate on the production action.

    Koko: Every time. The cue changes, the move does not. That consistency is exactly what makes this the highest-yield tell. You do not have to reason from first principles — you see the word, you reach for the mechanism.

    Sam: Now, the trap the exam puts next to this one — the option sitting next to it will say something like 'strengthen the system prompt' or 'add an instruction emphasising that the step is mandatory', right?

    Koko: That is exactly the trap, and it is a well-dressed one because strengthening a prompt does improve compliance. The problem is it improves it to something less than one hundred percent. And the scenario just told you one hundred percent is the requirement. A stronger instruction still leaves a non-zero failure rate. So does adding a reviewer who asks nicely. Adding emphasis does not change the architecture.

    Sam: So the discriminator is: does this response eliminate the failure path, or does it reduce its probability? If it only reduces, it is not the answer.

    Koko: That is a clean way to put it. Eliminates versus reduces. Keep that distinction sharp because examiners will phrase options to sound equally rigorous, and that is the cut.

    Sam: Now — there is a counter-trap here too. Not everything gets a hook. Where is that line?

    Koko: This is important and it often comes up in a second question that follows the first. The rule is: hooks are for things that are stable, checkable, and consequential. A compliance step that must fire before publishing — stable, checkable, high consequence. Put it in the architecture. But preferences about tone, about response length, about how formal the model sounds — those stay in the prompt. If you wire a hook for those, you have built rigidity that nobody can override when circumstances change. And circumstances always change.

    Sam: So the counter-trap is over-engineering — treating a preference as a hard constraint and locking it into a gate where it has no business being.

    Koko: Right. The exam will sometimes offer that as an option — an architecture that enforces something that should be soft. It looks like rigour but it is actually fragility. The instinct we landed in part one was about matching the level of formality to the level of stakes. This is the same rule wearing different clothes: hard enforcement for hard requirements, prompt guidance for everything else.

    Koko: Alright, loop control. This is one of those places where the exam is very specifically testing whether you know what actually drives the agent, not what you assume drives it. So: what is the one and only thing that tells an agent loop whether to continue?

    Sam: The stop reason. Not the content of the reply — the stop reason field on the response object. Tool use means execute the tool call, append the result, go again. End turn means the work is done, exit the loop.

    Koko: Exactly. Think of it like a traffic light. You do not decide whether to drive by reading the expression on the face of the driver in front of you. You read the light. The stop reason is the light.

    Sam: And the trap the exam sets up is a scenario where the model's prose sounds completely done — something like 'I have finished all the steps' — but the stop reason is still tool use.

    Koko: Right, and the trap option will be something like: parse the reply text, detect completion language, and break the loop. That is what they want you to reach for. The correct move is: ignore the prose, read the stop reason, keep iterating.

    Sam: The inverse shows up too. Reply sounds tentative, almost like the model is mid-thought, but the stop reason is end turn. The loop should exit.

    Koko: Yes, and people get tripped up there too. They see hedging language and think they should send another turn to coax out a real answer. But end turn is end turn. The reply is not the signal; the stop reason is.

    Sam: So the discriminator when the question gives you a reply that looks complete is: do not look at the reply at all for the loop decision. Look only at the stop reason field.

    Koko: That is the whole rule, yes. Now, same family — second tell. The scenario describes an agent that either never terminates, or terminates way too early, after just one or two iterations. What does that point at?

    Sam: A broken loop. Either it is reading the wrong field entirely and never sees end turn, so it spins forever — or it has no iteration bound, so one bad response can send it off indefinitely. The fix is two things together: a hard iteration bound, and a proper stop-reason check.

    Koko: Right. The bound is not optional — it is there because even a correctly written stop-reason check can miss edge cases. The bound is your safety net. And the trap beside this one is: the option that says fix the prompt so the model signals completion more clearly. That sounds reasonable, but it puts the control logic in the prose, which is exactly where we just said it does not belong.

    Sam: So the tell for the whole family is: any mention of looping, continuing, or finishing in an agent scenario. The instinct the exam rewards is — go to the stop reason, not the text. And if the loop is misbehaving, the answer is a bound plus a stop-reason check, not a prompt change.

    Koko: That is the whole module in two sentences. Stop reason is the traffic light. If the loop is broken, fix the loop, not the words.

    Sam: Alright, three signals. First one: the scenario says the subagent did not know about X. We covered in part one that subagents start with isolated context — they inherit nothing from the coordinator.

    Koko: Exactly. Think of it like handing someone a sealed envelope. Whatever is inside that envelope is all they have. The coordinator's conversation history, its reasoning, the findings it accumulated — none of that bleeds through. So the move when you see that signal is explicit, complete handoff. You pass the full findings in structured fields, not as a dangling reference.

    Sam: And structured fields matter because they keep content separate from metadata. So the subagent can tell what a finding is versus what context surrounds it.

    Koko: Right. The trap the exam sets next to that move is an option that says something like: summarise the findings above and pass them along. And the problem is that phrase 'above' — there are no findings above in the delegation call itself. It is a prompt pointing at a context window the subagent does not have.

    Sam: So the option looks reasonable because summarising sounds like good hygiene, but it breaks because the reference is empty.

    Koko: Precisely. Now, second signal.

    Sam: The scenario says this takes too long, and the subtasks are independent of each other. The tell for parallelism.

    Koko: And what does the move look like mechanically?

    Sam: You emit multiple delegation calls in a single response — all of them at once, not one at a time. Because if you delegate one, wait for the result, then delegate the next, that is serial by construction. The wall-clock time is the sum of all the subtasks.

    Koko: Good. The analogy I like: you have four errands across town. Sending one courier, waiting for them to come back, then sending another — that is just a slow queue. Dispatch all four at once and the total time collapses to the longest single errand. The exam will offer an option that delegates sequentially but frames it as 'ensuring each result is validated before proceeding.' That framing sounds responsible. It is also the thing that makes your system slow.

    Sam: And the independence of the subtasks is what makes parallel emission safe. If they depended on each other's output, you would have a genuine reason to chain them.

    Koko: Exactly — and that is the discriminator. Independent tasks: emit in parallel. Dependent tasks: chain. Do not let 'being careful' become an argument for serial when the tasks have no data dependency.

    Sam: Third signal: restrict what the subagent can do. The option sitting next to the right answer will say something like scope the coordinator's instructions to limit the child's behaviour.

    Koko: And that is the trap — scoping the coordinator does not scope the child. What a subagent may do is a property of its definition: the description, the system prompt, the tool restrictions you set when you define it. Those are the walls. The conversation that spawns it cannot reach back in and change those walls after the fact.

    Sam: So if the scenario says a subagent is calling tools it should not have access to, the fix is in the agent definition — tighten the tool list there — not in how the coordinator words its delegation message.

    Koko: Right. The instinct the exam rewards: permission lives in the definition. If you see a restriction requirement and the answer touches the coordinator's prompt instead of the child's definition, that answer is wrong regardless of how cautious it sounds.

    Koko: Alright, hooks. This is one of those places where a single word in the scenario does all the work, and if you miss it you pick the wrong lifecycle event every time.

    Sam: The word pair from my notes is 'prevent' versus 'normalise'. Those are the two signals, and they point at completely different sides of execution.

    Koko: Exactly right. 'Prevent', 'block', 'stop it happening' — that scenario is telling you the action must never reach the tool. Think of a bouncer at the door. The bouncer does not wait for someone to cause trouble inside and then write a report. The bouncer intercepts on the way in. So the move is a hook that fires before the tool executes.

    Sam: And the option sitting next to it in those questions will be a hook that runs after execution — same lifecycle area, different timing, and the exam is specifically checking whether you noticed the word 'prevent'.

    Koko: Right, because after execution the action has already happened. You are describing it at that point, not preventing it. One step too late.

    Sam: Flip side: 'normalise', 'reshape', 'the formats are inconsistent' — that is the after signal. The tool has run, something came back, and it needs to be transformed into one consistent shape before the model sees it.

    Koko: Good. A translator at the exit, not a bouncer at the entrance. The instinct the exam rewards is: prevention is before, transformation is after. Those two words — prevent and normalise — should each trigger an immediate directional reflex.

    Sam: The trap on the normalise side would be reaching for the before-hook because it feels more proactive, even though there is nothing to intercept — the inconsistency only exists in the output.

    Koko: Exactly. Now let's move to the delegation pair, because this is where people quietly lose marks. When a parent agent spins up a subagent, there are two hooks available. One fires when the subagent starts. What does it do?

    Sam: It observes. Just observes. It can see that the subagent is starting, log it, inspect what it was handed — but it cannot intervene in what the subagent does.

    Koko: And the second one fires on finish. That one can gate completion — it can block with a reason and send the subagent back to do more work. So if a scenario says 'require approval before the subagent's task is marked complete', that is the finish hook.

    Sam: The discriminator I'd use: observe on start, gate on finish. Neither one rewrites anything.

    Koko: That last part is the trap worth carrying out of this module. Neither hook rewrites the subagent's output. And here is the quiet failure mode: someone writes a hook on the finish event intending to clean up what the subagent returned. It compiles, it runs, and it silently does nothing. No error, no warning — the cleanup just never happens.

    Sam: So the scenario tell for that trap is something like 'modify the subagent's output before the parent sees it' — and the answer is that you cannot do that from either delegation hook. You would need a different mechanism entirely.

    Koko: Correct. The exam loves that one because the code looks plausible. The instinct: if it says rewrite, neither delegation hook is your answer. If it says observe, start hook. If it says gate or require approval, finish hook. Three tells, three moves, zero rewrites.

    Sam: Alright, picking up from where we left part one — domains as decision rules, scoring, all of that is in the rearview. Module five. Tool interfaces. First signal in my notes is the agent picking the wrong tool, or not being able to distinguish between two.

    Koko: Right, and the move there is not what most people reach for. It is not prompting. It is the tool description itself, written specifically for a reader who is deciding between this tool and the one sitting next to it. Differentiation, not eloquence.

    Sam: So the description is doing discrimination work. Not explaining what the tool does in the abstract — explaining what makes it this one and not the neighbor.

    Koko: Exactly. Think of it like a menu in a restaurant that has three dishes that all sound like pasta. The description has to tell you why you would order this one instead of the other two. If it just says 'pasta with sauce,' the waiter — in this case the model — is going to guess.

    Sam: And the trap the exam sets up is adding a line to the system prompt that says something like 'prefer tool A when you see X.' The option will be worded attractively — it sounds like a quick fix.

    Koko: The trap is exactly that. A system-prompt preference is a bandage over a design flaw, and crucially it does not travel with the tool. The moment that tool is used in a different agent or a different context, the preference is gone and the confusion is back. The description travels with the tool always. That is the durable fix.

    Sam: Good. Second signal: the agent treated a failure as data. Meaning it returned a success payload whose text content happened to say something went wrong.

    Koko: Yes, and that is a structural error. The move is to signal failure with the error flag — whatever your interface provides as the failure channel — not to return a successful envelope with bad news inside it.

    Sam: Because the caller is reading the envelope first, not parsing the text for sentiment.

    Koko: Right. If the box is labeled 'success,' the downstream logic treats it as success. You have hidden the failure inside a wrapper that says everything is fine. That is how agents silently do the wrong thing for a long time.

    Sam: Third signal, and this one has a specific trap: the agent retried and retried. The move is structured metadata — a category, a retryable flag, a human-readable explanation — and you read those in that order.

    Koko: In that exact order, yes. The first question is not 'how long do I wait before trying again.' It is 'can a retry ever succeed here at all.' The retryable flag answers that before you do anything else.

    Sam: Because a validation error and a permission error will fail identically on every retry, forever. And a business rule violation does not resolve on any number of attempts.

    Koko: Precisely. Those errors look the same at the network layer. The trap — and the exam will absolutely put this option in front of you — is retrying everything with exponential backoff because backoff feels cheap and safe. It is neither, when the error category tells you retrying is pointless. You have wasted cycles, held resources, and potentially hammered a downstream system for no reason.

    Sam: So the instinct the exam rewards across all three of these is: go to the structural layer first. Description, not prompt. Error channel, not payload text. Metadata category before retry logic.

    Koko: That is the throughline. Patches at the surface level always have a scope they escape from. Structure stays put.

    Koko: Alright, tool selection. This is one of those areas where the exam hides the discrimination inside the phrasing of the scenario, not in the task itself. So let's drill the tells.

    Sam: The one I want to nail down first is the content-versus-path split, because the scenarios that test it look almost identical on the surface.

    Koko: Perfect place to start. Here is the rule in one sentence: if the scenario asks you to find where something is called, find every usage, or search for a string — that is content search, full stop. The question is asking you to look inside files.

    Sam: And the tell for path matching is when the scenario is about matching files by extension or location — the content of those files is irrelevant to the question.

    Koko: Exactly. Path matching is about the shape of the filesystem. Content search is about what lives inside the nodes. Think of it like the difference between reading a library's catalogue by shelf number versus actually opening a book to find a sentence. Same building, completely different operation.

    Sam: The trap the exam sets is putting a path-matching option right next to the content-search answer. The option will say something like 'match files whose names contain the function name' — which sounds related but cannot see inside any file at all.

    Koko: Right. And the worked example that makes this concrete: imagine a deprecated helper function whose name also appears in a changelog entry and in some vendored third-party code. Path matching finds you every file that has that name in its path. Content search finds you every file that contains that string anywhere in its body. Those are different lists.

    Sam: And content search alone still is not enough — you then have to read the surrounding lines to separate a real call site from a mention in a comment or a changelog.

    Koko: That is the two-step the exam rewards: content search narrows to candidates, reading context confirms the real call sites. Skipping the second step is how you mistake a changelog mention for live usage.

    Sam: Now the other tell in the notes — 'change part of a file'. That one is a targeted edit anchored on unique text.

    Koko: Yes. The instinct is: find the smallest unique anchor that identifies exactly the lines you want to change, and edit against that anchor. The mechanism is specificity — you are telling the tool 'this exact text, in this exact spot'.

    Sam: And when the anchor is not unique? The option sitting next to the right answer will say 'retry with more surrounding context and hope the match resolves'.

    Koko: That is the trap. Hoping is not a move. If the anchor is not unique, the move is to extend the anchor until it is, or to replace across all matches deliberately. Those are intentional choices. Adding surrounding text and resubmitting is just retrying ambiguity with more ambiguity.

    Sam: And whole-file write has its own clean tell — that is for whole-file replacement, not for surgical edits.

    Koko: Correct. Whole-file write when you are replacing the whole file. Targeted edit when you are changing a part. The exam will blur that boundary by describing a large change — but the discriminator is not the size of the change, it is whether you are preserving anything in the file. If you are preserving anything, you anchor and edit. If nothing survives, you write.

    Sam: So the first signal in this block — 'it works for me and not for a new joiner.' That is a placement tell, right? Something is sitting at the wrong level.

    Koko: Exactly right. Think of it like a sticky note on your personal monitor versus a sign on the shared whiteboard. If only you can see it, it is user-level configuration. The moment the rule needs to be consistent across everyone on the team, it belongs at project level. New joiner can't see your monitor.

    Sam: So the instinct the exam rewards is: anything the team must share lives in the project configuration. The symptom 'works for me, breaks for them' is the examiner saying the rule is scoped too narrowly.

    Koko: That is the whole move. Scope it to the project, not the person.

    Sam: Second signal: 'this convention applies to files spread across dozens of directories.' The option sitting next to the right answer will say to put a directory-level config in each of those directories, one per folder.

    Koko: Right, and that is the trap the item is built around. Directory-level configuration covers exactly one directory. If the file type roams, the rule has to follow it — so the move is path-pattern rules that travel with the file type wherever it lives. One rule, any path.

    Sam: Like a dress code that applies to the employee regardless of which building they walk into, not a dress code pinned to the door of one room.

    Koko: That is the one. Keep that image — the rule travels with what it governs, not with a fixed location.

    Sam: Third signal: 'check what actually loaded.' That is the diagnostic. The memory diagnostic reports which files are in play — it reports, it does not trigger a reload or re-run the configuration.

    Koko: Correct. And the mechanism matters here: configuration loads from its location on its own. The diagnostic is a read-only view of that state. You use it to see what is actually active, not to force anything to happen.

    Sam: The trap here is adding another copy of the instruction higher up in the hierarchy.

    Koko: Yes — and here is why that is insidious. It looks like it fixes the problem because suddenly the behavior changes, but all you did was hide the placement bug. The original misplaced rule is still there. You have papered over the crack instead of filling it.

    Sam: The instinct the exam rewards is: use the diagnostic to locate the bug, then fix the placement. Not mask it.

    Koko: Exactly. Diagnose, then move. Don't duplicate.

    Sam: Last signal in this block: multi-file change with several viable approaches. The tell is breadth plus multiple defensible designs. That is the trigger to plan first.

    Koko: And the discriminator is important — it is not how consequential the task feels, it is not how nervous you are about it. The trigger is specifically that combination: the change touches many files and there are genuinely multiple defensible ways to do it. That uncertainty about approach is what makes planning the right first move.

    Sam: The option next to the right answer will say to proceed directly because the task is well-defined. Examiners dress it up so that the scope sounds clear even though the approach is not settled.

    Koko: That is the exact dressing. 'Well-defined requirements' does not mean 'settled approach.' When the design space is still open, you plan before you touch files. That is the instinct.

    Sam: So picking up where we left off — output tells. The first one I flagged in my notes is the format keeps varying. That phrase is practically a flare gun in a scenario.

    Koko: It really is. And the move it calls for is demonstration over description. Not a paragraph explaining what you want the output to look like — two or three concrete input-output pairs that show the shape directly.

    Sam: Right, because telling the model 'return a JSON object with a name field and a score field' is describing the shape, but showing it an actual example pair just locks it in faster.

    Koko: Exactly. Think of it like teaching someone to fold a napkin. You can write a very thorough instruction manual, or you can just do it once in front of them. The few-shot examples are the napkin fold. The trap is reaching for more prose — longer, more detailed description — when the format keeps drifting. More words about the shape is not the answer. Concrete examples are.

    Sam: Okay. Second tell in my notes: machine-parseable output. And the move there is a declared schema through tool use.

    Koko: Yes. Once the requirement is 'something downstream has to parse this reliably,' you stop hoping the model produces valid structure and you enforce it by declaring a schema through the tool-use interface. The model is now constrained to emit output that satisfies that schema.

    Sam: Which brings me to the discriminating question, because I know examiners push on this. What does a schema actually buy you?

    Koko: It removes the syntax class of error. Missing bracket, wrong key name, unexpected nesting — gone. The schema makes those structurally impossible.

    Sam: And the option sitting next to it in the exam will say something like 'schema validation confirms the response is correct.'

    Koko: That is exactly the trap. Schema-valid is not the same as correct. A response can satisfy every single structural rule and still be completely wrong about the world. The schema checks syntax. It cannot check whether the facts inside the structure are true. So if a scenario says 'we need to verify the output is correct,' that is asking for a check the schema simply cannot perform — you need a different mechanism for that.

    Sam: Got it. Structural integrity versus factual accuracy. Third tell: validation failed. And the move is retry with the error fed back in.

    Koko: Right. The second attempt needs information the first attempt did not have. You take the specific validation error — whatever the validator actually said — and you include that in the next call. Now the model knows what went wrong.

    Sam: As opposed to a blind retry, which just samples the same distribution again.

    Koko: Which tends to fail the same way. A blind retry is like submitting the same form twice because the first one got rejected — without reading why it was rejected. The distribution has not changed, so the output is likely to have the same problem. The error message is the signal. Feed it back.

    Sam: So the tell in the scenario is probably something like 'the retry also failed' or 'repeated attempts are not resolving the issue' — which is the exam flagging that whoever designed this is doing blind retries.

    Koko: Exactly. That phrasing is the signal that the fix is to add the validation error to the context, not to retry harder or retry more times. Information into the loop — that is the move.

    Sam: So the first signal in this block is the codebase that does not fit — hundreds of files, unfamiliar system. The cue in the scenario is usually something like 'the agent needs to understand a large legacy codebase' and the question is how it should approach that.

    Koko: Right, and the framing I want you to hold is this: the context window is the scarce resource. Not time, not compute — the window. Once it is full, something gets evicted, and if the relevant part got pushed in early and buried, you have lost it before you even got to the answer.

    Sam: So the move is directed and incremental — find the entry points, read only what the actual execution flow reaches, follow the imports that carry it, and carry summaries forward instead of the raw files.

    Koko: Exactly. Think of it like packing a bag for a day hike. You do not bring the whole camping store in case you need something. You look at the route, you pack for that route. The window is the bag. Summaries are compressed gear.

    Sam: And the trap is the option that looks like diligence — read everything so nothing is missed.

    Koko: The trap here is thoroughness wearing a costume. It sounds responsible. It is actually the fastest way to spend the window on files the flow never touches and then evict the one function that mattered. The exam will absolutely dress that option as the careful, professional choice.

    Sam: Moving to the second signal: the agent contradicted something it established earlier in the session. The scenario will flag that as inconsistency or drift. What is the discriminator there — because the option sitting next to the right answer will say 'switch to a larger model with a bigger context window.'

    Koko: And that is the classic misdirection. A larger window delays the problem; it does not fix the mechanism. What you are looking at is context loss — the earlier grounding has degraded or been displaced. The move is summarisation and re-grounding: compress what was established, bring it back to the front of the context explicitly, and anchor the agent to it before continuing.

    Sam: So the signal is the contradiction itself, not the size of the task. The size question is a distractor.

    Koko: Precisely. The contradiction is the symptom, and the symptom points at window management, not model capacity.

    Sam: Third signal: the scenario asks how confident the agent is, or whether the system should trust the agent's output on a sensitive decision. The exam will offer an option that reads something like 'prompt the model to return a confidence score between zero and one.'

    Koko: And the trap here is that a number the model produces about itself is not calibrated to your definition of a false positive. The model does not know your stakes. A stated confidence of zero point nine means something different in a low-stakes autocomplete than in a medical flag.

    Sam: So the move is a stated categorical condition plus provenance for each claim — not a self-reported number.

    Koko: Yes. You are not asking the model how sure it is. You are asking it to state the condition under which its answer holds, and to say where the claim comes from. Those two things you can actually evaluate. A floating number you cannot. That is the instinct the exam rewards: replace self-reported confidence with checkable structure.

    Sam: So across this whole block the through-line is the same: the window is finite, everything you load competes for it, and the correct move is always the one that preserves the relevant signal rather than the one that looks like more coverage.

    Koko: That is exactly it. Coverage is the trap. Precision is the move.

    Koko: So we left part one with the domains mapped as decision rules. Let's drill the cost and throughput signals now, because this is where people donate points they should keep. The anchor signal is a cluster: you see 'cheaper', or 'overnight', or something like 'large volume, nobody waiting on results'. All three of those are pointing the same direction.

    Sam: Asynchronous batch. The processing window is long, latency is not guaranteed, and you submit the work in bulk rather than one request at a time.

    Koko: Exactly. Think of it like dropping a stack of forms in the overnight mailbox. The post office will get to them, but they are not standing there watching you. You get throughput, you get the cost saving, and you give up the handshake. That trade is the whole point.

    Sam: And the instinct the exam rewards is: if nobody is blocking on the response, batch is the right tool. The question I always want to nail down is where the first trap lives, because there are three of them here and they are distinct.

    Koko: First trap is the word 'blocking'. The scenario will look like a batch candidate — maybe it's a pipeline, maybe it's a developer job — but somewhere in the description a person or a system is sitting there waiting for the answer before it can proceed. The trap here is routing that to batch anyway because the cost saving is attractive. Blocking means the latency guarantee matters, so it stays synchronous regardless of volume.

    Sam: Right, and the second trap is the one about scheduling. The scenario will say something like 'results usually arrive quickly' — maybe it observed fast turnaround in testing. The option sitting next to the correct answer will say to schedule against that typical turnaround, which is what the examiners want you to reach for.

    Koko: And the move is to schedule against the worst case, not the average you happened to observe. The processing window is long by design and there is no latency guarantee baked in. 'Usually fast' is not a contract. If you build your downstream timing around the lucky case, you will break on the day the queue is long.

    Sam: Third trap is the agentic loop. The tell is something like 'the workflow calls tools across several turns' — that is multi-turn, stateful, each step depends on the last. Batch handles self-contained requests.

    Koko: Exactly. Batch is a one-way drop. An agentic loop needs to react, branch, call a tool, get a result, decide what to do next. That has to stay synchronous. The trap here is seeing 'this loop is expensive' and reaching for batch to cut costs. You would be handing a conversation to the overnight mailbox and asking it to argue back.

    Sam: That is a clean discriminator. Self-contained equals batch candidate. Multi-turn stateful equals synchronous, cost pressure or not. Now the second signal in these notes is different — 'reduce spend on a working system'. The move there is not the same as choosing batch.

    Koko: Completely different lever. The system already works. The instinct the exam rewards is: use smaller models on the narrow, well-specified steps — the ones where a lighter model is genuinely sufficient — and cache what repeats. Those are targeted reductions that preserve behavior.

    Sam: And the option sitting next to the correct answer will say to remove checks or reduce validation to cut call volume. That is what examiners put there because it sounds efficient.

    Koko: Right, and the trap there is treating reliability as optional overhead. Fewer checks is not a cost optimization strategy on a working system, it is a reliability cut dressed up as one. Smaller model on a narrow step, cache the repeating pattern — those are the moves.

    Sam: So in part one we built the decision framework — domains as filters, scoring as signal. Now we're drilling the toolbox, and the organizing principle here is job, not alphabet.

    Koko: Exactly. The exam never asks you to list tools. It puts you in a scenario and asks which one. So we group by what the tool is for, and the right one pops out.

    Sam: File operations first. Read takes a whole file, write replaces a whole file, edit makes a targeted change. The discriminator is scope.

    Koko: Right. Edit is the default the moment you are touching part of a file. Think of it like a scalpel — you anchor on a unique piece of text and cut exactly there. Write is a bulldozer. You would not use a bulldozer to fix a typo.

    Sam: And the anchor has to be unique. If it is not, you extend it until it is, or you replace across all matches. The failure is loud either way — it never silently edits the wrong place.

    Koko: That loudness is a feature. The trap on file operations is reaching for write because it feels simpler. The option will say something like 'use write to update the configuration block' — three lines changed, everything else overwritten.

    Sam: Finding things next. Grep searches file contents, glob matches file paths. 'Where is this function called' is contents. 'Every JSON file in this directory' is paths.

    Koko: Say that discriminator twice, because it is the single most reliably examined tool distinction on this surface. Contents versus paths. The scenario will give you a phrasing that sounds like it could go either way — your job is to notice which thing you are actually searching.

    Sam: Bash is the execution tool — builds, tests, version control. And it is the one where blast radius is worth thinking about deliberately, because it can do anything the shell can.

    Koko: Good framing. Bash is not dangerous, but it is powerful, so you scope it on purpose. That is the instinct the exam rewards — deliberate scoping, not avoidance.

    Sam: Delegation: the task tool spawns a subagent. And the coordinator has to have task in its allowed tools or nothing spawns. Each subagent gets an agent definition — description, system prompt, tool restrictions.

    Koko: The trap there is assuming the subagent inherits the coordinator's permissions. It does not. The definition is its own declaration.

    Sam: Configuration surfaces: memory files at user, project, and directory level. Project level is the sharing boundary — that is what the team sees.

    Koko: Path-pattern rules handle conventions that follow a file type. Skills live in frontmatter with their own allowed tools, an argument hint that prompts for a parameter, and context forking so a verbose skill does not pollute the session. Slash commands: project-scoped for the team, user-scoped for yourself.

    Sam: Diagnostics: the memory command reports which configuration files are loaded. It reports — it does not trigger loading. And plan mode is the move before multi-file work where several designs are viable.

    Koko: The trap on memory command is treating it as a reload mechanism. It is a window, not a lever.

    Sam: Hooks by verb: pre-tool-use intercepts before the call and can block. Post-tool-use transforms after. Subagent-start observes a spawn. Subagent-stop can gate a finish by blocking with a reason.

    Koko: And none of them rewrite a subagent's output. That is the boundary. The exam will offer an option where a hook modifies what the subagent produced — that is the wrong move.

    Sam: Last one: session control. Resume continues a named conversation. Fork branches from a shared baseline to explore divergent paths. The exam will offer both in the same scenario.

    Koko: And they are not the same move. Resume is linear continuity. Fork is when you need to run two versions of a design without one contaminating the other. The signal is divergent paths — that word is almost the answer.

    Sam: So part one gave us the six shapes — the frame you recognise on sight. This is the execution layer: what you actually do with the case study before you touch a single item.

    Koko: Exactly. And the core problem is that candidates read a brief the way they read a news article — front to back, one pass, absorbing the story. The brief is not a story. It is a specification, and almost every answer is already buried in it. You have to mine it.

    Sam: Four extractions, in order. First one: who acts, and on whose authority.

    Koko: Right. Find every actor and check whether their permissions are their own. If the brief says each user must retain their own upstream authorisation, that sentence just told you: any option that hands the service one shared credential is dead before you finish reading it.

    Sam: And if the brief names a coordinator and a set of specialists, that is the signal that delegation, context passing, and allowed-tools scoping are all in play.

    Koko: Yes. The brief is already telling you the domain before the item asks the question.

    Sam: Second extraction: the absolutes. Approval before a refund, disclosure before publish, no writes to production. The cue I use — if the brief bothers to state something in absolute terms, it is not scene-setting.

    Koko: That is exactly right, and here is why it matters mechanically. Every absolute in the brief is a deterministic-enforcement item waiting to be asked. You already know the move before you read the options. The trap is an option that makes the rule probabilistic — routes it through an LLM judgement call instead of a hard check. If the brief said never, the answer cannot be usually.

    Sam: Third: volume and latency. Long logs, huge payloads, hundreds of files — that is the context-management signal. And a brief that separates a fast interactive path from a large overnight backlog has just handed you the synchronous-versus-batch discriminator.

    Koko: One path has a human waiting. The other has nobody waiting. That sentence is the whole discriminator. Think of it like a diner versus a factory kitchen — one seat is watching you plate the dish, the other does not care if it takes four hours.

    Sam: And the fourth extraction is what the brief confesses about the prototype. Raw payloads returned, agents guessing on empty lookups, failures surfacing as raw text.

    Koko: Every confession is an item. But here is where the trap lives: the fix the brief names is almost never the keyed answer. The exam wants the fix for the class the problem belongs to — not the surface patch the brief describes.

    Sam: So the option that just quotes the brief's own proposed fix is the distractor.

    Koko: Precisely. The item is asking whether you understand the pattern, not whether you can repeat the paragraph.

    Sam: Two closing habits. Re-read the brief between items in the same case, because a detail that looked like colour on the first pass turns out to be load-bearing three items later.

    Koko: The same brief is answering fifteen questions. A detail about batch frequency or an offhand mention of a third-party lookup — that is not flavour text. It is a question you have not reached yet.

    Sam: And the second habit: notice what the brief does not say. If it never mentions compliance, an option that invents a regulatory requirement is adding a constraint nobody gave you.

    Koko: Inventing constraints is exactly as wrong as ignoring them. The trap there is that compliance-flavoured language sounds authoritative, so it feels like the safer pick. It is not safer — it is wrong, because the brief did not authorise that constraint.

    Sam: Mine the spec. The answer is usually already in the room.

    Koko: The answer is always already in the room. You just have to stop reading it like a story.

    Koko: Alright, we have been through every mechanism in this pair of episodes. This is the part you replay in the car park on the way in. I'm going to read a cue, you give me the move in one clause. No elaboration unless something genuinely needs a word of it. Ready?

    Sam: Ready. Go.

    Koko: Always, never, must, guaranteed, compliance, one hundred percent.

    Sam: Enforce — do not instruct.

    Koko: Prevent, block, stop it happening.

    Sam: Intercept before the call.

    Koko: Normalise, reconcile, inconsistent formats.

    Sam: Transform after.

    Koko: Too late by one step.

    Sam: Post-tool-use trying to do prevention's job.

    Koko: Did not know, missing context, summarise the above.

    Sam: Pass findings explicitly.

    Koko: Too slow across independent subtasks.

    Sam: Multiple delegation calls in one response.

    Koko: Restrict what it can do.

    Sam: Put it on the agent definition, not the parent.

    Koko: Spawned but nothing happened.

    Sam: The delegation tool is not in allowed tools.

    Koko: Keeps going, never finishes.

    Sam: Read the stop reason and bound the loop.

    Koko: Stopped early.

    Sam: Same field, same fix.

    Koko: The reply looked final.

    Sam: The reply is not the signal.

    Koko: Where is this called.

    Sam: Search contents.

    Koko: Every file of this type.

    Sam: Match paths.

    Koko: Change three lines.

    Sam: Targeted edit.

    Koko: Anchor is not unique.

    Sam: Extend it or replace across matches.

    Koko: Replace the whole file.

    Sam: Write.

    Koko: Run the tests.

    Sam: Bash.

    Koko: Works for me, not for the new joiner.

    Sam: Project level, not user.

    Koko: Applies across fifty directories.

    Sam: Path rules.

    Koko: Which files actually loaded.

    Sam: The memory diagnostic — it reports, it does not trigger.

    Koko: Adding another copy higher up.

    Sam: That hides the placement bug.

    Koko: Multi-file, several designs.

    Sam: Plan first.

    Koko: Team-wide command.

    Sam: Project scope.

    Koko: Personal command.

    Sam: User scope.

    Koko: Verbose exploratory skill.

    Sam: Fork the context, return the summary.

    Koko: Needs a parameter at invocation.

    Sam: Argument hint.

    Koko: Format keeps varying.

    Sam: Demonstrate with examples.

    Koko: Machine-readable output.

    Sam: Schema through tool use.

    Koko: Schema-valid but wrong.

    Sam: Schema removes syntax errors only.

    Koko: Validation failed.

    Sam: Retry with the specific error.

    Koko: Retried and failed identically.

    Sam: It was never retryable.

    Koko: Empty result set.

    Sam: That is a result, not an error.

    Koko: Failure reported as success text.

    Sam: Use the error flag.

    Koko: Agent picked the wrong tool.

    Sam: Differentiate the descriptions.

    Koko: Fixed it in the system prompt.

    Sam: Bandage on a design flaw.

    Koko: Does not fit in the window.

    Sam: Explore directed, summarise forward.

    Koko: Read everything so nothing is missed.

    Sam: That is how you exhaust the window.

    Koko: Contradicted itself later.

    Sam: Context loss — re-ground it.

    Koko: How confident is it.

    Sam: Its own confidence is not calibrated.

    Koko: Where did that claim come from.

    Sam: Provenance per claim.

    Koko: Cheaper, overnight, nobody waiting.

    Sam: Batch.

    Koko: Someone is waiting.

    Sam: Synchronous.

    Koko: Results usually arrive fast.

    Sam: No latency guarantee — plan the worst case.

    Koko: Multi-turn tool calling.

    Sam: Stays synchronous.

    Koko: Reduce spend.

    Sam: Smaller model on narrow steps, cache what repeats.

    Koko: Resume the investigation.

    Sam: Resume by name.

    Koko: Compare two approaches.

    Sam: Fork from the baseline.

    Koko: Rate limited, raw failures.

    Sam: Structured error, backoff, escalate.

    Koko: Good. Now the case-scan set — read these as a group. Each user keeps their own authorisation.

    Sam: Per-user scoping, never one shared credential.

    Koko: A coordinator and specialists.

    Sam: Delegation, context passing, allowed tools.

    Koko: The brief states a rule in absolute terms.

    Sam: Enforcement item, not colour.

    Koko: Long logs, huge payloads, a repo nobody can hold.

    Sam: Context management.

    Koko: A fast path and an overnight backlog in the same brief.

    Sam: Synchronous versus batch — and the brief just gave you the discriminator.

    Koko: The prototype returns raw payloads or mirrors endpoints one-to-one.

    Sam: That confession is an item.

    Koko: The brief never mentions compliance.

    Sam: An option that invents one is adding a constraint nobody gave you.

    Koko: A detail that looked like colour.

    Sam: Re-read it — the same brief answers fifteen questions.

    Koko: That is the whole run. Now let me give you the one sentence you carry in. We said it in part one when we talked about how the exam is structured, and every module since has been evidence for it: the move is almost always the mechanism, not the wording. The scenario will use softer language. The right answer will name the exact mechanism. That gap is where points are lost.

    Sam: And the trap in a lot of those cases is an option that gets the goal right but names the wrong layer — right outcome, wrong mechanism.

    Koko: Exactly. Walk in holding that sentence. On the way there, run the cues back to yourself — just the signal, just the move. If one sticks, that is the one to look at again. The flashcards and the quiz for both parts of this lesson are at KokoAI Academy, that is koko knows dot A I. You have done the work. Go get it.