CCA-F Exam Day — Part Three

Chris: So here is where I am. Part one is solid, the drills from last time are solid. What I keep coming back to is — what did we not cover? What is the thing that lands on the day and I have not pre-loaded it?

Koko: Good question to ask right now rather than Tuesday morning. The honest answer is: coverage is not even. The exam draws four scenarios from a pool of six, and most practice material leans hard on the same three or four. Customer-support agent, multi-agent research system, CI pipeline, code generation — those get drilled to death. Two shapes get far less time.

Chris: Which two?

Koko: Developer productivity — an agent exploring an unfamiliar or legacy codebase, using built-in tools and MCP servers. And document extraction — high volume, strict target schema, limited human review capacity. If either of those lands on your paper, it is the one you have drilled least. That is worth ninety seconds of pre-loading right now rather than a surprise on the day.

Chris: All right, load me on developer productivity first.

Koko: The instinct is precision over breadth. Think of a new hire on their first week in a giant warehouse. They do not walk every aisle — they ask where the thing is kept, go to that shelf, read the label, take what they need. The agent does the same: content search to find where something is called, path matching to find files by type, then read incrementally along the execution flow rather than pulling the whole repository into the window.

Chris: So it is window survival — delegate verbose discovery so the context does not fill up with noise.

Koko: Exactly. And when it is time to change something, reach for a targeted edit anchored on unique surrounding text rather than a whole-file write. The trap here is designing for completeness — a broad read-everything sweep that looks thorough but burns the window and loses precision.

Chris: Right. Precision over breadth, delegate the verbose discovery, targeted edits. That is the developer productivity shape.

Koko: One line, perfectly done. Now document extraction. Different pressure entirely. The instinct here is schema honesty. Imagine a form processor that, when a field is missing, writes a plausible-sounding guess in neat handwriting. That is the worst possible outcome when downstream systems are treating the output as fact.

Chris: So the schema itself has to carry that honesty — optional and nullable fields for values a document may legitimately not contain.

Koko: Yes. And when a value is genuinely ambiguous, an explicit unclear marker in the output rather than a guess. If a document shows both a stated total and numbers that imply a different calculated total, the extraction reports both — it does not silently pick one.

Chris: And retries carry the specific validation error, not just a generic re-run. So the system knows exactly what was wrong.

Koko: Right. And confidence is calibrated per field, not one global score across the whole document. A high-confidence address line and a low-confidence figure in the same extraction need different signals for the reviewer. One global score hides that.

Chris: So document extraction: schema as tool input, nullable fields, explicit unclear values, report both totals, specific errors on retry, per-field confidence. The trap being — the option that chooses a plausible guess over an honest unclear, because it looks cleaner.

Koko: The exam will absolutely put that option right next to the correct one. It will look like the more decisive, production-ready answer. It is not. Schema honesty is the whole game in that shape. Pre-load both of those and neither scenario is a surprise.

Chris: Okay, so this is one of the things neither part one nor part two touched at all — there is actual arithmetic on some items.

Koko: There is, and almost nobody warns candidates about it. The shape that recurs is: you are given a deadline, you are given a worst-case processing window, and the question asks something about scheduling. The sum you need is simple — deadline minus worst-case processing window equals the latest moment anything can be submitted.

Chris: So if results are needed within thirty hours and processing can take up to twenty-four, everything has to be submitted within the first six hours.

Koko: Exactly right. Six hours is your entire submission window. Think of it like shipping a parcel before a public holiday. If the depot closes in thirty hours and transit takes up to twenty-four, you have a six-hour window to get the box there — not a thirty-hour window.

Chris: And if you are submitting continuously rather than in a single batch?

Koko: You slice that six-hour window. Every batch that goes out has to fit inside a slot sized so nothing launches later than the worst case allows. The window shrinks, the slices get smaller, but the constraint is the same — you are always working backwards from the deadline, not forwards from now.

Chris: Here is the discriminator I want to nail down. The stem will have an observed turnaround — something like, in testing it usually finished in four hours. Why does that number not enter the calculation?

Koko: Because usually fast is not a contract. The published worst-case window is the number with a guarantee attached. Your observed four hours is a data point with no guarantee attached. You can admire it, you can mention it in a design review, but you cannot schedule against it.

Chris: And the option sitting next to the right answer will say something like schedule against the typical turnaround since it is usually much faster.

Koko: That is precisely what examiners put there. It sounds efficient, it is what a lot of teams do in practice, and it will fail the moment the system hits its worst case on a day that matters. The trap is comfort with averages. The instinct the exam rewards is: find the guaranteed number, use only that.

Chris: So the procedure the moment I see a deadline and a processing window in the same stem is to do the subtraction on paper before I read the options.

Koko: Do it twice, actually. Write it out, check it, then look at the options. The correct answer will match your number. If it does not, recheck your arithmetic before you second-guess the approach — the approach is right. And if any option references the typical or observed turnaround instead of the published worst case, that is the wrong answer regardless of how reasonable the rest of the wording looks.

Chris: Work backwards from the deadline, use the guaranteed window, ignore the observed average. Simple once you know to expect it.

Koko: Simple and almost always correct. The exam is not testing your arithmetic — it is testing whether you know which number carries a guarantee. Get that instinct right and the sum takes ten seconds.

Koko: Right, so I want to be upfront about something. Parts one and two taught three rules hard — because hard is how they stick — and each one has a boundary we did not mention. Not because we were wrong, but because the sharp version lands first and the nuance lands later. This is later.

Chris: Good. Because I noticed the escalation rule in part one felt almost binary. Tone does not trigger escalation, the stated request does. That held up all the way through the drill last time. But I kept wondering about the in-between — someone is clearly frustrated, they have not explicitly asked for a human, so what actually happens?

Koko: That is exactly the boundary. The full shape has three steps, and the exam will test whether you know all three. First, acknowledge the frustration — name it, do not skip it. Second, offer something concrete, a real resolution not a holding message. Third, escalate when the person reiterates that they want a human, after you have done those two things.

Chris: So a first expression of dissatisfaction is not yet the trigger. It is the second ask — after something real has been offered — that flips it.

Koko: Exactly. And the part from part one that does not soften at all: an explicit request for a person is still immediate and non-negotiable. That has not moved. The trap here is an option that has the system offering one more resolution attempt after someone has already said 'I want to speak to a human.' That is the wrong move, and it will look reasonable because it is technically helpful.

Chris: Okay. Second one — self-review. Both episodes were pretty firm that a session reviewing its own output is the weaker check. Independence beats effort. That one I have no quarrel with on correctness questions. But I noticed the notes draw a line somewhere.

Koko: Right. The rule holds when the question is whether a subtle defect crept in — did we miss something, is the reasoning sound, did we overlook an edge case. For that, you want independent eyes. But there is a different job, which is completeness. Not 'is this correct' but 'is this consistently complete' — is the policy context present, are the timelines stated, is there a next step for every action item.

Chris: And for that, checking against explicit concrete criteria is a legitimate self-directed check, because the criteria are doing the work that independence would otherwise do.

Koko: That is the clean way to say it. The trap is treating those two jobs as the same job. The option next to the right answer will often swap them — it will suggest independent review for a completeness checklist, or self-assessment for a correctness question. The exam is testing whether you can tell those apart.

Chris: Third one. Ambiguity. Part one was clear: ambiguous identity is a question for the person, not a problem for the system to solve, because guessing the closest match has consequences. But low-stakes build tasks feel different to me — and I think the discriminator is stakes and reversibility.

Koko: You have it. That is the exact discriminator. If a wrong guess costs something you cannot take back, you ask. If nothing irreversible happens when you are wrong, you state your assumptions and proceed — because then the person stays informed, they can correct you, and no damage is done.

Chris: So it is not that ambiguity always means stop. It is that irreversible consequences mean stop. Low-stakes, reversible, build context — state and proceed is actually the right answer.

Koko: And the trap there is applying the identity-ambiguity rule universally. The option next to the right answer will say to pause and ask for clarification even when the task is a draft that can be revised in ten seconds. That looks careful. It is actually the wrong instinct in that context — it blocks progress without protecting anything.

Chris: So the three boundaries are: escalation has a three-step shape before it fires, but an explicit human request still fires immediately. Self-review is fine for completeness against explicit criteria, not for correctness. And ambiguity means ask when the cost of a wrong guess is irreversible, state and proceed when it is not.

Koko: That is the appendix to parts one and two. The rules were right. The edges are what you just said.

Chris: So I've done part one, I've drilled part two, and what I keep noticing is there are these small mechanical facts underneath everything — the kind of thing that isn't a domain, it's just how the system actually works. Is that what this block is?

Koko: Exactly that. Five things. None of them are new topics — they're the load-bearing screws behind everything you already have. And the exam will poke at them sideways, through scenarios, not by asking you to define them directly.

Chris: Start with memory then, because that one keeps coming up. No memory between calls — we said it in part one. But what's the mechanical reason behind it?

Koko: Right, so every call carries the whole conversation as its input. There is no separate memory store being consulted — the history is literally resent each time. Think of it like a fax machine that has to retransmit every previous page before it can send the new one. That's why a long conversation costs more and takes longer. The growing history is the cost. This is the single fact sitting underneath everything both episodes said about protecting context.

Chris: So context length and latency and cost are all the same problem wearing different hats.

Koko: One root, three symptoms. Good. Now stop reasons — part two ran the loop pair, end-turn and tool-use, but the exam has more of these. What did we leave out?

Chris: The option next to end-turn in a lot of questions will be the output limit — and I've seen people confuse 'the model stopped' with 'the model finished'. Those are different things.

Koko: Completely different. Output limit means the response was cut short, not finished. Then you also have a stop sequence being hit, a refusal, and a pause-turn — which is a server-tool loop that hit its iteration limit and needs you to re-send. And if the context window is exceeded, that's its own stop reason too. The trap here is treating any stop as a clean finish. Only a stop sequence counts as a clean finish. The loop pair still drives the loop — that part you drilled last time stands — but now you know the full map.

Chris: Okay. Config files importing other files by path — that came up in the notes and I want to make sure I have it right. The exam will frame this as saving context, won't it.

Koko: That's exactly where it goes wrong. Importing a file by path keeps the config modular — the standards live in their own files, they get pulled in by reference rather than copied inline. It organises the structure. But the trap here is assuming it saves context, because it doesn't. Imported files load along with everything else. Loading only when relevant is what path-scoped rules and skills are for — that's the actual mechanism for keeping things out of context.

Chris: So 'modular' and 'lean on context' are separate levers.

Koko: Separate levers. Now, personal skills shadowing team skills — this one is quiet and dangerous.

Chris: The scenario will probably look like someone wondering why they stopped getting team updates. And the option next to it will say 'rename the team skill'.

Koko: The direction is the opposite. A personal skill with the same name as a shared team skill takes precedence and silently shadows it — you stop receiving the team's improvements without any notification. The fix is to give your personal variant a different name. The trap here is any answer that touches the team-side file, because you don't own that. The action is always on your end.

Chris: Last one — batch identifiers. Each request in a batch carries an ID that ties the result back to its input, which is what lets you resubmit only the failures.

Koko: That's it. The instinct the exam rewards is 'resubmit selectively, not wholesale'. If a question gives you a partial batch failure and asks what you do next, the ID is why you can target just the failures. The trap here is any answer that re-runs the whole batch — that's the expensive wrong move, and it ignores what the identifier is for.

Chris: Five screws. All of them load-bearing. Let's keep going.

Koko: So here is what the first two parts did not give you: the steps in order. You have all the pieces from part one and part two — the domain logic, the signal-to-move drilling — but we never strung them into a single procedure you run the same way on every item. That is what this is.

Chris: Right, and the order matters, you said. That is the bit I want to nail down. What breaks first if I run them out of sequence?

Koko: You lose items you understood. Which is the most painful kind of loss. So: step one, before you read a single option, read the response-count line. How many answers does this item want?

Chris: Because if it wants three and I pick two confidently, I have already lost it regardless of whether my two were correct.

Koko: Exactly. Think of it like a combination lock. You can have every digit right and still not open it if you stop one short. The response-count line is that last digit. Read it first, not last.

Chris: Step two is the operative verb. We drilled the signal words last time — always, never, must, guarantee, prevent, consistently. If one of those is in the stem, I am in deterministic territory.

Koko: And the moment you spot one, you eliminate anything that is just a wording change, a stronger instruction, a reminder, or a nudge toward more care. Those are not mechanisms — they are wishes. You are usually down to one or two options before you have done any real work.

Chris: The trap there, as the exam builds it, is that the soft options sound responsible. The option sitting next to the right answer will say something like 'remind the system to handle this carefully' or 'add a note to the prompt encouraging accuracy.'

Koko: Precisely. And the mirror trap runs the other way: if the stem says improve or tends to or more consistent, the heavy deterministic machinery is the over-correction sitting in the list to catch you. The instinct the exam rewards is matching the strength of the mechanism to the strength of the claim in the stem. No stronger, no weaker.

Chris: Step three is the brief. The same case brief covers around fifteen items, so anything stated up front — an approval requirement, a latency constraint, who holds which authorisation — is often the whole answer.

Koko: And notice what the brief does not say. An option that invents a constraint nobody gave you is wrong in exactly the same way that ignoring a stated one is. The brief is the boundary of the world. Stay inside it.

Chris: That one catches people who are being thorough in the wrong direction — adding safeguards that seem sensible but that the brief never authorised.

Koko: The trap is good intentions without a mandate. The exam is not asking what you would add if you were in charge — it is asking what the brief supports. Those are different questions.

Chris: And if two options survive all of that, step four: which one still holds on a bad day. Not the average case — the degraded case, the failure case, the edge.

Koko: Right. The option that is only correct when everything is working is not the answer the exam wants. Then you commit — and you read the response-count line one final time before you mark anything.

Chris: So let me run it back in order. One: response-count line before the options. Two: find the operative verb, match mechanism strength to claim strength, eliminate soft fixes if the stem is deterministic. Three: mine the brief for stated constraints, reject any option that invents one. Four: if two survive, ask which holds on a bad day, then commit, then re-check the response-count line.

Koko: That is the whole procedure. Four steps, each one capable of ending the item early, in the order that saves the most time and loses the fewest points. You do not need to be faster — you need to stop in the right place.

Koko: That is the whole series. The briefing in part one, the drills in part two, and this appendix. Nothing new goes in from here.

Chris: Which is exactly right, because the last hour before the exam is not the time to learn anything. That is the five domain sentences from part one and the cue list from part two, and nothing else.

Koko: Precisely. You are not cramming a new chapter at that point. You are warming up an engine that is already built.

Chris: So let me say what I am actually taking into that room, and you tell me if anything is missing. The move is almost always the mechanism, not the wording. When two options both look defensible, I take the one that still holds when something goes wrong.

Koko: That is the line. That single sentence covers more questions than any individual topic we touched.

Chris: And the four qualities that test it: predictable, recoverable, observable, and honest about what the system does not know.

Koko: Right. Think of it like a bridge. A good bridge is not just strong when the weather is perfect. It is designed so that if one cable fails, the deck does not vanish. Predictable means you know what it does. Recoverable means a failure is not final. Observable means you can see it happening before it becomes catastrophic. And honest about limits means the bridge has a posted weight limit rather than quietly hoping nobody drives a lorry across it.

Chris: And on the exam, the option that sounds clever but quietly assumes everything goes right is almost always the trap.

Koko: Every time. The examiners are not testing whether you can design for the happy path. They are testing whether your design survives the unhappy one.

Chris: The procedure from this episode is the other thing I am keeping: read for the signal, name the move, check the adjacent option, then ask which one still holds under failure. That sequence makes the discrimination visible instead of leaving it as a feeling.

Koko: And that is the point of a procedure. You do not want to be inventing a method under time pressure. You want to be running one you have already rehearsed.

Chris: Part two did all the rehearsal. This just gave it a wrapper.

Koko: A very tidy wrapper. Right. There is nothing left to add, and I mean that as a compliment to where you are, not an excuse to stop early.

Chris: Good to hear.

Koko: Go pass it. And the day you pass — that same day — open your calendar and mark the twelve-month renewal. Do not rely on a reminder email arriving at the right moment. Put it in yourself while the feeling of having just done this is still fresh.

Chris: Noted. Day one, renewal date goes in.

Koko: The flashcards and the practice quiz for the whole series are at KokoAI Academy on koko knows dot A I. If anything from these three episodes felt like it needed one more pass, that is where to go.

Chris: Which is also where I would send anyone else who is sitting this. It is a clean place to drill without having to untangle which part covered what.

Koko: You have done the work. You know the domains, you know the cues, you know the procedure, and you know what the exam is actually rewarding. That is everything. Go show them.