Claude Models, Prompting & Context Engineering

Koko: Here is the one instinct that governs this entire domain: match the lever to the observed failure, and try the cheap lever first. That's it. That's the whole domain in a sentence. Everything else we cover today is either an application of that instinct or a description of what happens when you ignore it.

Chris: And I want to be specific about why that instinct is so easy to violate at this level. In a design review someone asks why you didn't just upgrade the model, and if your answer is 'we tried the prompt work first and measured the gap,' you're in good shape. If your answer is 'we went straight to the bigger model because the task felt hard,' you're in trouble. The exam is testing exactly that reasoning chain.

Koko: That's the frame for Domain Two. Thirteen percent of the Professional exam, the smallest technical domain, and the one where habits from the Foundations level mislead the most. At Foundations the question is 'what does this technique do.' At Professional the question is 'which lever do you pull, and what does pulling it cost.' So let's start with model choice, because that's where the instinct is easiest to get wrong.

Chris: Walk me through the two documented starting points, because I want to be precise about when each one is defensible.

Koko: There are two, and the documentation names both explicitly. The first is efficiency-first: you start with a fast, economical model, you test thoroughly against a real evaluation set, and you upgrade only where an actual capability gap shows up. The second is capability-first: you start with the strongest model available for the task, you optimize the prompt for that model, and then you move down as the workflow gets more constrained and you understand where the ceiling actually is.

Chris: And the criteria you're weighing across both paths are the same four things: capabilities, speed, cost, and effort. The docs are explicit that tuning the effort parameter is often a better lever than switching models at all. I want to flag that because it surprises people — you can get meaningful quality improvement just by adjusting how much thinking the model does, without touching the tier.

Koko: Exactly right. Now let's make that concrete with two scenarios, because the discriminator between efficiency-first and capability-first is not 'which one feels safer.' It's the economics of a wrong answer.

Chris: Support summarization versus contract review. I've shipped both of those shapes.

Koko: Perfect examples. Support summarization starts efficiency-first. Volume is high, the task is bounded — you're extracting a structured summary from a ticket, not reasoning about novel edge cases. A missed summary costs you a slightly worse handoff. You run your evaluation set, you see where the smaller model falls short, and you upgrade only those steps.

Chris: And contract review inverts that completely. You start capability-first because a missed indemnity clause or an unfavorable termination condition is genuinely expensive. You want to know the ceiling before you start negotiating it down. You run the frontier model, you tune the prompt against it, and then you ask: where can we safely step down without giving up the coverage we need?

Koko: And in both cases you keep the same evaluation set throughout, so when you compare across tiers you're comparing apples to apples. The docs call building that evaluation set the most important step — before you've picked a model, before you've written a system prompt, you know what 'good' looks like and you can measure it.

Chris: That's the part teams skip, and it's exactly why they can't justify their model choice later. If you don't have the eval, you don't have the argument.

Koko: Here's the analogy. Think of an escalation desk at a call center. First line handles the volume — most calls, most of the time. The specialist takes the hard one. Nobody staffs the whole desk with specialists because the cost is absurd. And nobody sends every call to the specialist to be safe, because now the specialist is buried in password resets. The specialist is expensive precisely because they're reserved for the cases that need them.

Chris: And the escalation logic is decided in advance, not on a hunch. You have criteria: if the issue type is X, escalate. That's what your evaluation set gives you — the criteria.

Koko: The takeaway, and carry this into the room: build the evaluation set first, then let the numbers pick the tier. Before you reach for a different model, reach for the effort parameter and for prompt work. And when you do use multiple tiers, consider pairing them so most of your tokens bill at the lower rate.

Chris: The pairing point is worth naming precisely. The documented multi-model patterns give you two shapes. An executor that escalates hard decisions to a stronger advisor. And an orchestrator that delegates bulk work to cheaper workers. In both cases the frontier model is only sitting on the cases where the difficulty actually is.

Koko: Which brings us to the trap, and it's a clean one. The exam will present a system and offer an answer that gives one model to the whole product. One model, end to end, across every step. And the answer sounds reasonable because it's simple.

Chris: The version of this I keep seeing on teams is: 'we standardized on the flagship model so we don't have to manage routing logic.' Which sounds like an engineering discipline decision, but what it actually means is you're billing flagship rates for steps that a smaller model handles fine, and you never built the eval that would tell you which steps those are.

Koko: The misconception is that model choice is a product-level decision. It's a step-level decision. Each step in your pipeline can and should be evaluated independently. The frontier model goes where the difficulty is. Everywhere else, cheaper and faster is not a compromise — it's the correct answer.

Chris: So when the exam asks you to defend a model selection, the answer has three parts: what the eval showed, why this tier clears the bar, and where in the pipeline that tier actually sits.

Koko: That's the instinct. Match the lever to the failure. Try cheap first. Let the eval pick the tier, not the feeling that the task is hard. That framing runs through every module in this domain, so it's worth having it locked in before we go further.

Koko: So here is the thing that trips up a lot of architects who have been doing this a while. Zero-shot, few-shot, and chain-of-thought are not a quality ladder. They are not small, medium, and large. They are different tools for different failure modes, and the exam will absolutely test whether you know which failure calls for which tool.

Chris: Right, and the failure-first framing is doing real work there. You do not reach for a technique because the task feels important. You reach for it because you observed a specific kind of output breaking.

Koko: Exactly. So let me just state the fact plainly. Zero-shot stays the default for tasks that are unambiguous — the model knows what you mean, the output shape is obvious, it works. Few-shot is your fix when you are seeing inconsistent formats or edge-case drift — the model is doing the right thing conceptually but the shape keeps wandering. And for reasoning failures — the model is reaching a wrong conclusion, not just a weirdly shaped one — that is where adaptive thinking comes in, with an effort sweep against your eval set.

Chris: The effort sweep piece is worth holding on to. You are not just turning thinking on and walking away. You are running your evals at different effort levels and finding the point where the reasoning errors disappear without paying more than you need to.

Koko: Yes. And manual chain-of-thought — the explicit step-by-step prompt — is the fallback for when thinking is off or not available on that path. It is not the normal control lever for a thinking-capable model.

Chris: So walk me through the concrete scenario, because I think that is where the distinction really lands.

Koko: Sure. You have two steps in a pipeline. First step is an extraction step — pulling structured data out of unstructured text. The output shape keeps drifting: sometimes a field is a string, sometimes a list, sometimes it just omits an edge case. That is a format and edge-case failure. You add three diverse examples that cover the tricky cases, and it settles. That is few-shot doing exactly what it is supposed to do.

Chris: And the second step?

Koko: Second step is a pricing-eligibility decision. The model is producing the wrong verdict — not inconsistent formatting, wrong reasoning. So you enable adaptive thinking, you sweep effort levels against your eval set, and you also constrain what the model returns: the conditions it checked and the final verdict. Clean, auditable, right.

Chris: That last piece matters a lot in practice. You are not asking it to show all its work. You are asking it to return structured evidence — the specific checks performed, the verdict. That is different from just getting a stream of internal reasoning back.

Koko: That distinction is going to come back when we hit the trap, so hold that thought. First, the analogy, because this one is genuinely useful. A clinician picks the treatment from the symptom, not from a potency ladder. Nobody prescribes the strongest thing on the shelf just because the patient is important. You diagnose, then you prescribe.

Chris: And the failure mode of prescribing by importance is real on engineering teams. A task feels high-stakes so someone reaches for the most visible, heavyweight technique without checking whether it actually addresses what is breaking.

Koko: Which leads directly to the takeaway. Classify the failing output before you touch the prompt. Is the output the wrong shape? Examples. Is it the wrong conclusion? Thinking plus an effort sweep. Do you need an auditable record? Ask for the evidence, the checks performed, the decision rationale — not just more tokens.

Chris: The version of this I keep seeing on teams is people defaulting to a step-by-step prompt the moment a task feels complex, even on a model that reasons natively. The reasoning is there, you are just adding latency and output tokens and sometimes nudging it down a path it would have handled better on its own.

Koko: That is exactly the trap, and it has a sharper edge than it looks. Manual chain-of-thought adds output tokens and latency, and it can over-prescribe a reasoning path. If the model supports thinking, the right move is to enable thinking and tune effort — not to write out the steps yourself and hope the model follows them. The explicit step-by-step prompt is a deliberate fallback. It is not the default.

Chris: And the option next to it on the exam will probably frame chain-of-thought as the safe, auditable choice for any reasoning task — implying that visible intermediate steps are what make a decision auditable.

Koko: Right, and that is the misconception to name clearly. Do not ask for private thinking as your audit artifact. The model's internal reasoning process is not the same as an audit record. If auditability is a real requirement, you request the evidence — the specific conditions checked, the logic applied, the verdict reached. That is what a downstream system or a human reviewer actually needs. Intermediate steps in the output are not a substitute for that.

Chris: So to put it in one sentence: diagnose the failure mode first, match the technique to that diagnosis, and when auditability is the goal, ask for structured evidence — not visible thinking.

Koko: That is the instinct the exam is testing. Nail that and you are in good shape for this domain.

Koko: So here is the fact, stated plainly. The context window is a budget, and in a long-running agent there are exactly four things competing to spend it: tool definitions loaded upfront, accumulated tool results, conversation history, and thinking blocks. Four claimants, four documented remedies, and the remedies are not interchangeable.

Chris: And the docs are pretty specific about which fix maps to which source. Tool search keeps definitions out of the window until something actually needs them. Context editing removes old tool results server-side once they have done their job. Compaction condenses the conversation history server-side. And prompt caching — and this is the one that trips people up — prompt caching does not shrink the window at all. It changes what the repeated part costs, not how many tokens it occupies.

Koko: Exactly right. Caching is a cost lever, not a size lever. Now, the scenario, because this is where it gets real. Picture a research agent that has run for two hundred turns and is sitting at the ceiling. The team reaches for context editing. They set a trigger threshold, a keep count so the last few tool results survive — because some results are still load-bearing — and they exclude one specific tool whose output they cannot afford to lose.

Chris: The keep count and the exclusion list make sense. What I want to make sure people understand is the minimum-clear setting, because that one has a non-obvious reason behind it.

Koko: Yes, and this is the mechanism worth internalizing. Clearing tool results server-side invalidates the cached prefix. If you run a clearing pass and it recovers fewer tokens than the cache rewrite costs, you have taken a net loss on that operation. The minimum-clear setting exists so you do not trigger a cache invalidation for a trivial gain. Clearing is not free.

Chris: So the setting is basically a floor that says: do not bother unless we are recovering at least this much, because the rewrite will eat whatever we saved.

Koko: Exactly. And one more thing on remedies before the analogy — the docs now prefer compaction over context editing for most long-running conversations. Compaction condenses history server-side, and for agents with deep conversation trees that is usually the heavier source of bloat. Context editing targets tool results specifically. Knowing which one is fatter in your particular agent determines which lever to pull.

Chris: Which is why diagnosing before acting is the move. I have seen teams reach straight for whatever setting is closest to the one they already configured, and it does not help because the problem was in a different claimant entirely.

Koko: That is the whole instinct, and let me give you the analogy first and then say it directly. Imagine a suitcase with a weight limit and four people packing it. When it will not close, you cannot fix that by weighing the bag again. You have to find out who packed what. Upgrading to a bigger suitcase just delays the argument — it does not tell you that one person snuck in a full winter coat.

Chris: And the four people have different things in there. One brought tool definitions, one brought a pile of old tool results, one brought two hundred turns of conversation, and one brought thinking blocks. You need to know the breakdown before you decide whether to leave something at the hotel or compress it.

Koko: Exactly. So the takeaway — and this is the instinct to carry into the exam room — find out where the tokens actually went before choosing a remedy. The four fixes address four different sources. That minimum-clear setting exists precisely because clearing is not free.

Chris: The discriminator question I would press on is: what is the difference between a context problem you fix with compaction versus one you fix with context editing? And the answer is: history versus tool results. Compaction is for the conversation growing too long. Context editing is for accumulated tool results that have already served their purpose. Separate sources, separate remedies.

Koko: Good. Now the trap, and this one is clean and dangerous. The option sitting next to the right answer on this exam will say: use a model with a larger context window. And the logic sounds reasonable — more room, problem solved.

Chris: The version of this I keep seeing on teams is people treating a window upgrade as a fix rather than a deferral. They ship to a bigger model and the problem comes back six weeks later at turn three hundred instead of two hundred.

Koko: Because a longer window changes when you hit the wall, not whether you hit it. And there is a second cost hiding in that choice: attention and performance degrade as irrelevant material accumulates. Context editing and compaction exist not only because they save money — they exist because removing dead weight improves focus. A model that is not wading through two hundred turns of stale tool results is doing better work on what is in front of it.

Chris: So the misconception is that context management is purely an economics problem. It is also a quality problem.

Koko: That is the one to write on the wall. Bigger window, same dead weight, degraded attention. The fix is always: diagnose the claimant, apply the matching remedy, and respect the cost of the remedy itself.

Koko: Alright, last one. Two reuse mechanisms, one distinction, and if you get this distinction you get the exam question. Here is the fact: prompt caching and Agent Skills both save money, but they save different things. Prompt caching pays less for the same tokens — you are still sending a large prefix, but the cache read costs a fraction of the base input price. Agent Skills send fewer tokens — the guidance that is not needed for this request never arrives in context at all.

Chris: So caching is a volume discount and Skills are scope reduction. You are buying the same groceries cheaper versus not putting the thing in the cart.

Koko: Exactly right. And the pricing detail matters here. Cache reads come at a fraction of base input price, cache writes carry a markup. With the default five-minute cache, break-even lands on the second hit. With the one-hour cache, you need a third hit to come out ahead.

Chris: Which means if a prefix only appears twice in five minutes, the five-minute cache is just barely worth it — and if it appears once, you paid more than you saved.

Koko: Correct. So traffic pattern matters as much as prefix length. Now let me put both mechanisms together in a concrete scenario. Document-review service. Every request has the same system prompt, the same policy extract, the same tool definitions — all of that is stable. Then comes the variable document, which is different every time. You place one cache breakpoint on the last block that is identical across every request. Everything before that point is read from cache. The document goes after. That is the whole move.

Chris: And separately, this service has three domain procedures — say revenue recognition, contract compliance, and data privacy. All three used to live in the system prompt, burning tokens on every call regardless of what the user asked about.

Koko: Right. So those three move into Skills. Progressive disclosure means at startup only the Skill's name and description sit in context. The full instructions load when that Skill is triggered. A request about revenue recognition loads that one procedure. The other two never arrive. You did not defer them — you did not bring them.

Chris: That is the analogy I keep using with teams: caching is a season ticket — same journey, lower fare. Skills are packing lighter — you left the thing at home.

Koko: I love that. Keep it. Now the takeaway for the exam, and this is precise so stay with me. Put the cache breakpoint on the last block whose prefix is identical every time. If you place the breakpoint after anything that varies — a timestamp, the user's name, the user's question — the lookback never matches, and you pay the write penalty on every single request. The cache is writing and never reading.

Chris: I have seen this in the wild. Someone places the breakpoint at the bottom of the prompt for convenience, and the very last line is something like 'Today is' followed by the current date. Cache miss every time, write charge every time.

Koko: Perfect illustration. And for Skills: reach for them when guidance is conditional. If a block of instructions only applies to some requests, it should not be in the system prompt paying full price on every call.

Chris: The version of this I keep seeing misconfigured on teams is the Skill description treated like documentation — a summary of what the Skill contains rather than a trigger signal.

Koko: That is the trap, and it is worth saying clearly. The description is trigger metadata. It is the only part of the Skill that is always in context. It has to say both what the Skill does and when to use it, in the words a real request will actually contain. If the description says 'handles accounting standards' and users say 'how do I book this revenue,' the model may not connect those. The Skill sits there unused while the model improvises from whatever else is in context.

Chris: Write the description for the incoming message, not for yourself.

Koko: That is the instinct. Write it in the language of the request.

Chris: Okay. Before we close — you want me to run the order of operations?

Koko: Please.

Chris: Classify the failure first. Then try a parameter — temperature, stop sequence, whatever fits. If that does not land, try the prompt — instructions, format, examples. If the problem is what the model can see, fix the context. And only after all of that do you change the model. Changing the model is the last lever, not the first.

Koko: That is Domain Two in one breath. Classify, parameter, prompt, context, model — in that order. Every time. That sequence is what the exam is testing when it hands you a scenario and asks what to try next.

Chris: And the Professional exam is not asking whether you can do it — it is asking whether you can defend why.

Koko: Exactly. So carry the instincts: stable prefix goes before the breakpoint, variable content goes after. Skill descriptions are triggers, not summaries. Conditional guidance belongs in Skills, not in the system prompt. And you do not reach for a bigger model until the simpler levers are exhausted.

Chris: Four modules, one domain, and I think we got there.

Koko: We did. Drills and flashcards for Domain Two are waiting for you at KokoAI Academy — find everything at koko knows dot A I. Go lock this in. You have got this.