Integration

Koko: Integration is nineteen percent of the Professional exam. That is the largest single domain on the whole test, ahead of Solution Design. And the thing that unifies every objective in it — tool config, authorization, protocol choice, retrieval, observability — is one instinct: a connection is rent, not a purchase.

Sam: That framing is exactly what I want to get sharp. I have wired plenty of these systems. What I want is the rule that decides which connection, not the mechanics of writing one. Because the writing part is fine — it is the choosing that costs you on the exam and in production.

Koko: Right. And the most literal place that instinct bites you is the tools you attach. So let's start there with the fact. Tool definitions live in the context window. Every single one you register gets sent on every request — used or not. The model pays that cost before a single user word arrives.

Sam: And it is not just a token cost. That matters for selection quality too.

Koko: Exactly, and we will get there. First the number: Anthropic's guidance calls out roughly twenty tools as the threshold. Under about twenty, caching the definitions and keeping them loaded is fine. Past that, the guidance is to defer — send one tool-search tool and let the model discover the rest on demand.

Sam: So discovery costs one extra turn of latency, but you trade that for a much smaller baseline context.

Koko: One extra turn, yes. Now the scenario. An operations agent. Over time, different teams added integrations — ticketing, policy lookup, scheduling, monitoring, a handful more. By the time anyone counted, the agent had fifty-one registered tools. Baseline context was substantial before the user typed a single character. And the team noticed tool selection had degraded — the model was routing to the wrong tool more often.

Sam: I have seen exactly that. The symptom looks like a reasoning problem but it is actually a crowding problem. Fifty near-neighbor definitions and the model is essentially doing a search across all of them with every call.

Koko: Perfectly put. They switched to on-demand discovery — standing definition count dropped to one, the tool-search tool — and selection got better. Not just cheaper. Better.

Sam: Because the model is no longer choosing among fifty options. It retrieves a short, relevant list and chooses from that.

Koko: Here is the image I use for this. The tradesman's van. If you load every tool you have ever owned into the van, you pay fuel for all of it on every job. And when you get on site, you spend longer hunting the right wrench than actually turning it. The problem is not that the tools are bad — it is that carrying all of them all the time is the wrong architecture.

Sam: And the exam equivalent of the van is the context window. Heavy before the job starts.

Koko: So the takeaway to carry into the room: count the tools and measure the baseline context before you touch anything else. If you are under roughly twenty and the set is stable, cache the definitions and leave them loaded. Past that, defer discovery. And the judgment is not how many tools the platform supports — it is how many are plausibly relevant in a single turn.

Sam: That last part is the discriminator. Plausibly relevant in a single turn. An agent could technically reach any of fifty tools, but in any given turn it probably needs three.

Koko: Now the trap. The misconception I see around this is that bloat is only a cost problem, so caching is the fix. Caching changes the price of those tokens — it does not change their presence. And presence is what degrades selection. If your symptom is the wrong tool being chosen, cheaper definitions do not help. Fewer visible definitions do.

Sam: The option sitting next to this on the exam will say: cache the tool definitions to reduce overhead. Which is true and sometimes correct — but it is not the answer when the symptom is selection quality degrading.

Koko: And the counterweight — because progressive discovery is a real trade-off — is that discovery is an extra step that can itself fail. The tool-search call could return nothing, or return the wrong set. So you are not just buying cheaper context, you are adding a failure mode.

Sam: Which means the answer is not always defer. It is: measure the baseline, count what is plausibly relevant per turn, then choose. Under twenty and stable, load and cache. Past that, defer and accept the extra step — and design for the case where the search step fails.

Koko: That is the whole module in one sentence. A connection is rent, tool definitions are the first place you pay it, and the Professional question is always what it costs when nothing is using it.

Koko: So before we go further — three ways to wire something up, and the exam will make you choose between them. First: MCP, which exposes tools, resources, and prompts to any client that speaks the protocol. One server, many consumers, and when the tool surface changes you don't redeploy every caller — they discover it. Second: a direct API or CLI call, code you own, no protocol layer, cheaper and tighter, bespoke per consumer. Third: agent-to-agent delegation, where you hand over a goal and get back a summary — not a payload, a conclusion.

Sam: That third one is the one teams underuse. The caller doesn't want forty pages of competitive data — it wants the summary. So it delegates to a specialized agent and gets back exactly that.

Koko: Right. And that distinction — payload versus conclusion — is load-bearing on the exam. Now here's the scenario so this lands concretely. A company has a ticketing system. Five different assistants need it, and the tool surface changes monthly — new fields, new actions. They expose it over MCP. Same company, one internal pricing service, one consumer, contract hasn't changed in a year — they call it through a plain typed client. And when they need a competitive landscape summary, they delegate to a specialized agent, because the work behind that summary is not theirs to decompose.

Sam: The pricing service case is the one I see people over-engineer. The temptation is to wrap everything in MCP because it feels like good architecture hygiene. But if you have one consumer and a stable contract, you just added a protocol layer, a server to operate, and an authorization boundary you didn't need.

Koko: Which is exactly where we're going in a minute. But first — the analogy, because this one is genuinely useful. Think about plumbing. A standard threaded fitting is worth machining when a dozen appliances are going to connect over the years. One fixed pipe between two points in your basement? You weld it. A bathroom remodel? You hand that over as an outcome — you don't give the contractor a list of pipe cuts.

Sam: Weld the basement pipe. I'll remember that.

Koko: So the takeaway — the instinct to carry in — is: pick by consumer count and churn. Many consumers, or a surface that moves, means a protocol. One consumer, fixed contract, means a direct call. A goal you cannot decompose from outside means delegation.

Sam: What I'd add from practice: the discriminator between direct and MCP isn't about how modern the stack is. It's purely about distribution. Does this capability need to reach several agents, or change without touching every caller? If yes, the protocol earns its keep. If no, it's overhead.

Koko: That's the exact line the exam tests on. Which brings us to the trap — and it's a confident-sounding wrong answer. The misconception is that MCP is the modern replacement for calling an API. It isn't. It's an interoperability and discovery layer. It earns its keep when one capability serves several agents, or when the surface changes without a redeploy.

Sam: The version of this I keep seeing on teams — and the exam will surface it as an option — is the idea that adopting MCP signals better architecture. So people reach for it on single-consumer, stable internal calls. What they've actually done is add operational surface for no distribution benefit.

Koko: Exactly. A standard protocol changes distribution, not comprehension. If there's nothing to distribute to, you haven't gained anything — you've just added weight. One consumer, stable contract: direct call. Full stop.

Koko: So MCP's authorization model puts the server in the role of an OAuth 2.1 resource server. Two requirements carry almost all of the security weight. First, the client sends a resource parameter that names the canonical URI of the server it actually intends to call. Second, the server validates that the incoming token was issued specifically for it as the intended audience. The spec is explicit on this: no other token may be accepted, and no token may be transited to a different server.

Sam: Right, so audience validation is the server's job, not an optional check. The token arrives, the server looks at the audience claim, and if that claim doesn't match its own canonical URI, the token is rejected outright.

Koko: Exactly. Let's make that concrete. Say an assistant holds two tokens — one minted for an internal analytics server, one for a third-party integration server. Each is audience-bound to exactly one canonical URI. If a token meant for the analytics server gets routed to the third-party server, the third-party server rejects it. Doesn't honor it, doesn't pass it along. And if the scope is simply insufficient for the requested action, you get a 403 back naming what's missing.

Sam: Which also means misrouting a token doesn't silently succeed — you get a hard rejection rather than a quiet authorization hole. That matters a lot for auditability.

Koko: It does. And here's the analogy that makes the mechanism stick: think of a hotel key card. It opens one room in one hotel. The lock checks the card's binding to that specific door — it doesn't look at your face, doesn't care that you're a guest in good standing. Wrong pocket, wrong lock, it refuses. Audience binding works the same way. The token's binding to a canonical URI is the check, not who holds it.

Sam: And the design implication follows directly from that: you mint tokens per server, not per user. One server, one audience claim, one minting event.

Koko: That's the instinct to carry into the exam room. Design the token boundary per server. Ask three questions: which server is each token minted for, does that server validate the audience claim, and what could an agent reach with a token it should never have received. Those three questions find the holes.

Sam: The third question is the one teams skip. They think about what the token allows, not about what becomes reachable if the token ends up somewhere unexpected.

Koko: Which is exactly where the trap lives. The misconception here is that authenticating the user is the security design. It isn't. The real risk is the confused deputy — a correctly authenticated agent that gets induced to use its authority on the wrong object or the wrong server. The agent has valid credentials, the user logged in successfully, and none of that stops the exploit. Audience binding, scope minimization, and requiring approval on sharp actions address the confused deputy. A login page does not.

Sam: There's a second identity problem that shows up alongside this in practice. An application credential tells you which application is calling — it says nothing about which end user is behind the request. The version of this I keep seeing on teams is people assuming the credential carries the user context, so they skip the user permission check. But user permission has to be enforced in your own code: once before you assemble context, and again before you execute anything.

Koko: Two gates, not one. The exam will put an answer next to yours that says 'the user authenticated successfully, so the system is secure.' That answer is describing a necessary condition, not a sufficient one. Audience binding and scope minimization are what close the deputy problem. Authentication is just the starting line.

Koko: One thing worth saying flat out before we go further: Anthropic publishes no official guide to retrieval-augmented generation, chunking, or indexing. Search the documentation for retrieval or RAG and you get nothing — worked examples exist only in Cookbook recipes. So this objective is testing reasoning you construct, not a page you memorized.

Sam: That actually changes how you study it. You can't pattern-match to a spec, so the exam has to test whether you can reason about the tradeoffs. Which means the discriminators are going to be about judgment, not recall.

Koko: Exactly right. So let's build the reasoning from scratch. The core fact: a chunk should be the unit that a question is actually about. Not the unit that's convenient to split on, not the unit your parser defaults to — the unit that answers the question.

Sam: The scenario I keep coming back to with teams is a policy assistant. If users ask about clauses — what does clause four-point-two require, what's the effective date of this rule — then a clause is your chunk, not a paragraph, not a page.

Koko: And what has to travel with that clause chunk?

Sam: The section heading and the effective date, on every single chunk. Because the chunk is what gets retrieved, and if the heading didn't make it, the model has no way to cite which policy it came from. The answer becomes unattributable.

Koko: Here's the analogy that makes this stick. Imagine photocopying pages out of a bound manual. You cut the page just below the section heading — you get a perfectly legible copy and a completely unusable one. Nobody reading it can tell which policy it belongs to. That's a chunk that lost its metadata.

Sam: And the other half of the scenario is the lookup strategy. In that same policy assistant, if the user types a policy number — say P-dash-four-seventeen — that's an exact identifier. There's no semantic neighborhood around it. Vector search will get you policy-adjacent content; it won't reliably get you that record.

Koko: So the instinct to carry out of here: match the strategy to what the data and the query actually are. Exact identifiers go to a lexical lookup. Conceptual or exploratory questions go to embeddings. Aggregates — how many claims exceed a threshold — belong to a structured query over a database, not a prose retrieval at all. And whatever you retrieve, you ground the answer using the citation mechanism the API does document.

Sam: The question I'd add, because the exam does test this direction too: ask first whether retrieval is warranted at all. If the corpus is small and stable — say, a dozen internal guidelines that almost never change — it might just belong in the prompt. Reading it in full is cheaper than building an index for it.

Koko: That's a real discriminator and worth holding. Now — the trap. The misconception here is that a bigger context window retires retrieval entirely. The reasoning goes: if I can fit more text in the prompt, why do I need an index?

Sam: And the option sitting next to it on the exam will say something like: given the extended context length, replace the retrieval pipeline with full-corpus prompting. Which sounds like a simplification but is actually a regression on three things at once.

Koko: Cost per call, attention dilution, and audit trail — you lose all three. Retrieval isn't only a capacity workaround. It's a precision and provenance mechanism. When a sentence appears in the answer, you need to know which passage it came from. Pasting in the whole corpus destroys that trace, and no context window brings it back.

Koko: So here is the fact the exam is testing. Every lever you can pull — effort setting, service tier, batch mode, streaming, caching — moves latency and cost and quality in different directions. Effort trades intelligence for speed and cost. Service tier changes availability under load, not raw speed. Batch trades immediacy for a discount. Streaming changes perceived latency, not total time. Caching cuts time to first token. The exam does not ask which lever exists. It asks you to justify why you pulled it — and justification means naming the budget it was chosen against.

Sam: Right, and a budget without a percentile is not a budget. 'Fast' is not a target.

Koko: Exactly. Here is the worked scenario. A trading-desk assistant has a stated target: ninety-fifth-percentile response under two seconds on the interactive path. To hit it, the architect takes a lower-effort setting and adds caching. Both choices are on the record, along with what they cost on accuracy. That same firm runs overnight reconciliation — no latency target, just 'done by morning' — so that job goes to batch and earns the discount. Two workloads, one stove, two completely different clocks.

Sam: That two-clock image is doing real work. The way I phrase it internally: interactive paths and async jobs should never share a latency target, because optimizing for one will actively harm the other.

Koko: And that is the analogy. Think of a kitchen running a lunch counter and a catering order off one stove. If you average the counter tickets with the catering prep, you get a number that tells the person standing at the counter absolutely nothing about when their sandwich arrives.

Sam: Which is also why the catering order does not need a two-second clock. It needs a finish-by-morning clock.

Koko: Precisely. So the takeaway: write the budget as a percentile with a window before you touch any configuration lever. Split interactive paths from async ones. And when you pull a lever, record which one and what it cost — lower effort costs something on accuracy, batch costs immediacy. That record is the justification the exam is asking you to produce.

Sam: What is the discriminating question on this? Because the options will look close.

Koko: The trap is that latency is one number for the system. The misconception is: compute an average across all paths, set a single target, tune against it. But averaging hides the tail — and a p-fifty target means half your users are already above it by definition. A quarter of your users can have a genuinely bad time and you are still inside the number. The exam will offer you a single-system latency figure and ask if it is sufficient. It is not. Per user-visible path, stated as a percentile, is the answer.

Sam: Same kitchen, different problem: the smoke alarm goes off for a fire. It stays completely silent for a sandwich that is merely wrong.

Koko: That is the fact about observability. Uptime, error rate, p-ninety-five — none of those can see a system that is up, fast, and answering badly. The system looks healthy. The user got a confident wrong answer and has no idea.

Sam: The version of this I keep seeing on teams is: we have a dashboard, it is green, we are done. But the dashboard is only watching the smoke alarm. What you actually need is a record of what the system decided.

Koko: Say more about what that record contains.

Sam: The assembled prompt, the parameters you sent, the stop reason, the retrieval results, the tool outputs. Everything the model had in front of it when it answered. Not more dashboards — a decision log.

Koko: And that is the takeaway. Log what the system decided, not just whether it responded. The exam will ask what monitoring catches a wrong-but-fast answer. Error rate does not catch it. A decision log does. The trap is conflating operational health — latency green, errors low — with answer quality. They are orthogonal. A system can pass every infrastructure check and fail every user in the same second.

Sam: Alright, six modules in. Let me see if I can say the through-line back in one breath. You count your tools and measure your baseline first. You pick your coupling style by how many consumers you have and how fast the schema changes. You bind the token to the server and you enforce user permission in your own code — twice. You chunk to the unit that actually answers the question. You write a latency budget per path, not one budget for the whole system. And you log what the system decided, not just what the user said.

Koko: That is exactly it. And notice the order — that is the order a real integration uses them. You do not reach for token scope until you have picked your coupling, and you do not pick your coupling until you have counted your tools and measured your baseline. The sequence is load-bearing.

Sam: The one that took me longest to trust in practice is the logging rule. Teams want to log the user's input because that feels like the record. But the decision the model made — which tool it called, what it retrieved, what it chose not to surface — that is the thing you actually need when something goes wrong.

Koko: Right. The user's words tell you what they asked. The system's decision tells you what it did with that. Those two things are not the same, and you need both.

Sam: On the exam specifically — the pattern I keep seeing is that the question describes something already connected. The system is already working. And then it asks what it costs, or which of two options is better. Both options look reasonable on the surface.

Koko: And that is the exam habit to name before you walk in. When two answers both look workable, the exam is asking whether you know the standing price of the thing that is already connected. Context spent every turn. A trust boundary that just got a little wider. A subsystem you now have to operate. The answer that names the cost is almost always the one they want.

Sam: And the recurring right answer tends to be narrower rather than cleverer. Fewer tools visible to the model. A tighter token audience. A chunk that still carries its heading. A budget written per path.

Koko: Narrower rather than cleverer — I want that on a card above the monitor. Because the temptation when you are building is to add. More tools, broader scope, smarter retrieval. And each addition is individually defensible. But the exam is checking whether you can see the cumulative price.

Sam: A connection is rent.

Koko: A connection is rent. You pay it every turn, whether or not you needed everything you brought along. The six rules are really just six ways of asking: what is this connection going to cost me, and is it worth it?

Sam: Good way to end the domain.

Koko: The drills and flashcards for Domain Three are waiting for you at KokoAI Academy — find them at koko knows dot A I. Run them until the six rules come back in order without prompting. That is the test. You have got this — go show them you can defend the choice.