Solution Design & Architecture

Koko: Here is the one instinct that runs through everything in Solution Design and Architecture: every architectural choice is a purchase, and the exam wants you to name the price. That is not a metaphor, it is literally the scoring logic. The defensible answer is the one that says what it gave up.

Sam: That framing is exactly what I want out of this. I have shipped these systems. What I need are decision rules sharp enough to defend in a design review, not just recognize on a slide. So let's get precise.

Koko: Good. And this domain is worth being precise about — Solution Design and Architecture is seventeen percent of the Professional exam, the second-largest domain, and it is where every other domain gets its marching orders. Choose the shape wrong here and no amount of prompt tuning or evaluation rigor rescues you later. So let's start with the first purchase.

Sam: The pattern choice.

Koko: Exactly. There are three named patterns, and they sit on a single axis: how much does the model decide at run time. First, an augmented LLM — one call, retrieval or tools attached, and your code owns the control flow entirely. Second, a workflow — a sequence you wrote, with the model filling in named steps inside it. Third, an agent — the model chooses its own steps and its own stopping point. More autonomy buys adaptability. It is paid for in predictability, auditability, and cost.

Sam: And the payment is real. An agent on a bad day can take ten times the tokens of the workflow that would have handled the same task, and you only know it after the fact.

Koko: Right. So let's make this concrete. A claims-triage intake desk sees four document types and nothing else — an FNOL form, a medical summary, a repair estimate, a police report. A deterministic router plus three fixed extraction steps handles it end to end. Known token cost per claim. Same trace every time. Fully auditable.

Sam: That is a workflow, maybe even just an augmented LLM depending on how the extraction is structured.

Koko: Exactly. Now take the same team's exception desk — they see anything at all. Faxes, handwritten addenda, third-party legal letters, documents in four languages. You cannot enumerate that. So that desk gets an agent with a tool set and a turn cap. Same company, same building, two completely different patterns — because the inputs differ.

Sam: And that is the discriminator, isn't it. Not the complexity of the task, not the seniority of the team that built it — the variance in the input space.

Koko: That is the whole thing. Now here is the image I want you to carry. You do not hire an improvising chef to assemble the same four sandwiches all day. You hire one for the kitchen where the order could be anything. The improvising chef is more expensive, harder to audit, and on a slow day they might still improvise anyway — which is not what you wanted.

Sam: And if you have hired them for the sandwich line, you have paid for creativity you are never going to use.

Koko: Paid for it in nondeterminism and a wider audit surface. So the takeaway — the instinct to carry into the room — is this: measure the variance in the input before you choose the pattern, then take the lowest-autonomy pattern that covers it. Where the set of inputs is enumerable, enumerate it in code and let the model do the language work inside each branch. Name the price before you agree to pay it.

Sam: The version of this I keep seeing go wrong on teams is treating the agent pattern as the mature, grown-up version of a workflow. Like the workflow is a stepping stone and eventually you graduate to an agent.

Koko: That is exactly the misconception the exam will test. Autonomy is not a maturity level. It is a price paid for input variance. If the inputs do not vary, you have bought nondeterminism, a wider audit surface, and an unbounded token bill, and received nothing in return. A workflow that handles its domain perfectly is not a less sophisticated system — it is a better-matched one. On the exam, when a scenario enumerates its input types, that enumeration is the answer. It is telling you the variance is bounded, which means the pattern is bounded too.

Sam: So the question to ask every time is: can I list the inputs? If yes, list them in code.

Koko: If yes, list them in code. That is the domain's instinct in its cheapest form.

Koko: Alright, multi-agent systems. Here is the fact, stated plainly: a subagent is a separate instance with its own context window. Its intermediate tool calls, its retrieved chunks, its chain-of-thought — all of that stays inside it. The only thing that comes back to the parent is the final message. So when you decompose a task into subagents, what you produce is a smaller parent context. Not a bigger workforce. A smaller parent context.

Sam: That reframe matters a lot in practice. The reason to reach for multi-agent isn't compute parallelism for its own sake — it's that the parent would otherwise drown in intermediate state it doesn't need to hold.

Koko: Exactly. And Anthropic names three specific conditions worth delegating for. Parallelization of independent subtasks. Specialization, meaning a subagent gets a different system prompt and a different tool set tuned for that one job. And escalation — routing a hard subset of the work to a more capable model for just that step.

Sam: The escalation case is the one I see underused. You don't have to run your whole pipeline on the strongest model. You can use a lighter model for orchestration and send only the hard reasoning steps upmarket.

Koko: Good point. Now let's make this concrete. Due-diligence assistant. You have a hundred and twenty filings to read. You fan out three subagents; each one reads forty. Each subagent's reading piles, its extracted passages, its intermediate notes — none of that comes back. What comes back to the parent is three summaries. The parent holds three documents instead of a hundred and twenty.

Sam: And because those three subtasks are independent of each other, wall-clock time is the slowest one, not the sum. You're not waiting for subagent two to finish before subagent three can start.

Koko: Right. And then there's a fourth subagent in this scenario — read-only by tool restriction — that checks the assembled draft against the extracted quotes. That's specialization. That agent has exactly the tools it needs for verification and nothing else.

Sam: The tool restriction part is doing real work there. The verification subagent can't modify anything, so you've enforced that boundary in the architecture rather than in the prompt.

Koko: Now the analogy, because this is the one that makes it stick. You hire three researchers and ask each of them for a memo on their assigned filings. What lands on your desk are the memos. You do not get their yellow-highlighted printouts, their browser tabs, their handwritten notes. You get conclusions. That's the entire benefit of the arrangement.

Sam: And if you wanted their reading piles, you wouldn't have hired them — you'd just read it yourself. The point of delegation is the summarization boundary.

Koko: So the instinct the exam rewards: delegate when the subtask is independent, when it needs a different tool set or system prompt, or when it needs a stronger model for one specific step. And when you delegate — put everything that subagent needs into the delegation prompt itself. It inherits nothing from the parent. Nothing. The delegation prompt is the contract.

Sam: That last part catches people. Subagents don't automatically see the parent's conversation history. If the delegation prompt doesn't contain it, the subagent doesn't know it.

Koko: Which is a feature, not a bug — until someone treats it like a bug. Here's the trap: teams discover that a subagent is doing mediocre work, and they conclude it needs more context, so they pipe the parent's full conversation history into the subagent. That sounds reasonable. It is the failure.

Sam: Because you've just reintroduced the context pressure that the decomposition existed to relieve. You're back to a large context, but now it's fragmented across the parent and multiple subagents, and you've lost the isolation too.

Koko: The misconception is that sharing context is generosity. It's not — it's a sign that the split was made in the wrong place. If a subagent genuinely needs the parent's full history to do its job, that task should not have been delegated in that way.

Sam: The exam option sitting next to the right answer will usually offer to share the full conversation with the subagent, and it will be framed as giving the subagent what it needs to do good work. That framing is exactly the failure mode.

Koko: When you see that option, flag it. Good work from a subagent comes from a well-constructed delegation prompt — scoped, self-contained, with the tools it needs and nothing it doesn't. That's the contract. Fix the prompt, not the isolation.

Koko: So we have input, processing, output — and almost every architecture diagram drawn in a workshop stops right there. Three boxes, arrows between them, done. The blueprint actually has a fourth element: the feedback loop. And it is the one that determines whether the system gets better after it ships.

Sam: That matches what I see in practice. The first three get debated for weeks. The feedback edge gets a sticky note that says 'logging, TBD' and nobody revisits it.

Koko: Exactly. And there is a real difference between logging and a feedback loop. Logging is capture. A feedback loop is capture plus a named consumer plus a route back into future behavior. All three have to be there.

Sam: So what does that look like when it is actually built? What are you capturing and what is consuming it?

Koko: Here is the concrete version. An invoice-coding assistant writes three things on every transaction: the code it proposed, the code the reviewer accepted or overrode, and the retrieved evidence it used to make the call. Now you have a record of the disagreement — the delta between the model's answer and the human's answer.

Sam: And that delta is the valuable thing. Not the successful cases — those just confirm it worked.

Koko: Right. Next quarter's regression set is built out of real production disagreements rather than cases someone invented at design time. And the override rate — how often the reviewer changes the code — becomes the number the steering group reads. That is a feedback loop. The consumer is the evaluation set. The route back is prompt revision or a rebuild of the retrieval index.

Sam: The discriminator I keep using with teams is: does anything outside this run read the record? If nothing outside the run reads it, you have telemetry, not a feedback loop.

Koko: That is exactly the test. And here is the analogy that makes the mechanism stick. A thermometer tells you the temperature. A thermostat changes it. Most architecture diagrams ship a thermometer. The feedback edge is what makes it a thermostat.

Sam: So the design question you have to answer before you close the diagram is: what is the named consumer of this record, and what does it change?

Koko: Name it. Is it the evaluation set? A routing threshold? A prompt revision? A rebuild of the retrieval index? If you cannot name the consumer, you have not drawn a feedback loop yet. That is the instinct to carry into the exam room.

Sam: The option sitting next to the right answer on this one is usually something about retry logic — the framing being that retry is a form of feedback because the system is responding to an error.

Koko: And that is the trap, so let's name it clearly. A retry is error handling inside one run. It changes nothing about the next run. A feedback loop crosses run boundaries and has a named consumer. A retry that leaves no record is the absence of a feedback loop, not an instance of one.

Sam: The key word is boundaries. The retry lives inside the boundary of a single execution. The feedback loop reaches past it.

Koko: Exactly. And the exam will test whether you can see that difference when the scenario dresses up retry logic in language like 'the system adapts when it fails.' It is not adapting. It is recovering. Those are different things. Draw the feedback edge, name what consumes it, and if the record never leaves the run, go back and draw it again.

Koko: Here is the fact, stated plainly. Residency, retention, and approval requirements are not a review step at the end of design. They delete options. Two concrete edges make this real. First: workspace geography is set at creation and cannot be changed afterward. Second: certain models are designated Covered Models, and Covered Models require thirty-day data retention, which means they are not available under a zero-data-retention arrangement at all. Those are hard eliminations, not soft preferences.

Sam: The way I think about the Covered Model point is that the constraint is baked into the model's terms, not into your deployment config. So there is no flag you flip to make it ZDR-compatible. The option just isn't on the table.

Koko: Exactly right. And the workspace geography one is even more unforgiving because it looks like an infrastructure detail until you realize there is no migration path. Once the workspace exists in a given region, it stays there.

Sam: What does a real scenario look like where both of those come into play at once?

Koko: Take a regulated insurer. Their data handling rules require that no prompts be stored at rest, so they need zero-data-retention across their main workflow. The most capable model in the catalog is a Covered Model. That model requires thirty-day retention. Those two facts together mean the most capable model is simply unavailable to them for that workflow. Full stop.

Sam: So the design question shifts immediately. You are not scoring models against each other on accuracy anymore.

Koko: Right. The design that wins is a mid-tier model running under ZDR for the whole main flow, plus a second workspace with thirty-day retention enabled for a narrow, consented escalation path — cases where the user has explicitly agreed to retention and the use case justifies the more capable model. Two workspaces, different retention policies, chosen entirely by a retention rule. No benchmark was consulted to make that call.

Sam: The discriminator I keep coming back to when I explain this to teams is: the filter reduced the option set before scoring started. The benchmark is answering a different question than the governance constraint is.

Koko: That is the instinct exactly. Here is the analogy I use to make it stick. You choose the building site before you hire the architect. The site determines what can be built. Some sites cannot be re-zoned after the fact. You do not design the building and then discover the zoning.

Sam: And the workspace geography case is precisely that. The region is the site. Once you have poured the foundation, you are not moving it.

Koko: So here is the takeaway. In discovery, you collect residency, retention, approval, and audit constraints and you apply them as a filter over the full option set before you score anything on accuracy, latency, or cost. And crucially: you write down which options the filter removed. That list is the answer to the question your review board is going to ask — why didn't you use the top-ranked model.

Sam: The documentation piece matters more than people expect. I have been in reviews where the architecture looked conservative and the room wanted to know why. Having a written record that option X was eliminated by constraint Y, not by preference, is what closes that conversation.

Koko: Now the trap. The misconception here is that you pick the best model first and then get it approved. The assumption underneath that is that approval is a process you go through, and if you work the process hard enough, you get to keep your first choice. But some constraints cannot be satisfied by configuration after the fact. Learning them late does not mean adjusting. It means rebuilding.

Sam: The version of this I keep seeing on teams is someone who has scoped the whole solution around a specific model, has benchmarks to defend it, and then hits the retention requirement in the legal review. At that point you are not tweaking. You are re-architecting, and you have lost the time you spent on the benchmarks.

Koko: The immutable workspace geo is the clearest illustration. There is no setting that fixes it once the workspace exists. So the sequence that looks like it saves time at the front — pick the best thing, then figure out compliance — is the sequence that costs the most time at the back. Constraints first, options second, scoring third.

Koko: Last one. And this one is about how you prove the thing worked. Here is the fact, stated plainly: efficiency, transformation, productivity, cost, and performance SLAs are five different claims with five different measurements and, usually, five different owners. A proposal that claims all five has committed to none of them — because afterward, nobody can say whether it worked.

Sam: I have sat in that review meeting. The system shipped, something improved, and three people are arguing about which number counts. Because nobody named the primary one before the build.

Koko: Exactly. So here is the scenario. An accounts-payable automation program names cycle time as its primary pillar. Expressed as a performance SLA: ninety-fifth percentile under four hours. And critically — the baseline for that number was captured for six weeks before the build started. Cost per processed document is the secondary pillar. And exception rate is named as the counter-metric that must not degrade, with a threshold that triggers a pause.

Sam: That last piece is doing a lot of work. The counter-metric is what keeps a speed win from being hollow. If you halve cycle time and your exception rate triples, you have not won anything — you have just moved the problem downstream to a human queue.

Koko: Right. And the analogy I keep coming back to: a diet that promises weight loss, better sleep, more energy, and improved mood has promised nothing. Because no single result can falsify it. Any outcome can be called a partial success. That is exactly what a five-pillar proposal does.

Sam: One number that can be wrong. That is what makes a commitment real.

Koko: So the takeaway is: name one primary pillar, measure its baseline before you build, name the counter-metric that would make a win hollow — and then attach an owner in the business, not on the delivery team, to each number.

Sam: That last part trips people up in practice. The delivery team cannot own the business metric. They can own the system behavior, but the number belongs to whoever runs that process.

Koko: Now the trap. And this one is worth slowing down for. The misconception is that token spend is the cost pillar. It is not. Token spend is one line in the total cost of the workflow, which also carries human review time, exception handling, the evaluation work the system now needs permanently, and the incident cost of the failure modes you accepted. A program that halves token cost and doubles review time got more expensive. The exam will absolutely offer token spend as the cost answer.

Sam: The version of this I keep seeing on teams is someone optimizing hard on inference cost while the review queue quietly grows. The total cost of ownership went up. They just were not measuring it.

Koko: Total cost of the workflow. Not one line in it. Alright — Sam, you want to pull the thread on everything we covered today?

Sam: Yeah. So the way I think about the order: first you measure input variance, because that tells you which pattern is even appropriate. Then you delegate for isolation and you write the contract before you wire anything up. Then you draw the feedback edge and name who consumes it — no feedback loop without a named consumer. Then you filter by constraint before you score anything, because scoring options you cannot meet wastes the room's time. And finally you name one primary pillar with a baseline and a counter-metric. That is the sequence a real design uses.

Koko: Five rules, five decisions, in the order the problem actually asks for them. That is the through-line of this domain.

Sam: And the exam habit that ties it together — when two options both look defensible, the Professional exam is testing whether you know what each one costs. The answer that names the trade-off is nearly always the one they want.

Koko: That is the instinct. Not which answer sounds right. Which answer knows what it is giving up. Carry that into the room and you will recognize the question even when the surface details are different.

Sam: If you can name the cost of the choice you did not make, you are ready for this exam.

Koko: The drills and flashcards for Domain One are at KokoAI Academy — koko knows dot A I. Everything from input variance to business value, sequenced the way the exam uses it. Go build something, then come back and prove you can defend it. That is what the Professional is for.