Stakeholder Communication, Lifecycle & Operational Enablement

Koko: Every answer in Domains Six and Seven is an artifact created before the moment it is needed, in a form somebody else can check. That is the one instinct that covers all fourteen ideas in these two domains, and if you carry nothing else out of this lesson, carry that. Now — why are we here? Twenty-one percent of the Professional exam lives in these two domains. That is the largest single slice in this series. And it is the part with no Foundations equivalent. Foundations asked whether you can build. This asks whether the organization can trust you afterward.

Chris: And it is what most prep material skips, which makes it the highest-yield half-hour on the exam. I want to say that plainly up front because I have watched a technically sound program lose its executive sponsor, and when I went looking for what would have prevented it, the answer was always a document that did not exist yet.

Koko: That is exactly the right frame. Domain Six is what an outsider is told. Domain Seven is what an operator can see. One question, two audiences. And the first artifact in the sequence — the one that has to exist before anything else — is structured discovery.

Chris: Walk me through what that actually is, because the word discovery gets used loosely.

Koko: Here is the fact. Structured discovery is a written instrument, not a conversation you hope goes well. It is a fixed question set, and each question is attached to the specific decision it unblocks. You run it identically with everyone. The output is a document, not a summary of a meeting.

Chris: The identical-with-everyone part is doing real work there. If the question set changes by stakeholder, you cannot compare answers, and you end up with a patchwork of constraints that contradict each other by the time you get to design.

Koko: Exactly right. So here is the scenario. Twelve questions, grouped by the decision each one settles. The first cluster covers where data may be processed — residency, jurisdiction, the hard boundaries. The second covers what may be stored and for how long. The third covers what runs unattended and who has approved that. The fourth asks who approves by name — not a role, a person. And the highest-yield question in the whole set is the last one: what was tried before and why it stopped.

Chris: That last one is underrated. In my experience it surfaces more real constraints than anything else on the list, because the previous attempt usually failed on something the organization never wrote down.

Koko: It is the question that turns institutional memory into architecture. Now, the analogy. Think of a surveyor's report before you buy a house. A surveyor never asks what you would like the kitchen to look like. The report tells you which wall is load-bearing. That is what structured discovery does. It does not ask which features the stakeholder would love. It tells you which walls you cannot move.

Chris: And you need that before you draw the floor plan, not after.

Koko: That is the takeaway. Ask what the system may not do before you ask what it should do. Prohibitions delete architectures. Wishes rarely do. A wish for real-time processing is a nice constraint to have. A prohibition on cross-border data transfer eliminates an entire class of deployment options — and the sooner you know that, the cheaper your choices are.

Chris: The version of this I keep seeing on teams is the feature questionnaire — rate these capabilities one to five, that kind of thing. It looks thorough. It produces a ranked list. And it contains exactly zero constraints.

Koko: That is the misconception the exam is testing. The feature questionnaire reads as rigorous. It has a lot of rows. But it asks what people want, not what they cannot have. So the residency rule, the approval chain, the previous failed pilot — none of that surfaces. And it reaches you at the security review, long after the architecture is set and the options have stopped being cheap. The trap on the exam will be a discovery process that feels structured because it is documented and consistent, but is asking the wrong category of question entirely.

Chris: Structured form, wrong substance.

Koko: Structured form, wrong substance. Each question in a real discovery instrument has to be traceable to a decision. If you cannot name the decision a question unblocks, the question does not belong in the instrument. That test — does this question unlock a decision — is how you tell the two apart on the exam.

Koko: So there are actually two halves to one job here, and most people only do one of them. The first half is recording the decision — not just the winner, but what was chosen, against which alternatives, under which constraints, at what cost, and what would make you revisit it. The second half is translating that trade-off into the metric the reader already owns. A CFO owns cost and risk. A head of operations owns throughput and exceptions. A compliance officer owns evidence and auditability. If you hand any of them a trade-off in your units, it hasn't been communicated.

Chris: That second half is where I see teams fall down. They produce a technically rigorous comparison and hand it to the business side, and the business side approves something they don't fully understand — which means the moment costs show up, there's no sponsor left.

Koko: Exactly. So let's make both halves concrete. The record is one page, five headings: context, options considered with what each buys and costs, the decision in one sentence with a date and a named decider, consequences, and a revisit trigger — something like a volume threshold that means 'if we hit this, we come back.' That last one is the piece people skip most.

Chris: The named decider matters more than it sounds. A decision with no owner can't be formally reopened, so it just gets relitigated informally every few months when someone new joins the team.

Koko: Right, and that's where the analogy lands. The record is the written judgment behind a verdict. The verdict tells you who won. The judgment tells you why — so the next case argues against the reasoning instead of relitigating the facts from scratch. If you only record the winner, six months later someone proposes the rejected alternative and nobody can say why it lost.

Chris: And the trade-off table is the second artifact — three columns, one row per option. The option, what it buys in the reader's own metric, and what it costs in that same metric. Dollars per month, cases per hour, whatever the reader's scorecard actually tracks.

Koko: That's the one. And the analogy I like for it is a doctor explaining the same prognosis three times — to the patient, the family, and the insurer. Same clinical fact, three different conversations in three different units. A surgeon who lists only the benefits has obtained agreement, not consent.

Chris: That distinction — agreement versus consent — is a useful test. If the stakeholder can't articulate what they're giving up, you haven't finished the communication.

Koko: Which is why the takeaway here is: write the record while the rejected options are still fresh, name the decider, then write the cost column first. If you can't fill the cost column, the analysis isn't done yet — you're not ready to present.

Chris: The version of this I keep seeing in the field is a presentation where all three options appear with their upsides, one is highlighted as recommended, and the costs are either absent or buried in an appendix. It approves fast because nobody's objecting to anything. Then the first real cost lands and the sponsor feels surprised — even though the cost was always there.

Koko: That's the benefits slide, and it's one of two traps. Three options, upside only, one highlighted. It turns every later cost into a surprise, and that's how a sound design loses its sponsor. The second trap is subtler: recording only the chosen option and its benefits in the decision document. Six months on, the rejected alternative gets proposed again and nobody remembers why it lost. That's the record without the judgment — you've got the verdict but no reasoning to argue against.

Chris: So both traps are really the same failure at different stages — omitting the cost side. One at presentation time, one at documentation time.

Koko: That's a tight way to put it. Cost column first, always — whether you're writing the record or building the table.

Koko: Two documents need to exist on paper before anything goes live. A service-level statement and an accuracy statement. They are actually one question asked in different units: what has this organization agreed to accept? And the answer only counts if it is specific enough to be falsifiable.

Chris: The thing I keep seeing teams skip is the measurement window. They write down a target number but not how or over what period it gets evaluated, which means the number has no teeth.

Koko: Exactly. A service-level statement is unenforceable unless every row carries four things: the metric, the target value, the measurement method and window, and what happens on a breach. Remove any one of those and you have a wish, not a contract.

Chris: Same structure applies to the accuracy statement. Ninety-two percent sounds good in isolation, but the question is what was agreed about the other eight.

Koko: Right, so here is the worked example. One row in the service-level statement reads: accuracy, ninety-two percent, measured on the frozen regression set, evaluated per release, breach blocks the release. That row has all four pieces. Beside it sits a one-page accuracy statement, signed off before build. And the accuracy statement does three things the number alone cannot do.

Chris: It separates the two error types, because a false positive and a false negative do not cost the same thing to this business.

Koko: That is the first thing. The second is that it translates the residual into something a human can picture — forty cases a month landing on a named team. Not eight percent error rate, forty cases, a named team. And the third is a pause threshold: the number at which the organization has agreed in advance to stop and reassess.

Chris: That last one is the one that actually gets the conversation done before build, because everyone is still optimistic and willing to commit.

Koko: Which is the analogy I want to use here. Think about the difference between a shop assistant saying we will get it to you soon versus a docket that has a date, a measurement, and what happens if it slips. Those are different kinds of object. One is a gesture of goodwill. The other is a receipt. You want your documents to be receipts.

Chris: The instinct I carry from that is: derive every target from something actually measured, and get the residual accepted in writing while the room is still in the optimistic phase.

Koko: That is exactly the takeaway. Now the traps, because the exam puts two very specific wrong versions next to the right answer. The first is a line that reads ninety-nine point nine percent uptime, sub-second response — and nothing else. No window, no consequence, no note about which dependencies the team actually controls. That line will appear in a distractor and it will look thorough because it has impressive numbers.

Chris: The option sitting next to it on the exam will often frame that as the complete service-level statement, which it is not.

Koko: The second trap is the live demo as the commitment. The room watches six curated inputs run cleanly, calibrates on that, and when the first real error appears after launch it reads as a defect rather than as the residual everyone should have agreed to in writing. The misconception is that a successful demo constitutes an accuracy agreement. It does not. The demo is a gesture of goodwill. The signed accuracy statement is the receipt.

Koko: Let's talk about the handoff, and specifically about what makes one real versus what makes one feel real. The fact here has two halves and the second one is what makes the first load-bearing. A handoff is complete when somebody who was not in the design can operate the system at three in the morning. That's the test. And an architecture diagram does not pass that test.

Chris: That framing lands for me because the other thing teams get wrong alongside handoff is how they describe lifecycle phases. They report phase as a status — we're in build, we're in design — when a phase is actually defined by the artifact that lets you exit it.

Koko: Exactly right. The artifact is the gate. So let's run the gate table. Discovery closes on a constraint set and a measured baseline. Build closes on an evaluation set that has passed its thresholds. Handoff closes on a runbook — and not just written, rehearsed by the receiving team. Monitoring is the one row in the table that closes on nothing, so instead that row names a review cadence.

Chris: The runbook structure matters here too. From what I've seen in practice, the document earns its keep through seven sections: what the system does and does not do; its dependencies and how each one fails; failure modes seen so far with symptoms and fixes; how to pause it and who is authorized to; where logs and cost figures live; the evaluation set and its thresholds; and a named owner for configuration and tool permissions. That last one is the section that usually goes missing.

Koko: Named owner for permissions — yes. Because at three in the morning you need a person, not a team alias. Now for the analogy, because this one has to stick. Handing over an architecture diagram is handing over the drawings for a boiler, not the boiler manual. The manual tells you which fuse trips when the kettle and the heater run at the same time, and it has the number to ring when something goes wrong at three in the morning. That is what the runbook is.

Chris: And for the phase gate piece, the analogy I keep using is a driving test versus a driving course. You leave the learner stage because an examiner signed something, not because you had enough lessons.

Koko: That's a good one. The takeaway to carry into the exam: write the runbook from the on-call person's questions, not from the design decisions. Record what normal looks like so you can recognize what isn't. And report progress against artifacts, not percentages. The most dangerous phase in the whole lifecycle is the transition into operation, because the people who will run the system never chose what running it takes.

Chris: On that point about dangerous phases — the version of this I keep seeing on teams is that transition risk gets treated as a documentation problem to solve after launch, rather than as the reason the runbook has to exist before handoff closes.

Koko: Which sets up both traps neatly. Trap one: handing over the diagram and the repository. Neither document says what to do when the system behaves oddly, so the first incident escalates straight back to the people who built it. The receiving team has no floor to stand on.

Chris: And the exam option sitting next to the right answer on the phase reporting question will say percent complete. That number is unfalsifiable — you cannot look at it and know which phase was skipped. It's also how a program sits at ninety percent for two months without anyone being able to name what is actually missing.

Koko: Percent complete hides the skipped phase. Artifact signed by a named reviewer shows you exactly where the gap is. That's the discriminator. If a question describes a handoff and asks what's missing, look for the runbook. If it describes progress reporting and asks what's unreliable, percent complete is the answer.

Koko: Deprecation is the one lifecycle event that comes with published mechanics. Models move through four states: active, legacy, deprecated, and retired. The critical distinction is between deprecated and retired — a deprecated model keeps accepting requests, it still works, but it carries a retirement date. After that date, requests fail. Anthropic commits to at least sixty days' notice before retiring a publicly released model.

Chris: And it's not just the model version — parameters retire on the same schedule. Temperature, top-p, top-k were removed in API version twenty twenty-six, ten, twenty-one. So if your code is passing those parameters and the API version flips, you're not getting a graceful degradation, you're getting a hard failure.

Koko: Exactly right. The mechanism is the same — a date, after which it stops working. So here's the scenario I want to walk through, because this is what the exam tests on. The migration notice should go out the week the deprecation is announced, not the week before the retirement date.

Chris: Which sounds obvious but teams routinely flip those. I've seen the notice sent with maybe ten days left on the clock.

Koko: Right, and the notice needs four things in it: the retirement date, the affected surfaces with their owners named, the replacement model's results on the existing regression set, and the cutover and rollback plan. If any of those four are missing, a stakeholder somewhere is going to get a surprise.

Chris: The regression set piece is the one I'd emphasize. You can't just say the replacement is equivalent — you have to run your frozen evaluation set on the new model and show the numbers. That's the validation a business owner can actually sign off on.

Koko: And to do that you need to know which surfaces are using which model. That means exporting usage by API key and by model version as a standing practice, not as a scramble when the notice lands. The analogy I keep coming back to is a lease with a notice period. The letter arrives months ahead, the building works perfectly until the date, and the tenant who filed the letter unread is the one moving out over a weekend.

Chris: The lease analogy is good because it also captures the asymmetry — the landlord did everything right, the notice was sent, the period was honored. The problem is entirely on the receiving end.

Koko: So the instinct to carry in is this: make deprecation notices a named responsibility — one person owns it — export usage by key and model on a schedule, and re-run the frozen evaluation set on the replacement before you communicate anything to stakeholders.

Chris: The version of this I keep seeing on teams is treating the deprecation as an engineering ticket. Engineer swaps the model, validates it internally, closes the ticket, and the business owner finds out when someone asks why the outputs changed.

Koko: That's the trap exactly. The misconception is that a model swap is a technical event and communication happens after the swap is done. But the business owner was never told the system was being changed — they were just told, at some earlier point, that it had been validated. So from their perspective the validated system changed underneath them without notice. On the exam, if you see an answer that sequences the stakeholder communication after the cutover, that's the wrong answer.

Koko: Here is the fact to anchor everything in this section. Configuring tooling for a team is an access-control exercise before it is a productivity one. Settings live at different scopes — organization, workspace, project, individual — and the scope is what decides who can override a value and who can audit it. Skills belong to the same story: a Skill packages a procedure as a directory the model loads on demand, and custom Skills do not sync between surfaces.

Chris: The sync point matters more than people expect. I have seen teams build a Skill on one surface, assume it is available everywhere, and then wonder why behavior diverges in production.

Koko: Exactly. And scope governs Skills the same way it governs any other setting — location determines who controls the thing. So here is the worked scenario. A platform team commits a shared code-review procedure as a project-scoped Skill in the repository. Every engineer on the project loads the same version. A change to that procedure goes through code review, just like any other code change. Meanwhile, key issuance and spend limits sit at the workspace scope, where only administrators can touch them.

Chris: So the review procedure is project-scoped because it needs to travel with the codebase and be versioned alongside it, and the spend limits are workspace-scoped because you want centralized control that individual engineers cannot override.

Koko: That is exactly the discriminator. The question is not what is convenient to configure — it is who needs authority over this value and what audit trail you require.

Chris: The phrase I use with teams is: if a single engineer can silently change it, it is not a standard, it is a preference.

Koko: That is a clean way to put it. Here is the analogy that makes this stick. Think about a recipe pinned on the kitchen wall versus a chef's personal notebook. The pinned copy is what makes two shifts produce the same dish. If the recipe only lives in one chef's notebook, the second shift is improvising. A Skill loaded from a shared project scope is the pinned recipe. And a Skill that comes from outside your organization — think of it as an ingredient from a supplier. You audit suppliers before you put their product on the menu.

Chris: And on the Skills internals — the description field is doing real work that I do not think is obvious. That is not documentation for humans, that is the trigger metadata. The model uses it to decide when to load the Skill.

Koko: Right, and it is the only part of a Skill that is always in context. So the phrases you put in the description should be the phrases your team actually uses when they need that procedure — not formal names, the real spoken language of your workflow.

Chris: The takeaway I carry from this: for every setting, ask two questions. Who can change it? And does it follow the person or the project?

Koko: Those are the two questions. Now, the traps — both worth naming explicitly. The first misconception is that standardizing means writing a document telling everyone to configure things the same way. That is not standardization, that is a suggestion. Configuration that is not enforced at a scope somebody administers can be ignored silently, and you will never see it in an audit log.

Chris: The version of this I keep seeing on teams is a shared wiki page labeled company defaults, with no enforcement mechanism behind it.

Koko: Wiki page, Confluence doc, Slack message — none of those are access control. The second trap: a Skill is not documentation. The misconception is treating it like a README that explains a procedure to humans. A Skill gives the model instructions and can contain executable code. The model acts on it. If you commit a Skill at project scope without review, you have committed instructions that every engineer's session will execute.

Chris: Which is why the supplier audit framing matters. You would not add a new code dependency without review. Same discipline applies.

Koko: Same discipline, same gate. Scope is access control. Skills are instructions. Both questions — who can change it, does it follow the person or the project — have to be answered before you ship the configuration, not after something diverges.

Koko: Last stop. And it is a good one, because this is the module that ties the whole domain together — you can only defend decisions you can see, and you can only see what you arranged to see before deployment.

Chris: Which is exactly where teams get caught. They build the system, ship it, then go looking for data that was never collected.

Koko: Right. So the fact first. Telemetry is not one thing — it is three independent signals with separate switches. Metrics give you token and cost counters. Log events give you prompts and tool results. And traces, currently in beta, give you a span per interaction, per model request, and per tool call — and when you have a subagent, its spans nest under the parent's.

Chris: That nesting matters operationally. A flat log tells you something was slow. A trace tells you which span was slow and whose child it was.

Koko: Exactly. And two more things sit on top of all three signals. First, identity attributes describe the credential — so every event names your service account unless you explicitly inject the user identity. Second, cost is reportable by key, by workspace, and by model — but only by dimensions that were present at request time. You cannot backfill a dimension you forgot to set.

Chris: We learned that one the hard way on a multi-product deployment. Four products sharing one key — at the end of the quarter there was a single cost number and no way to split it.

Koko: That is the scenario baked into the notes. A team reports the agent is slow. The trace shows the model requests are fast. One tool span is dominated by its permission-wait child span. The latency is a human approval queue, not the model.

Chris: And without the trace you would spend a week optimizing prompts that were never the problem.

Koko: The analogy I want you to carry is a flight data recorder. It records the aircraft's parameters by default. The voice channel only records if that channel was fitted before the flight. It names the aircraft, not the pilot. And it cannot be installed after the accident.

Chris: So the service account is the aircraft registration, and per-user attribution is the voice channel — you fit it before departure or it is gone.

Koko: Perfect restatement. The takeaway is this: turn telemetry on at the first deployment, verify data is actually arriving — because export failures are silent — and decide attribution and cost boundaries before launch, not after.

Chris: Now the traps, because the exam has four of them clustered here and they are all plausible-sounding.

Koko: First trap: telemetry captures what the agent did. The misconception is that logging is on by default for content. Structural telemetry is on by default. Content — prompts, tool results — is opt-in.

Chris: Second: the version I keep seeing on teams is treating per-user attribution as a nice-to-have dashboard feature. It is not cosmetic. A service-account trail can only answer the application did something. An audit trail answers who caused it.

Koko: Third trap: granularity can be added later. It cannot. One shared key across four products makes per-product cost unrecoverable retroactively.

Chris: And the exam's favorite: a bad answer means a bad prompt. The trace data says otherwise — most failures live in tool execution and permission waits, outside the model entirely.

Koko: Alright. Let us close the whole lesson. Chris, you have shipped these systems — give me the shape of what we covered across all seven modules in your own words.

Chris: Fourteen ideas, one shape. The constraint set, the decision record, the trade-off table, the accuracy statement, the runbook, the gate table, the migration notice, the enforced scope, the telemetry, the open trace — every single one of them created before the moment it is needed. Not as documentation for documentation's sake. Because the exam, and honestly production too, keeps asking the same question: when two options both look defensible, which one holds a document somebody outside the team can actually check?

Koko: That is the discriminator across Domains Six and Seven. Not which answer sounds most technically sophisticated. Which answer is the one a regulator, an auditor, or your on-call engineer at two in the morning can verify independently.

Chris: Pre-authorized, pre-scoped, pre-documented, pre-instrumented. If you are deciding it at incident time, you are already late.

Koko: Carry that in. The drills and flashcards for Domains Six and Seven — all of it — are at KokoAI Academy on koko knows dot A I. Go build the muscle memory before exam day.

Chris: You have done the hard thinking. Now it is just reps.

Koko: You have got this. Go show them what an architect who can defend their choices looks like.