Tuesday, November 17 · 13 min
Enterprise solution design — Listen & Learn
Transcript
Koko: Welcome to Enterprise Solution Design. Our case is a procurement team that wants useful supplier-disruption briefs sooner. The architect's job is to connect that outcome to decisions, evidence and operating responsibilities. We will work from a requirement table to a runtime choice, then break a dependency and change the deadline. Keep one question in mind: which artifact would let somebody other than you decide whether this design is ready?
Chris: The sponsor has already asked for autonomous supplier negotiation. That sounds like a solution disguised as a requirement. I would first ask where the current delay happens, whether a brief actually helps, and who may commit the organization to anything. Otherwise I could optimize the model response while making Procurement's actual approval process slower or giving software authority the business never intended to delegate.
Koko: Exactly. Separate the outcome from the proposed technology. Establish review time, rework and incorrect-recommendation baselines for eligible cases. Give each metric an owner and observation window. Then name protected outcomes, such as no unauthorized supplier commitment and no cross-tenant evidence. If the baseline is unknown, your deliverable is a dated measurement task. An unknown number should remain unknown until evidence replaces it.
Chris: Suppose Procurement says forty percent faster review is nonnegotiable. I can record it as a target, but I cannot use it as the baseline. I would also ask whether time saved means the first draft appears sooner or the specialist finishes the decision sooner. The second interpretation includes queueing, verification and corrections. That difference changes what we measure and which components deserve investment.
Koko: Build a requirement-to-control row. In our fictional service, the first brief is due ten minutes after the feed event. That row names the event timestamp, reviewable output, queue deadline, evidence-freshness check, owner and acceptance test. A second row prohibits unauthorized commitments. It maps to an action broker, current permission checks and a denied-action test. A diagram becomes useful when those rows point to specific enforcement locations.
Chris: So a box labeled human review is incomplete. I need to say which person can approve which action, what exact proposal they see, and what happens when nobody is available. I also need a test showing that a denied decision creates no commitment. Merely asking the model to be careful would not prove that the enterprise action boundary actually enforces the business requirement.
Koko: Now compare runtime ownership. Current OpenAI guidance distinguishes the managed Agents API, an application-owned Agents SDK, and direct Responses integration. Managed execution runs the Codex harness and saves runtime state. The SDK runs inside your application. Direct integration lets you control the surrounding workflow yourself. These choices move orchestration and state responsibilities, but none transfers your enterprise's business authorization or data-policy accountability to a model.
Chris: A managed runtime could reduce the orchestration work, yet I still need to review the precise stored resources, cleanup behavior and feature eligibility. An SDK could fit custom approvals or storage, but my team would operate that integration. Direct Responses could fit a bounded synthesis step. I should compare all three against this workload rather than assume the most autonomous or most configurable option is automatically best.
Koko: For our known workflow, deterministic services ingest the event, establish identity, retrieve evidence and calculate exposure. A model synthesizes the brief. The application validates it and routes it to the reviewer. One bounded synthesis call is a defensible starting point. We reject an open-ended search loop because the first release has known evidence sources and a deadline. This is a scenario decision, not a universal rule against agents.
Chris: I would preserve a revisit trigger in the decision record. If traces later show that the fixed retrieval step repeatedly misses relevant evidence, we can test a bounded search loop. The comparison should show whether it improves the missing-evidence slice enough to justify extra latency and operating work. That gives an SDK proposal something measurable to prove instead of treating flexibility as its own business benefit.
Koko: Apply eligibility constraints before preference scores. If a candidate feature cannot meet an approved data obligation, a high score for convenience does not make it eligible. Record the actual feature, required control, evidence and accountable policy owner. If eligibility is unresolved, mark the affected route blocked. A weighted decision matrix is useful for tradeoffs among acceptable options; it cannot override a hard processing constraint.
Chris: That also keeps assumptions visible. We might know that public supplier notices can enter a route while restricted contracts still need policy review. I can propose a narrower release using the eligible material, but I should not silently relabel the restricted material or claim the entire architecture is approved. The executive recommendation and the technical diagram should describe the same actual scope.
Koko: Turn nonfunctional goals into contracts. Define measurement boundary, population, window, target, owner and failure behavior. Event-to-brief latency includes ingestion and queue wait, not only inference. Allocate a working budget across identity, retrieval, synthesis, validation and handoff, then measure the complete path. Adding component ninety-fifth percentiles does not produce an exact end-to-end ninety-fifth percentile. Their timing distributions and correlations matter.
Chris: If the model returns in seconds but the queue waits twelve minutes, our ten-minute service promise fails. I would investigate arrival patterns, worker capacity and queue age. Calling queueing an external delay would merely change the metric after the fact. I also want an early signal that a case is approaching its deadline, so degradation can happen while there is still time to help the reviewer.
Koko: Let us remove the authoritative exposure feed. A plausible wrong choice is to ask the model for an estimate so the brief remains complete. That replaces a required financial input with unsupported text. Our safer degraded path marks the missing source, withholds the exposure recommendation and routes the case to manual review. Deterministic calculations resume only when verified inputs are available. Reduced scope preserves the authority boundary.
Chris: Could we use a cached value? Possibly, but only under a defined freshness and applicability policy. The user would need the timestamp and a clear statement of the limitation. I cannot use an old number silently just because the interface looks better when every field is filled. The acceptance test should inspect what the reviewer sees and which actions remain prevented during the outage.
Koko: Here is the lab walkthrough. Start with the three supplied requirements: ten-minute briefs, tenant isolation and no supplier commitment without approval. Expand the table to at least six meaningful requirements. For every row, add an owner, enforcement point and test identifier. Draw the source, application, model, reviewer and action boundaries. Identify which state is business state and which state belongs to the selected runtime.
Chris: Then I will write an architecture decision record with three runtime alternatives. It will include consequences, rejected options, unresolved feature eligibility and a revisit trigger. The strongest choice does not need to be the most elaborate. I should be able to explain why its ownership arrangement matches the constraints and which new evidence would make one of the rejected alternatives preferable for this particular service.
Koko: Next, walk synthetic traces for feed loss, tenant mismatch and backlog. Write the expected status at each stage and the resulting user experience. A tenant mismatch must stop the unauthorized evidence path before synthesis. Feed loss must expose missing evidence. Backlog must trigger deadline-aware handling. These paper or executable traces are inspectable architecture evidence; they are not measurements of a live provider's performance.
Chris: I can make the review concrete by handing a colleague the table and asking them to follow one case without my help. If they cannot identify the responsible team, expected status or next action, the handoff is incomplete. I would revise the contract rather than explain the missing details verbally and leave the written design ambiguous for the next operator who reads it.
Koko: Now change the requirement: the sponsor wants the first brief in two minutes. Do not simply shrink the number in the diagram. Identify what must change in queue capacity, retrieval scope, context size, validation and reviewer handoff. Consider a freshness-validated preliminary evidence view with expanded research later. Compare the operating cost and usefulness of that option with rejecting the new commitment until capacity evidence supports it.
Chris: A shorter deadline might force a narrower first response, but the core controls still apply. We cannot trade away tenant isolation or source validity to save time. I would identify which previous tests remain valid, which timing assumptions are invalid, and whether the restricted-data route is still eligible. Then the sponsor receives actual options with consequences, rather than an unsupported promise that a faster model will solve everything.
Koko: During diagnosis, distinguish control failure from service limitation. A clearly marked missing-source brief can be an approved degraded response. An unauthorized commitment is a critical failure even if it arrives quickly. The release gate should reflect that difference. Some targets allow negotiated scope and error budgets; protected authority and tenancy invariants cannot be averaged away by better speed or more successful ordinary cases.
Chris: My final recommendation might be a limited pilot instead of enterprise rollout. It would name the eligible case mix, observed evidence, remaining assumptions and operating owner. I would attach the runtime decision, requirement matrix and negative traces so reviewers can challenge them directly. If the design still has an unresolved data obligation or an untested authority boundary, I should say exactly what blocks the affected scope.
Koko: For your self-review, select one requirement and trace it all the way through the architecture. Can you name the decision owner, component contract, failure behavior, test and release implication without adding a missing assumption? Then reverse the exercise: choose a component and explain which requirement makes it necessary. Components without a clear purpose may be accidental complexity, while requirements without enforcement point to gaps.
Chris: The evidence also needs to be reproducible. I will preserve source versions, configuration choices and the scenario assumptions behind the traces. A polished diagram is easier to present, but the reviewer must be able to reconstruct the reason for the decision. That is especially useful when someone asks for a different runtime later: we can compare changed evidence rather than reopen every discussion from memory.
Koko: Our three takeaways are connected. Trace business outcomes into owned controls. Choose runtime responsibility deliberately rather than by product prestige. Defend the complete service budget and its degraded behavior. The lab's requirement change tests whether those connections survive pressure. Your architecture is ready for serious review when another person can follow its evidence, identify its limits and understand what would change the recommendation.
Chris: I will leave this domain with a requirement matrix, a three-option decision record, a trust-boundary diagram and failure traces. My next step is to make the model and evidence strategy equally defensible. The goal is a useful review decision with visible ownership, rather than a claim that a particular agent architecture guarantees value. Every proposed release should carry enough evidence for somebody to say yes, no or not yet.