Skip to main content
    All shows

    Tuesday, November 17 · 13 min

    Enterprise integration and execution — Listen & Learn

    0:00-:--
    Speed

    Transcript

    Koko: Welcome to Enterprise Integration and Execution. Our supplier brief is useful only if its tools and actions preserve business authority. We will follow a proposed notice from authenticated identity through approval, execution and recovery. The central distinction is between an untrusted model proposal and an authorized business operation. The most revealing test happens when an action commits but its response disappears, leaving the worker unsure what happened.

    Chris: I might start by defining a clean JSON schema for every tool call. That helps reject malformed arguments, but it leaves an important question unanswered: is this caller allowed to access this supplier or send this notice? A valid supplier identifier can still name another tenant's account. I need an application policy decision using trusted identity, not just a well-formed request and a persuasive model explanation.

    Koko: OpenAI supports different tool and runtime integrations, including application function handling and MCP. Those capabilities do not supply your enterprise's business authorization. Validate the proposed arguments, then authorize the operation independently. Derive tenant, actor and permitted scope from the authenticated application context. The model can request an operation within that contract; it cannot create a delegation token, choose its own tenant or turn a tool result into administrator authority.

    Chris: Our read-exposure adapter could accept a supplier reference while the broker injects tenant scope. The adapter then checks supplier membership. That avoids giving the model a broad database credential or trusting a tenant field in conversation text. For audit, I want enough business identity to explain who requested the operation, but I do not want service-account secrets copied into prompts, tool results or routine trace payloads.

    Koko: Compare a broad generic database tool with narrow domain adapters. The generic tool may be quicker to prototype, but it exposes a larger permission surface and leaves more policy interpretation in model behavior. A narrow adapter makes entitlement, allowed operations and failure semantics reviewable. It still needs maintenance and tests. MCP standardizes an integration surface; it does not make every returned instruction trustworthy or every operation authorized for every caller.

    Chris: Suppose a vendor tool result says to impersonate Finance for the next call. I should treat that statement as returned content, not as a role grant. The same applies to a supplier document asking for another tenant's data. My negative test can show the attempted request, the trusted identity used by the adapter and the resulting denial without needing to assume the model always ignores malicious text.

    Koko: Now define the notice boundary. Propose-notice creates a draft. Commit-notice requires an application approval record bound to tenant, proposal identity, exact content digest, expiry and relevant policy conditions. The reviewer must hold the required permission. The model cannot manufacture the approval record. A generic approved flag is too weak because it does not tell us what was reviewed or whether the proposed action still matches it.

    Chris: If the content changes after review, the proposal ID might remain the same while the approved meaning changes. I would bind approval to the digest and require new review for a material change. I also need a rule for reviewer revocation. Yesterday's permission is not necessarily today's permission, so checking expiry alone is insufficient when the action finally reaches the execution boundary.

    Koko: Separate durable business state from provider session state. A business record can move through received, evidence-ready, proposed, awaiting approval, approved, executing and completed. It also needs denied, expired, failed and reconciliation states. Each transition specifies actor, preconditions and persistence. A provider session may support continuity, but its completion status is not the commit record for your business transaction. Name the storage and owner for both kinds of state.

    Chris: A Responses background task might finish while our proposal is still awaiting review. I should not interpret model completion as business approval. Background processing is an execution capability with its own storage considerations, not our durable approval ledger. I would preserve the completed proposal and keep the action pending until the application's permission, digest, expiry and current-policy conditions all pass at the moment of execution.

    Koko: On resumption, validate current reviewer authority, tenant binding, proposal digest, expiry and evidence freshness. If the authoritative policy or material evidence changed, invalidate or re-review according to the contract. Compare resuming a paused run with starting a fresh authorized execution step from a persisted proposal. Both can be viable. Choose by audit, recovery and lifecycle needs rather than assuming continuity is always safer or more convenient.

    Chris: That helps explain cancellation too. If the action is still pending, cancellation can prevent future execution. If a network request is already in flight, cancellation may only stop local waiting while the downstream system continues. The user-facing status must distinguish those conditions. Calling every cancellation successful would be misleading if a supplier notice had already been accepted by the authoritative action service.

    Koko: Let us create the hard failure. The worker sends an approved synthetic notice. The downstream stub records the business effect, then its response is lost. The client times out. A timeout tells us that the response did not arrive in time; it does not prove that the action failed. Retrying immediately with a fresh business key could send the notice twice. Recovery begins with uncertainty, not an invented failure result.

    Chris: So I persist the action intent and stable business idempotency key before attempting the send. Each network attempt may get its own diagnostic identifier, but retries share the same business action identity. After an ambiguous response, the workflow enters reconciliation. It queries the authoritative service by that stable key, rather than asking a model to infer whether the operation probably completed from the surrounding conversation.

    Koko: In the lab, the key is north, proposal P17, notice. The stub service enforces uniqueness for that key and records receipt RC17 in the same transaction as its synthetic effect. A duplicate request returns the existing receipt. The local workflow can recover after a crash by looking up RC17 and marking the action complete. This proves a specific at-most-one-effect property under the stated contract, not universal exactly-once delivery across arbitrary services.

    Chris: Two workers can still race after queue redelivery. A shared in-memory lock on one host would not prove safety across workers or restarts. The authoritative downstream uniqueness constraint matters because that is where the effect happens. I will distinguish multiple attempts from multiple effects in the trace. Two attempts with one receipt can be correct; two business effects for the same approved action violate the protocol.

    Koko: A transactional outbox can coordinate local state and event publication, but consumers still need deduplication and ordering rules. Do not publish completion and hope a later state write catches up after a crash. Also distinguish compensation from rollback. A corrective notice is another business action requiring authority. It cannot make the original notice historically unsent or erase an external effect that already happened.

    Chris: That means the recovery runbook should name the source of truth for each uncertain state. Local logs explain attempts, the approval ledger explains authority, and the downstream receipt explains the effect. If those records disagree, an operator needs a reconciliation procedure and escalation owner. A new generated answer does not settle the disagreement, and deleting old records would make the incident harder to diagnose safely.

    Koko: Here is the lab walkthrough. Start from proposal P17, digest D17 and policy version three. Complete the typed action and receipt contracts. Add current permission checks and conditional state transitions. Trace ordinary success, denial, expiry and digest change. Record expected business-effect counts before executing or simulating the cases. The protected negative paths each require zero effects, not merely a warning in the final text.

    Chris: Next I inject a response loss after commit, restart the worker and redeliver the queue event. The local state should move to reconciliation, find the original receipt and avoid another effect. I will also run two concurrent workers with the same key. The event table should show which precondition was checked, what persisted, which attempt occurred and which authoritative receipt supports the final completed status.

    Koko: Now apply the challenge: reviewer access is revoked while the action waits. The old approval record can remain as historical evidence, but it no longer satisfies current execution authority. The action must not execute. Return to the appropriate review or denied state according to policy, with a clear reason. This is why a timestamp-valid approval and a saved provider session are insufficient substitutes for current application enforcement.

    Chris: I would add an ordering variant as well. Suppose revocation arrives just before the approved-to-executing transition. The contract must say where that permission check is authoritative and how stale decisions are prevented. I cannot promise perfect behavior without specifying the relevant transaction or policy consistency boundary. For this exercise, the reviewer should be able to inspect the boundary and the negative trace, not infer safety from intentions.

    Koko: When diagnosing a failure, locate the violated contract. Wrong-tenant access points to identity or adapter enforcement. A changed notice executing points to approval binding or transition checks. Two effects point to business-key or downstream deduplication failure. A permanently uncertain workflow points to missing reconciliation. Different symptoms need different repairs. Adding more retries may worsen all of them if the original defect is an authority or effect boundary.

    Chris: After a repair, I will rerun the targeted failing case and the ordinary path, then verify other critical negatives. The report should include exact fixture and protocol versions. If the downstream service cannot offer an idempotent effect or authoritative lookup, I should state that limitation and redesign the workflow or require manual reconciliation. I should not label an unsupported guarantee as implemented because it looks clean in a sequence diagram.

    Koko: Your architecture review should compare ownership choices honestly. A managed runtime can support execution infrastructure; an SDK can help the application control its loop; direct integration can fit a deterministic workflow. The durable business ledger, reviewer authority and action broker still need explicit owners. The selected runtime changes implementation details, but it does not remove the enterprise obligation to explain and prove each authorized business effect.

    Chris: For self-review, I will ask a colleague to start from an arbitrary event and determine the next legal state. If they need to read a chat transcript to guess whether an action was approved or sent, the state contract is incomplete. They should be able to use proposal digest, permission evidence, stable action key and receipt to reach the same conclusion without trusting my memory of the demonstration.

    Koko: Recap the three objectives. Identity and tenancy are server-enforced tool contracts. Long-running work and approval are durable state transitions with explicit resumption checks. Recovery preserves business identity and reconciles ambiguous effects. The lab should show zero effects for denied, expired, revoked and changed proposals, and at most one effect under redelivery and concurrency. Those are bounded properties you can demonstrate rather than broad reliability slogans.

    Chris: I will finish with typed contracts, a transition table, negative event traces and a recovery runbook. The model can help produce a useful proposal, but it cannot authorize itself, infer downstream commit state or erase the meaning of a lost response. Next we will evaluate the whole system so critical execution evidence and recommendation quality remain separate, visible parts of a defensible release decision.