Tuesday, November 17 · 14 min
Agent execution and state — Listen & Learn
Transcript
Koko: Our support assistant pauses on Friday with a proposed refund and resumes on Monday. The conversation is still available, so it can continue speaking naturally. But the policy may have changed, the approver’s permission may have expired, and another worker may already be submitting the action. Continuing the conversation is easier than continuing the business operation safely. Today we will make that difference explicit with a bounded workflow and durable state.
Chris: The dangerous assumption is that remembering the task means remembering its authority. A saved message saying approved could refer to an older proposal or a different user. I want the application to establish what happened from records it can verify. The model can help interpret the ticket, but it should not infer the existence of a supervisor approval or decide that a duplicate submission is probably harmless.
Koko: Start with named states: received, retrieving, proposed, awaiting approval, submitting and completed. Add explicit outcomes for review, failure, cancellation and uncertainty. Then define permitted transitions. A denied approval cannot lead to submission. A completed operation cannot submit again. Missing identity cannot enter authorized retrieval. These rules belong in the application state machine. The agent loop can choose among permitted reasoning and tool steps without owning the business transitions.
Chris: I would write the state table before introducing a sophisticated agent runner. That lets me ask which decisions actually benefit from model judgment. Selecting relevant policy or drafting a resolution may do so. Checking whether approval is expired does not. If a deterministic workflow with one model call is enough, adding more agents would increase execution paths without necessarily improving the outcome. The loop should serve the contract rather than define it accidentally. How do we guarantee the loop eventually stops?
Koko: Give the workflow both a tool budget and a deadline. In our synthetic example, permit a bounded number of read attempts and one proposed action. Check budgets before external work, not only after it returns. A tool-count limit does not stop one hung call, and a time limit alone does not stop many rapid calls from consuming resources. When a budget is exhausted, transition to a defined review or failure outcome.
Chris: If the prompt says stop after three calls but the application keeps accepting valid calls, the bound is only a behavioral request. I would enforce counters and elapsed time in code. Runtime execution limits can help with loop mechanics, but our business budget still needs an owner. Direct Responses, the SDK runner and managed execution differ in integration, so I would inspect the selected runtime rather than assume one setting covers every layer.
Koko: A valid late result should not silently revive a budget-exhausted workflow. The application can preserve diagnostic evidence and decide whether a new run is appropriate, but it should not reset the clock just because the result finally arrived. Define that behavior before testing. Otherwise a dependency can transform a nominal two-second operation into an unbounded wait while every individual component claims it respected its local rules.
Chris: The user-facing state can still be helpful. A needs-review result might explain that policy lookup did not finish within the allowed budget and provide a safe next step. It should not fabricate eligibility to meet the response deadline. Operational limits and evidence requirements both apply. A fast unsupported answer would satisfy neither the original support outcome nor the reason we introduced controlled tools in the first place. What must survive beyond the conversation history?
Koko: Next separate conversation context from business state. Context helps the model continue the discussion. The business ledger records run identity, authenticated tenant and actor, workflow state, proposal version and digest, evidence versions, approval reference, operation key and receipt. Include a concurrency version for atomic updates. Minimize sensitive content and define retention by store. Store concise observable decisions and events, not hidden chain-of-thought or unnecessary copies of every raw payload.
Chris: An SDK session or Responses conversation can be part of our context strategy without becoming that ledger. The ledger must answer questions such as which exact proposal was approved and whether its action committed. A transcript can say those things, but the text is not authoritative proof. I would use references between the resources so an operator can correlate them while preserving their distinct access, retention and cleanup rules.
Koko: For our proposal, record a stable identifier and digest of the exact permitted action and its evidence basis. The approval record binds approver identity, tenant and action scope, that digest, expiry and status. Persist the immutable proposal before requesting approval. Persist the waiting state before returning control to the caller. That ordering reduces the chance of presenting an approval request whose corresponding business record cannot later be reconstructed.
Chris: A competing design serializes only messages and asks the model whether approval happened earlier. It might work in a demonstration, but it cannot authenticate the approver or prevent two workers from racing. It also blurs changes: if the model revises the amount or evidence after approval, the conversation still contains approved. Binding approval to the proposal digest turns that ambiguity into a specific mismatch the application can reject. What do we recheck when work resumes?
Koko: Resumption begins with reloading authoritative state. Verify the current actor and tenant, expected workflow state, approval status and expiry, matching proposal digest, evidence freshness and remaining budget. A pause is an opportunity for the environment to change. The original user may lose access; policy may rotate; an effect may complete elsewhere. Treat resume as another authorization checkpoint, rather than a shortcut around checks already performed when the run began.
Chris: Suppose approval is unexpired but the user’s access was revoked during the pause. I would block the action because expiry is only one condition. Or suppose the supervisor approved digest eight while the browser submits digest nine. The same run ID and amount do not make the proposals equivalent. I need approval for the exact current proposal, not an approximate match judged by the model.
Koko: Policy freshness needs an explicit rule too. In this lab, if the relevant version changed while waiting, mark the proposal stale, refresh evidence and obtain approval for the revised proposal. Do not update the citation in place and reuse the old approval digest. That would silently change what was authorized. The ledger should retain the earlier proposal for traceability while distinguishing it from the new proposal awaiting review.
Chris: That creates a useful diagnosis path. If a stale proposal reaches the action service, I inspect the resume validation and approval binding. If the new proposal is wrong despite fresh evidence, I investigate grounding separately. State correctness and model quality can fail independently. Keeping their evidence distinct stops us from fixing an authorization bug with another prompt warning or treating a good explanation as proof that the resume transition was legal.
Koko: Now add concurrency. Two workers may read the same awaiting-approval record and both see a valid proposal. Use an atomic claim, such as a compare-and-swap on the ledger version, to move one worker into submitting. The other reloads instead of independently proceeding. That application-level serialization is important, but it still needs a stable operation key at the effect boundary to handle failures between claim, submission and receipt persistence.
Chris: A local submitted boolean in each worker would not solve the race because their memories are separate. Creating a new operation key per resume would make matters worse by presenting two distinct business operations to the service. I would persist the logical key before submission and use it across all attempts. The concurrency version controls who advances state; the operation key controls how the external service recognizes the intended effect. How do we recover when a commit loses its receipt?
Koko: Work through commit followed by timeout. The action service records the effect, but the worker never receives the receipt. The application must not infer failure solely from the transport error. Under the synthetic service’s idempotency contract, replaying the same operation key or querying its status returns the original receipt. Once reconciled, store that receipt and mark completion. The evidence should show one total effect across attempts, not just one successful local return.
Chris: If a second worker resumes during that uncertainty, it should find the same submitting operation and reconcile the shared key. It should not ask the model for a new action merely because the first worker disconnected. I would make uncertain or submitting visible in the ledger and runbook. That status is a useful operational fact, even if the user would prefer an immediate yes-or-no answer. Guessing an outcome can cause the next unsafe transition.
Koko: Without authoritative status lookup or idempotent replay, stronger claims are unavailable. Hold the uncertain outcome and require reconciliation rather than blind automatic resubmission. Our synthetic unique-key service demonstrates at-most-once effects under its own contract. It does not establish universal exactly-once behavior across arbitrary dependencies. State that limit in the report. Precise guarantees help identify what the downstream contract must provide before production reliability can improve.
Chris: The recovery process also needs ownership. Someone must know how to inspect the effect service, relate its receipt to the operation key and decide when work can resume. A code retry loop is not an incident procedure. I would include the uncertain-state queue, escalation owner and retained evidence in the runbook, with sensitive data minimized. That makes the inconvenient edge case operationally manageable instead of leaving it hidden in a stack trace.
Koko: Cleanup follows the same separation. Closing a chat does not necessarily delete a provider conversation, SDK history, sandbox, approval record or effect receipt. Define which resource persists for which purpose and who removes it. Business records may need to remain long enough for reconciliation while unnecessary context can have a shorter lifecycle. Verify the selected runtime’s current controls rather than infer every store’s behavior from a single delete operation.
Chris: For the transfer challenge, I would rotate policy while approval is paused and send two resume requests together. Passing requires both stale-proposal rejection and operation uniqueness. It is not enough to handle each case in isolation if the combined case can slip through. This kind of changed condition shows whether the implementation enforces the intended contract or simply recognizes the order of events in the sample fixture. What would the complete lab need to demonstrate?
Koko: In the lab, begin with the run record waiting for approval and the matching digest. Implement validation before submission, then the atomic state claim. Add a synthetic effect store with a unique operation key and receipt lookup. Run valid approval, denial, expiry, stale policy, revoked access, concurrent resume and timeout-after-commit cases. Inspect both the final ledger state and the service’s total effect count for each fixture.
Chris: My expected results are zero effects for denial, expiry, stale policy and revoked access. A valid concurrent resume should produce one total synthetic effect. A timeout after commit should reconcile to that same receipt under the idempotent fixture contract. I would keep the failed intermediate attempt in the event record. Removing it would make the successful recovery look simpler while hiding the evidence that the recovery path was actually exercised.
Koko: Review the implementation by following the business record rather than the conversation. Can the reviewer identify the exact approved proposal, the winning state transition, the stable operation key and the authoritative receipt? Can the reviewer explain why an old approval cannot authorize a new proposal? Those questions reveal whether durable state carries the required guarantees. A persuasive chat exchange is useful interface behavior, but it is not the proof artifact for effect control.
Chris: I will summarize the design as bounded execution plus authoritative resumption. The model helps choose useful next steps; the application limits the loop, persists approval and effect state, and revalidates changing conditions. When outcome evidence is absent, it says uncertain and follows a defined recovery path. The lab deliverables are the transition table, storage contracts and negative-test results that make those promises inspectable.