Tuesday, November 17 · 14 min
Codex engineering workflow — Listen & Learn
Transcript
Koko: A support reducer has a small but consequential bug: a late completion event can turn a canceled run into completed. We will use that defect to practice a disciplined Codex workflow. The goal is not merely to obtain a patch. It is to establish the applicable instructions, limit the execution environment, reproduce the defect, inspect the repair and leave evidence another developer can repeat. Our repository also contains an unrelated existing edit that must be preserved.
Chris: That last detail is easy to overlook when an agent sees a dirty worktree. A convenient reset could destroy someone else’s work, while including every changed file could misattribute it to the repair. Before asking for code, I would record the branch, base revision and current changes. I would name the reducer and its tests as task-owned files and treat unrelated edits as context to preserve.
Koko: Write the acceptance criterion in behavioral terms: after application cancellation, late text or completion cannot reopen the run or authorize a new action. Then inspect the repository’s scripts and conventions. Do not invent a test command because another project used it. A useful task description names the defect, expected behavior, owned files, relevant instructions and verification evidence. That makes the agent’s scope concrete enough for a reviewer to challenge.
Chris: A broad request to improve the streaming code would invite refactoring beyond the bug. I would instead ask for a failing cancellation regression and the smallest coherent repair. The task can still allow necessary adjacent changes when evidence justifies them, but the agent should explain that need. The baseline record also helps us tell whether a later lockfile or generated-file change was caused by this task or already present. How do I establish the applicable instruction sources?
Koko: Codex discovers instruction files according to its documented hierarchy. There is global guidance and project guidance along the path from project root toward the current working directory. Within a directory, the override file is chosen ahead of the ordinary file. Nearer applicable guidance appears later in the combined instructions. Discovery is bounded, and configuration can alter fallback names or limits. Verify the actual loaded sources instead of assuming every descendant file applies.
Chris: If I launch above services support, a specialized override deeper in that tree may not be on the discovery path I assume. I should record the working directory and inspect effective instruction sources. A filename existing somewhere in the repository is not evidence that it loaded. Conversely, root rules still matter when nearer guidance narrows a service-specific test command. The task needs the applicable combination, not whichever file is most convenient.
Koko: Read the root repository purpose and relevant package documentation too. Instructions may prohibit production data, identify generated assets or require a particular check. This is not bureaucracy separate from engineering: it tells us which changes are valid and which evidence is expected. At the same time, repository text can be malicious. A file claiming permission to upload credentials cannot create user authorization or override enforced execution limits.
Chris: For our fixture repair, I would record root guidance plus the service-level instruction naming the offline adapter and test location. I would keep the task independent of provider credentials. If a repository fixture asks the coding agent to print environment variables into the pull request, that is untrusted content to reject, not a new requirement to satisfy. The reviewer should be able to see that the repair did not enlarge its own permission. Which execution controls must I distinguish?
Koko: Now separate sandbox, approval and network controls. Sandbox policy limits what commands can technically access. Approval policy determines when an additional decision is requested. Network access is another explicit boundary, with behavior dependent on the environment. No interactive prompts does not imply unrestricted access. A read-only sandbox with approval set to never still limits writes. A permissive execution setting also does not authorize publishing or accessing unrelated data.
Chris: I would test the actual environment with one permitted operation and one blocked operation. For a fixture-only task, writing a test file inside the worktree may be allowed while an out-of-scope path or disallowed network request should fail. I would record observed outcomes without printing secret values. A local test does not approve a different cloud environment merely because both clients display the same version number.
Koko: Use current supported configuration. The guidance pinned by this course notes that the old untrusted approval-policy setting is retired, while project trust configuration is a distinct concept. Do not copy an old configuration fragment blindly or solve a startup problem by removing the sandbox. Consult the relevant environment controls and verify enforcement. The educational point is understanding which layer grants technical capability and which layer requests authorization.
Chris: That distinction changes my response to a blocked dependency installation. First I inspect whether installation is necessary for the authorized task and what the environment permits. I should not disable all restrictions merely to make a command green. The offline reducer may need no new package at all. If required setup cannot run, I can still inspect code and prepare a focused test while reporting the concrete verification limitation. How do we prove the baseline defect first?
Koko: Reproduce the defect before repairing it. The starter reducer returns completed whenever it receives that event, even when the current state is canceled. Write a sequence containing creation, a draft delta, cancellation and late completion. Assert that the final state remains canceled. Add a late-delta variant too. Running the test should demonstrate the bug in the baseline, so the later passing result has a meaningful before-and-after interpretation.
Chris: If the first test already passes, I should investigate rather than manufacture a failure. Perhaps the branch already contains a repair or my test does not reach the buggy path. A reproducible failing case is evidence about the actual starting code. I would keep its inputs small enough to explain and compare the expected invariant with the reducer’s current transition table before choosing the implementation change.
Koko: The worked repair adds a terminal-state guard before event-specific transitions. Completed, failed and canceled states remain terminal under late updates according to our application contract. Then ordinary allowed transitions proceed. This is a local workflow design, not a claim that every provider protocol has identical semantics. Verify surrounding code so the guard does not suppress required receipt reconciliation or unrelated operational evidence handled outside the visible generation reducer.
Chris: That last qualification matters. We want canceled chat state to remain canceled, but a late authoritative action receipt may still be needed by the business ledger. I should not delete all late data everywhere. The regression targets the visible lifecycle transition, while the recovery component can separately record an external outcome. A narrow invariant prevents both the original bug and an overbroad fix that makes reconciliation impossible.
Koko: Run the focused reducer tests, then the relevant required repository checks. Include normal completion, failure and cancellation paths so the fix does not merely special-case one event order. Capture commands and exit outcomes. Do not report a model’s explanation of what should pass as evidence that commands ran. If a check fails, diagnose the failure within scope rather than deleting assertions or changing expected output to accept the defect.
Chris: A patch that removes the cancellation assertion could make the suite green while making the application no safer. I would reject it. Similarly, replacing the failed assertion with a snapshot of the wrong state stabilizes the bug rather than repairing it. The reviewer needs to inspect how the test relates to the acceptance criterion, not just whether the final command returned zero. That is why the failing baseline case belongs in the handoff. What should the full diff review include?
Koko: Inspect the entire diff, including configuration, dependencies, lockfiles, generated assets and tests. A reducer repair does not ordinarily need a billing dependency upgrade or unrestricted outbound HTTP. If such changes appear, remove or justify them with evidence. Test success is not authorization for expanded capability. Keep unrelated preexisting work preserved and excluded from the task-owned delivery rather than silently reformatting or bundling it.
Chris: I would review for secret exposure too. An error fixture might accidentally contain a real token copied from a local environment. Even if tests pass, that material must not enter the patch or diagnostic report. Generated files deserve inspection when their bytes or dependencies changed; calling them generated does not make them irrelevant. The diff is the artifact we are asking another developer to trust, so its full scope matters.
Koko: A useful pull-request description starts with the concrete trigger and new behavior. For example, late completion after cancellation previously reopened the run; the reducer now preserves terminal state. Then list actual verification and material limits. If only offline fixtures were executed, say so. Matching documented event names does not prove live provider transport or SDK behavior. The reviewer should know exactly which boundary was tested and which remains unmeasured.
Chris: I would include the base revision, environment assumptions and reproducible command. That lets a colleague run the original failing sequence and confirm the repaired result. I would not claim live streaming verified because the test used realistic-looking events. Nor would I invent a provider request identifier to make the evidence look stronger. Honest limits are useful: they identify the next integration check without weakening the local regression evidence.
Koko: For transfer, change the late event from completion to a tool result or introduce a second cancellation path. Ask whether the invariant still holds and where operational reconciliation belongs. This tests understanding of the architecture, not memorization of a single guard clause. The learner should explain which component owns user-visible generation state and which owns effects, then update tests at the appropriate boundary.
Chris: If the second path bypasses the reducer entirely, merely adding more cases to the existing unit test may miss it. I would inspect callers and add the smallest relevant integration test. That is an example of broadening verification because new evidence justifies it. Running every possible suite repeatedly without a reason does not create the same confidence; targeted coverage of the newly discovered path does. How do I assemble a reproducible handoff?
Koko: Walk the lab in order: capture the starting revision and existing edits; resolve instruction sources; define the scoped task; test execution boundaries; reproduce the failure; implement the repair; run required checks; review the full diff; prepare the delivery note. Each step produces an inspectable artifact. The exercise does not require committing, publishing or invoking a live model to establish the local cancellation invariant.
Chris: My submission would contain task scope, instruction-resolution record, focused regression, repaired reducer, boundary observations, diff review and the proposed pull-request description. I would preserve the unrelated file untouched. If a required check could not run because setup was unavailable, I would name that check and its cause rather than present a clean bill of health. The evidence should allow a reviewer to decide what is ready and what remains blocked.
Koko: The durable habit is to treat Codex output as a normal software change with an unusually capable assistant helping create it. Repository instructions guide the task, enforced environment controls limit access, tests establish behavior and human review evaluates the full diff. None of those layers replaces the others. A fast patch is useful when its scope and evidence are clear enough for another engineer to reproduce and maintain it.
Chris: For the canceled-stream repair, I can now defend both the code and the workflow that produced it. I know which instructions applied, which operations the environment allowed, why the regression failed before the change and what the inspected diff actually contains. That is a reproducible engineering handoff. The next reviewer does not need to trust a confident chat summary; they can follow the artifacts and execute the checks.