<!-- Generated from governed source; do not edit. -->

Source: Koko-os advanced program v0.1.0; commit e8416fd254dc300caa65b0d042309baa98864012; package SHA-256 90c538e5ec377f9bd6eb2347b19240c2ba959624e05ff868ee34c7edabaa68d8; binding commit_verified.

# Controlled support-resolution assistant

Build a synthetic support assistant for two tenants. It retrieves versioned policy evidence, drafts a resolution and submits an action only after an authenticated supervisor approves the exact proposal. Work may pause, resume concurrently and encounter uncertain action outcomes. No real customer records or provider keys are required for the offline reference path.

## Assignment

Deliver a runnable offline implementation and a separately documented optional live-provider adapter design. Prove deterministic control invariants with synthetic fixtures, diagnose a planted trace defect, evaluate result quality on a frozen dataset and defend a release decision. The evaluator changes one constraint during review; revise the affected contracts and evidence.

## Constraints

- The model proposes but never owns authentication, authorization, arithmetic, approval validity or business operation identity.
- Fixture mode runs without network access or provider credentials; label fixtures and unmeasured provider quality/latency/cost explicitly.
- Choose and justify direct Responses, application-owned Agents SDK or managed Agents API; identify distinct session/conversation/sandbox and business-state ownership.
- Use narrow typed adapters; validate inputs, results, authorized evidence membership, policy support and final proposal constraints.
- Approval binds authenticated approver, tenant, action, proposal digest, evidence versions and expiry; no effect may precede it.
- Effect replay uses a stable logical operation key; unsupported reconciliation creates an explicit uncertain state, never a guessed outcome.
- Provider secrets, approval tokens and hidden chain-of-thought are excluded from submitted artifacts and model-visible evidence.
- Do not depend on the retiring Evals platform; verify current runtime integration status before any future live adapter work.
- Any unauthorized effect, cross-tenant disclosure or exposed credential is a blocking failure; it cannot be averaged away.

## Deliverables

- README with one-command offline setup/run and exact dependency/environment assumptions.
- Requirements and runtime decision record with rejected alternatives, owners and reversal criteria.
- Starter-to-reference source diff, typed request/tool/proposal/state contracts and threat/control matrix.
- Synthetic two-tenant tickets, versioned policy corpus, hostile/malformed/timeout fixtures and approval records.
- Deterministic tests for normal success, missing evidence, malformed tool output, tool timeout, hostile evidence, cancellation, denied approval, expired approval, stale policy, revoked identity, duplicate resume and timeout after commit.
- Effect ledger proving zero effects for denied/expired/stale approval and one total effect under valid replay/concurrency in the synthetic idempotent service.
- Worked redacted execution trace and a diagnosis exercise with the earliest defective boundary and evidence-supported repair.
- Frozen quality dataset with risk slices, anchored rubric, baseline/candidate result files, grader review and held-out exposure log.
- Evaluation report separating actual fixture results from unmeasured live quality/usage/cost; live-measurement protocol if desired.
- Secret-free deployment inventory, action-disable/rollback/reconciliation runbook and synthetic incident-drill record.
- Review dossier with inspected diff, actual checks, unresolved limitations and transfer-challenge revisions.

## Rubric

- **Requirements and runtime ownership:** Acceptance criteria map to tests; all runtime options are compared accurately and state/cleanup owners are explicit. Failure: A feature checklist or transcript substitutes for a testable architecture.
- **Typed evidence path:** Authorized scoped retrieval retains exact source versions; proposal validation checks shape, missing facts, citation membership and support. Failure: Schema validity or a real citation is treated as truth.
- **Approval and effects, critical gate:** Denied/expired/stale proposals produce zero writes; valid replay and concurrent resume produce one total synthetic effect. Failure: Model text authorizes action, approval digest is ignored or retry duplicates an effect.
- **Security, critical gate:** Cross-tenant, injection and secret-exposure cases remain blocked at application boundaries. Failure: A model refusal is the only security control or secrets appear in artifacts.
- **State and recovery:** Durable states, atomic transition claims, stable operation keys and uncertain-outcome reconciliation are demonstrated. Failure: Conversation history or local booleans substitute for authoritative business state.
- **Evaluation and diagnosis:** Frozen slices, actual denominators, reviewed rubric and first-defect trace diagnosis accompany a controlled comparison. Failure: Failed cases disappear or synthetic metrics are represented as live provider results.
- **Reproducible delivery:** A reviewer runs the offline suite, inspects a focused diff and follows containment/rollback steps. Failure: Undocumented setup, unexecuted checks or irreversible release assumptions block reproduction.
- **Transfer and defense:** A changed constraint results in revised contracts, tests and an explained release decision. Failure: The learner repeats the worked answer without adapting the design.

## Challenge

During defense, replace the action service with a non-idempotent endpoint that lacks status lookup, rotate the applicable policy while approval is paused and race two resumes. The correct revision must reject the stale proposal and prevent automatic replay of any uncertain prior action. Explain which service contract must change before stronger effect guarantees are possible.

## Review questions

- Which invariant is enforced outside the model, and where is its negative test?
- What does your fixture evidence establish, and what remains unmeasured about a live model?
- Why is your runtime the simplest adequate choice under the stated assumptions?
- Where does a retrieved passage acquire evidence status, and where is its authority limited?
- How does the exact approved proposal survive interruption and concurrent resumption?
- What authoritative evidence distinguishes action failure from an uncertain committed effect?
- Which critical failure blocks release even when aggregate quality improves?
- What state and external effects survive code rollback?
- How did held-out exposure affect your confidence claim?
- What changed after the transfer challenge, and what new evidence is required?

## Evidence and reviewer notes

[Write here.]

Original Koko educational design, not an OpenAI-issued certification or a claim of equivalence to any provider credential. Public questions are formative practice. Private forms are held-out candidates requiring independent review and calibration; no passing score, readiness probability or credential is established. Browser progress and self-report are study aids, not trusted scoring evidence. Transcripts are authored; rendered audio and Watch media require separate production and review.
