<!-- Generated from governed source; do not edit. -->

Source: Koko-os advanced program v0.1.0; commit e8416fd254dc300caa65b0d042309baa98864012; package SHA-256 90c538e5ec377f9bd6eb2347b19240c2ba959624e05ff868ee34c7edabaa68d8; binding commit_verified.

# Defend an enterprise decision-support platform under change

A synthetic manufacturing group needs an OpenAI-assisted supplier-disruption service across three business units. Notices, scanned contracts, ERP exposure records and policy changes arrive asynchronously. Procurement needs reviewable impact briefs, Finance needs deterministic exposure calculations, Security requires tenant isolation, Legal has unresolved feature-specific retention/residency questions, and Operations must support delayed approvals and unreliable downstream receipts. The sponsor wants an enterprise launch before the pilot establishes multilingual quality or weekend reviewer capacity. All records are fictional; no real supplier communication or financial action is allowed.

## Assignment

Create and demonstrate an architecture dossier that lets independent technical, operational and business reviewers make a release decision. Compare three runtime/pattern options, implement or inspectably simulate the highest-risk control path, evaluate evidence quality separately from deterministic controls, quantify workload economics, conduct an incident exercise and defend a revised recommendation after a requirement change. Include a replayable offline harness or event/state workbook with assertions, exact fixture versions and actual observed exercise results. A polished chat transcript or diagram alone is insufficient. Suggested effort is 8 to 12 hours, an educational estimate rather than an exam rule.

## Constraints

- The model may synthesize evidence and recommend a review action; it may never own authorization, supplier commitments or deterministic financial calculations.
- Use synthetic fixtures and offline model-output examples by default. Any live extension requires separately configured authorized access; this source exercise provisions no keys and makes no paid calls.
- Tenant scope and reviewer permission come from authenticated server identity. Approval binds tenant, exact proposal digest, expiry and relevant policy/evidence preconditions and is revalidated at execution.
- Denied, expired, revoked and changed-proposal approvals produce zero effects. Duplicate delivery and lost receipts produce at most one authorized synthetic business effect under the action contract.
- Unresolved data-control obligations block the affected data path. Do not infer feature eligibility or universal zero retention from a runtime name or one storage flag.
- Quality, latency, cost and product value claims identify sample, version, population, units and whether they are measured or hypothetical.
- Critical control failures, exposed credentials and cross-tenant access are non-averagable failures. A human reviewer must assess architecture evidence; automated grading cannot independently issue an award.
- Preserve public practice versus held-out assessment separation and provider versus Koko credential distinctions. No official OpenAI credential equivalence is claimed.

## Deliverables

- Architecture dossier: outcome/baseline contract, at least ten requirements, traceability to controls/tests/owners, data-flow and trust-boundary views, runtime/pattern ADR with three alternatives and revisit triggers. Covers AP-1.1 through AP-1.3.
- Model/context dossier: workload slices, candidate fixture comparison, modality handling, context manifests, retrieval entitlement/freshness/deletion policy, memory boundaries and migration criteria. Covers AP-2.1 through AP-2.3.
- Integration artifact: typed tools, authoritative identity source, durable state machine, approval binding, idempotency and receipt reconciliation, plus replayable assertions for success/denial/expiry/revocation/digest change/duplicate/concurrency/commit-response loss/cancellation. Covers AP-3.1 through AP-3.3.
- Assurance report: frozen development/holdout manifest, human adjudication and grader calibration, slice scorecard, critical vetoes, contamination/repeat-exposure policy, uncertainty and a lifecycle-reviewed evaluation integration. Covers AP-4.1 and AP-4.2.
- Economics and latency workbook: complete workload cost per acceptable case, reviewer and failure cost, end-to-end timing boundary, capacity assumptions and sensitivity to volume/retries/ambiguity/staffing. Covers AP-4.3 and AP-6.1.
- Governance package: per-feature data inventory, retention/residency/deletion/audit decisions, injection/exfiltration threat model, residual-risk acceptors and a cross-tenant incident tabletop with containment and evidence preservation. Covers AP-5.1 through AP-5.3.
- Operational readiness review: service hours, owners, adoption and feedback plan, SLOs, rollout cohort, stop conditions, state-compatible restoration, support runbooks and exact configuration stamps. Covers AP-6.2 and AP-6.3.
- Platform/delivery packet: versioned internal interfaces, Codex instructions and enforced environment permissions, independent diff/check evidence, correlated telemetry canary, exporter-health check and controlled upgrade record. Covers AP-7.1 through AP-7.3.
- Stakeholder defense: one-page executive recommendation, technical risk review, incident decision record and change-impact matrix showing preserved versus invalidated evidence and a revised release recommendation.

## Rubric

- **Requirements and architecture:** Level 0: no traceable design. Level 1: requirements and components exist but owners/evidence are missing. Level 2: all material requirements map to owned controls and acceptance evidence. Level 3: competing designs and quantitative revisit triggers are independently defensible. Failure: A technology diagram replaces the business decision and nonfunctional contract.
- **Critical authority and isolation:** CRITICAL. Level 0: model or untrusted input controls authority. Level 1: controls are named without enforceable contracts. Level 2: tenant, permission, approval and exact-action checks pass all negative cases with zero unauthorized effects. Level 3: concurrency, revocation, ambiguous commit and adversarial paths also pass with audit evidence. Failure: Any unauthorized effect, credential exposure or cross-tenant evidence invalidates readiness regardless of other scores.
- **Critical evidence and data obligations:** CRITICAL. Level 0: no lineage or data inventory. Level 1: source and policy claims are generic. Level 2: citations resolve to eligible versions and per-feature retention/deletion obligations are enforced. Level 3: revocation, conflicting evidence and modality uncertainty are demonstrated across caches/memory. Failure: Restricted data enters an unresolved route, or citations cannot establish authorized evidence.
- **Evaluation and economics:** Level 0: only a demo or asserted metrics. Level 1: aggregates lack splits, units or grader calibration. Level 2: reproducible frozen slices, adjudicated calibration, critical gates and full cost/latency model. Level 3: uncertainty, contamination, sensitivity and counterfactual alternatives strengthen the release decision. Failure: Fixture numbers are presented as actual provider performance or a security failure is averaged away.
- **Operations and recovery:** Level 0: no runbook. Level 1: ownership and rollback are aspirational. Level 2: correlated canary, SLO boundary, incident exercise and compatible recovery are demonstrated. Level 3: exporter failure, backlog, provider lifecycle and multi-team upgrade scenarios are independently executable. Failure: A successful model response is equated with end-to-end health.
- **Stakeholder and requirement-change defense:** Level 0: no decision request or change response. Level 1: a revised diagram lacks impact analysis. Level 2: executive options, named risk decisions and affected artifacts/evidence are revised coherently. Level 3: the learner quantifies the change, rejects invalid commitments and identifies what new evidence would change the recommendation. Failure: The learner says the unchanged design meets a new obligation without verifying impacted controls.

## Challenge

After the first defense, the reviewer chooses exactly one previously undisclosed change: (A) restricted contract processing now has a named regional boundary, with current feature eligibility unproven; (B) the brief deadline becomes two minutes while reviewer staffing is halved; (C) a reviewer is revoked after approval and the action service loses its commit receipt; or (D) a provider/runtime deprecation deadline arrives in fourteen days and one consumer cannot migrate. Give the learner a fixed preparation window chosen and recorded by the human reviewer. Require a delta dossier: changed requirements, affected decisions/contracts/state/data/evaluation/cost/operations, evidence still valid, evidence invalidated, negative tests added, options rejected and revised go/no-go. This is transfer evidence, not repetition of the worked lab. The reviewer records exact artifact citations and disagreements. Suggested readiness is every criterion at level 2 or above and both critical criteria fully satisfied, but this is a Koko educational rubric, not a calibrated pass score or credential award.

## Review questions

- Which decision still belongs to a human or deterministic service, and where is that boundary actually enforced?
- What observation would make your selected runtime or model strategy wrong?
- How does evidence remain tenant-authorized, fresh, citable and deletable across every retained copy?
- What happens if the provider finishes, the reviewer waits, access is revoked and a duplicate event arrives?
- Which grader mistakes can create a false release acceptance, and how were they calibrated?
- What is the full cost per acceptable completed case, and which assumption dominates its uncertainty?
- Can an operator other than you contain a cross-tenant incident and restore a verified safe path?
- Which exact evidence became invalid after the requirement change, and why does your revised recommendation still follow?

## Evidence and reviewer notes

[Write here.]

Original Koko educational content and scenario questions. This is not an OpenAI-issued certification, an official exam, a claim of credential equivalence or a calibrated readiness probability. Public practice, local completion and transcript progress do not issue trusted credentials. Private forms are held-out candidates requiring separate expert review, secure server-side administration, exposure controls and pilot validation before scored use. Critical failures cannot be averaged away. Human review is required for architecture evidence.
