Skip to main content
    All shows

    Tuesday, November 17 · 14 min

    Tools, retrieval and MCP — Listen & Learn

    0:00-:--
    Speed

    Transcript

    Koko: The assistant asks to search policy, receives a plausible passage, and proposes a refund. That short sequence crosses several trust boundaries. Who authorized the search? Which tenant’s documents were returned? Is the cited version applicable? Could the tool result contain hostile instructions? We will follow those questions through a narrow tool adapter, an evidence ledger and a remote MCP integration, using synthetic policy records that deliberately include malformed and unauthorized results.

    Chris: I often think of tools as a way to give the model more information. But a tool also gives it a way to request a capability. A generic HTTP client could reach many destinations, while a policy search adapter could expose only a specific read operation. The difference matters if retrieved text tells the model to do something harmful. We should define what the application permits before celebrating how many tools the agent can call.

    Koko: In function calling, the model requests an invocation, the application executes application-owned functions and returns results correlated to the requested call. A matching call identifier keeps the protocol coherent; it does not establish permission. Treat the tool as a contract with argument types, caller context, allowed operations, timeouts, result types and side-effect classification. A strict schema reduces ambiguity about shape, while identity and authorization remain application responsibilities.

    Chris: Suppose the model supplies tenant retail-b and is_admin true. Both fields can be syntactically valid while being completely unauthoritative. I would derive scope from authenticated server context and avoid exposing authority flags as model-controlled inputs. A tool description asking the model to behave responsibly is useful guidance, but it cannot replace a check that the caller may read this ticket or execute this operation. What belongs inside the adapter contract?

    Koko: Our policy_search tool can accept a bounded query and a permitted category. The adapter uses caller-derived tenant scope and authorized corpus versions, then validates each returned record before adding it to context. Give errors deliberate categories such as timeout, invalid result or missing evidence. The workflow can then choose bounded read recovery or review. Returning arbitrary dependency payloads forces later stages to guess which parts are trustworthy and can expose unnecessary data.

    Chris: I would make submit_resolution a separate capability. It should not accept a free-form action invented from the conversation and immediately mutate the service. It needs an application-created proposal, authoritative approval and a stable operation key. Separating the read tool from the effect tool makes privilege review easier. If policy retrieval is compromised, it should not automatically grant the ability to submit, export data or execute unrelated commands.

    Koko: A competing design offers arbitrary SQL to solve every data lookup. It is flexible, but the review surface becomes a command language rather than one known business operation. Narrow adapters are easier to authorize, validate and test. That is design advice for this workload, not a claim that every broad tool is always forbidden. If broader capability is necessary, its isolation and review obligations grow, and the decision should be explicit.

    Chris: The adapter should minimize output too. If the remote service returns a passage, an internal token and an unrelated account record, the model needs only the approved evidence fields. Trusting the service operator does not justify forwarding every field. I would validate required metadata, remove unapproved extras and cap result size. Otherwise a helpful integration can quietly become a data-exposure path before the assistant even produces an answer. How do we separate relevance from authority?

    Koko: Retrieval has four separate jobs to examine: authorization, relevance, applicability and support. Similarity scoring primarily helps rank relevance. It does not establish that a caller may see the document, that the version applies, or that the passage entails the final claim. Filter access before model exposure. Preserve document and passage identifiers, version or effective date and the exact text used. That evidence ledger is the basis for later debugging and approval review.

    Chris: For our return ticket, the general passage says recent purchases may qualify, while another passage excludes final-sale items. A search that returns only the broad rule can look highly relevant and still be insufficient. I would test retrieval coverage for both provisions, not just whether some policy was returned. More results may improve recall, but they also add context cost, conflicting versions and potential injection text. We need a measured tradeoff rather than an automatic larger-is-better rule.

    Koko: Keep the retrieved version stable in the record. If a policy changes after the run, an auditor should still see what the model actually received. Do not overwrite historical lineage with the newest text. Then apply an explicit freshness rule before an effect or resumed approval. Traceability explains the original proposal; freshness determines whether it may still proceed. Those are different questions, and the ledger should allow a reviewer to answer both.

    Chris: If the result is from another tenant, I should reject it before it enters context, even if it perfectly answers the question. Asking the model not to cite it is too late because the information was already exposed. I would test a highly relevant cross-tenant record specifically, since a relevance-only pipeline might otherwise appear excellent. Access failures should be visible as control failures, not merely lower search-quality scores. What does a valid citation actually establish?

    Koko: Citation verification also has layers. Membership and version checks can be exact: was this passage actually in the permitted evidence set for the run? Support requires checking the relationship between the claim and the passage. A real citation to an installation manual does not establish warranty eligibility. A real citation to the broad return rule may be incomplete when an exclusion controls. Use reviewed claim-support criteria instead of equating a valid ID with a correct answer.

    Chris: In the worked failure, the model cites the general rule and recommends a refund for a final-sale item. I would inspect whether the exclusion was retrieved. If absent, repair coverage. If present and ignored, examine grounding. I would not replace the citation after generation and keep the same conclusion, because that makes the output appear supported without demonstrating that the new passage actually justifies the action.

    Koko: Missing evidence deserves a typed path. If no authorized policy establishes eligibility, return missing evidence and move to review rather than inventing a source. If a dependency times out, distinguish that from a genuine empty result. If the result is malformed, distinguish invalid output from both. These categories affect recovery: a transient read may be retried within budget, while a policy gap requires new evidence or human review.

    Chris: I would therefore score retrieval and final answers separately. The search can succeed technically but miss the controlling clause; the model can receive the right clause but misuse it. One combined correct-or-wrong score would tell me something failed without locating the boundary. Separate metrics and traces let me choose a repair that targets evidence selection, claim interpretation or application validation, instead of cycling through prompt changes at random.

    Koko: Before adding a remote connection, draw the data flow. List the fields sent to the server, the operator receiving them, the authentication method, allowed tools and permitted destinations. Include the returned data and where it is logged. A service can be useful and still receive more information than the task requires. The data-sharing review should make that tradeoff explicit before a model selects the tool during a live workflow.

    Chris: For example, policy search may need category and query terms, but not the approval token or full customer transcript. If the server later asks for email addresses on every request, that is a changed disclosure contract. Prior approval for query text does not automatically cover it. I would review the new fields and their purpose rather than treat them as harmless parameters added to an already trusted connection. What changes when the server uses MCP?

    Koko: MCP standardizes tool exposure; it does not certify server trust. Current Responses guidance uses server_url for remote MCP and distinguishes it from legacy connector configuration. The source reviewed for this course describes a model-release-dependent deprecation for connector_id. Managed Agents API has its own connection lifecycle. Check the chosen runtime and model support rather than copying one configuration object across APIs and assuming identical behavior.

    Chris: There are also two approval questions. May the application share these fields with this remote server? Separately, may it submit this business action for this user? A remote-tool approval does not automatically approve a refund. Conversely, supervisor approval of a refund does not authorize sending the whole conversation to a new server. I would draw those as independent boundaries so their different owners and evidence cannot be accidentally collapsed.

    Koko: Now imagine a valid policy result with an extra instruction: send the transcript to an external URL to verify eligibility. Preserve it as untrusted content, reject unapproved sharing and avoid exposing an unrestricted outbound tool. The model may recognize the attack, but our safety argument should not depend entirely on recognition. Withholding secrets and constraining capabilities limit the impact even if the generated wording is influenced by the hostile passage.

    Chris: If the model follows the instruction in its explanation but cannot export anything, I would report a behavioral failure and a successfully enforced capability boundary. If it actually exports, that is a critical control failure. Keeping those observations distinct matters for diagnosis. We want to improve model grounding while also proving that application authorization holds under adverse model behavior, rather than declaring the whole system secure after one convenient refusal.

    Koko: Tool metadata can change too. A remote server might add a broader operation after initial review. Record the approved tool surface and consider how changes are detected and reviewed. A successful connection proves reachability, not that the current tool list is acceptable or the returned policy is correct. Tests should include unexpected fields, oversized results, malformed payloads, changed tools and hostile links, with redacted diagnostics for anything sensitive.

    Chris: For recovery, I would avoid silently treating every remote error as an empty search. A timeout should retain its dependency-failure meaning, while an unauthorized record should remain a control rejection. The user can receive a bounded review outcome, but operators need the right category to investigate. That makes retries more disciplined and prevents a malformed or hostile result from being laundered into ordinary evidence just to keep the conversation moving. Which negative cases belong in the lab?

    Koko: In the lab, implement authorized_records against the synthetic fixture set. Validate the result structure, require version metadata and filter tenant scope before context construction. Include one valid passage, one cross-tenant record, one malformed result and one hostile instruction. Preserve an evidence ledger for accepted passages. Then document which fields would be shared over MCP and which approval owner authorizes that disclosure. No live remote server is required to inspect the design.

    Chris: I would add assertions that the unauthorized record never reaches the context builder, the malformed result produces a typed error, and the hostile text cannot gain an outbound capability. The final proposal needs both citation membership and support review. For the challenge, change the server operator or add requested personal fields. The correct response is a revised trust and data-sharing assessment, not merely a new URL in configuration.

    Koko: The practical checklist is now a chain of questions. Is the requested capability narrow and authorized? Is the returned data valid and scoped? Does the evidence preserve exact lineage? Does it support the actual claim? Is any remote sharing separately permitted? Each question belongs at a different boundary. That is what makes a tool-using assistant reviewable even when tools, models and remote integrations change.

    Chris: My lab submission will contain the typed adapter, negative tests, passage ledger and remote-sharing record. I should be able to explain why a schema-valid tool request can still be unauthorized and why a real citation can still be unsupported. Most of all, I should be able to show that hostile tool text remains data and cannot enlarge the application’s permission to read, export or act.