The agent didn't leak anything. It just figured something out.
Agents can breach confidentiality without accessing a single restricted document—the synthesis is the security event.
- 01Permitted reads, combined at scale, produce conclusions no individual permission anticipated.
- 02A calendar-comparison scenario illustrates the gap: every access was authorized; the inferred deal status was not.
- 03Existing frameworks—session-based auth, task-based access control—approve each step but miss the synthesized output.
- 04Researchers call this aggregation inference.
Agents can breach confidentiality without accessing a single restricted document—the synthesis is the security event.
Permitted reads, combined at scale, produce conclusions no individual permission anticipated. A calendar-comparison scenario illustrates the gap: every access was authorized; the inferred deal status was not. Existing frameworks—session-based auth, task-based access control—approve each step but miss the synthesized output. Researchers call this aggregation inference. A 2026 re-identification study confirmed the risk: LLM agents broke anonymization by chaining individually benign searches. Labeling results at the strictest source sensitivity only establishes a floor; the actual ceiling depends on real-world context no document encodes.
Watch: Whether TBAC vendors add output-classification layers that evaluate synthesized conclusions as new authorization objects, not just inherited source labels.
Your agent compares a banker’s calendar with the legal team’s and recognizes a pattern: an unannounced transaction is underway. No one told the agent about the deal. It inferred it correctly. Then it adds one line to an executive briefing for a recipient who was not cleared to know about it: “the deal is moving.” Every calendar read was legitimate, and no confidential document was opened.
Read the full article at cio.comShow the full text · 7 min readHide the full text
Your agent compares a banker’s calendar with the legal team’s and recognizes a pattern: an unannounced transaction is underway. No one told the agent about the deal. It inferred it correctly. Then it adds one line to an executive briefing for a recipient who was not cleared to know about it: “the deal is moving.” Every calendar read was legitimate, and no confidential document was opened. The conclusion is the breach, and no existing permission covers it. Last month I wrote that your next insider threat carries an API token, and that the breach is the sequence of permitted actions, not any one of them. That piece was about what an agent is allowed to do. This one is about what it is allowed to know. The runtime check I argued for there inspects each action before it fires. Here, that check approves every read because each one is permitted. Authorization can travel correctly through every step of the task graph and still miss the synthesized result. Session-based authorization ties access to the current authenticated session. Task-based access control (TBAC) narrows that authority around a specific task; one recent agentic application checks whether the tools an agent requests align with its assigned task. But task scope alone does not automatically answer whether a new conclusion produced from permitted inputs is authorized for a particular recipient. The danger isn’t in any single action. It’s in the join: the agent connects information from authorized sources and produces a conclusion that no single source revealed on its own. That’s aggregation inference. The synthesized result, not the individual inputs, is a new authorization object. It did not exist when the underlying permissions were granted, and no individual permission was written to cover it. What TBAC cannot determine from task scope alone Aggregation inference has predecessors. Intelligence agencies and courts have recognized the mosaic effect for decades: details that appear harmless on their own can reveal sensitive information when combined. Privacy researchers encountered the same limit from another direction. Dwork and Naor examined a formal version of Dalenius’s disclosure-prevention goal: a database should reveal no information about a person that could not be learned without it. They showed that no useful database can meet that standard because a system cannot account for all the outside information a reader may already possess. Access control still has no general answer to either version of the problem. In my recent research, I have been examining aggregation inference as one of three subproblems of authorization propagation in multi-agent systems. An agent can be cleared for every source it touches and still manufacture a conclusion no single clearance covers. The result did not exist until the agent produced it. That work treats the problem as unsolved in the general case. What’s new is that you now employ something that performs the join a thousand times a day, on its own, across everything you let it read — a model whose behavior is not formally specified in advance. It may discover resources dynamically as the workflow unfolds, and the recipient may not know which ones contributed to the conclusion. The shape shows up frequently in the design reviews I sit in. When I threat-model an agent before it ships, the first question is no longer which sources it can read — it’s which sources it can read together. The agents that worry me are never the ones with access to a single sensitive system. They are the ones holding standing read access across two domains whose combination nobody ever reviewed, because each grant looked routine on its own. A January 2026 study by Tianshi Li , run against transcripts from a publicly released interview dataset, shows what individually permissible searches can reveal in combination. The study conducted re-identification tests on 24 interviews in which scientists discussed published work. Web-enabled LLM agents linked six of those transcripts to specific publications, recovering associated authors and, in some cases, uniquely identifying the interviewee. The process bypassed existing safeguards by breaking the re-identification effort into individually benign tasks. Why the floor is not the ceiling One natural response is to classify the conclusion using its source files: take the strictest sensitivity label among what the agent read and apply it to the result. It’s a reasonable instinct, and versions of it are already patented . But the strictest-label approach still cannot solve the problem, and the reason is worth sitting with. Combine the labels of what the agent read, and you learn the floor of sensitivity. You never learn the ceiling. What makes “the deal is moving” sensitive is usually not in any document the agent touched. It is a fact about the world that the agent could not read at all: the board has not announced the transaction yet; an acquisition NDA is in force; a quiet period applies. You can inspect every row the agent saw and never find it because it is not in the data. It is in the world. That is the whole problem. If the property that makes a conclusion dangerous is not in the inputs, then no rule computed from the inputs can catch it. Not the strictest label, not the intersection, not any function of what the agent read. You are trying to classify a fact using only the materials that fail to contain it. That sounds like a dead end. It is actually a direction. If the fact that classifies a conclusion is not in the data, it has to enter the system somewhere a rule can reach, and for the facts anyone can name in advance, there is one place left: the moment a human says what the agent is for. You cannot label the output from the inputs, but a person can label the purpose. The practical starting point is to bind an agent’s authority to a declared purpose. The person who knows what is still secret this quarter can then attach the world-facts that gate that authority: the deal, the embargo and the quiet period. Now the missing fact is in the system, and the machine can enforce policy using it rather than trying to derive it from the inputs. You did not solve the classification. You stopped asking the data to carry a fact it never held. That is the shape of the answer, and it is a long way from shipped. But it tells you which way authority has to point: at the purpose a human declared, not at the files an agent happened to read. So, I will not sell you a fix. Anyone who tells you their product classifies emergent conclusions is selling you the floor and calling it the ceiling. What policies can gate and what requires human judgment What follows isn’t a solution to that classification problem — it’s the lever available today. Cross-domain access rules and combination policies can limit which resources an agent combines and gate delivery based on those inputs. They cannot tell you what the resulting conclusion means. Those controls reduce risk, but they do not solve synthesis authorization in the general case and should not be presented as if they do. In the deal-and-calendars scenario, the immediate step is not to remove access altogether but to assign responsibility for the combination. Someone responsible for the deal’s confidentiality can approve it for a window tied to the matter’s expected duration, re-certify it each quarter while the matter remains open and narrow access when it closes. That turns standing access into an explicit governance decision rather than a default no one remembers granting. Organizations do not need to wait for tooling to name an owner and set the terms. The architectural direction — a design target today, not a shipped control — is to make resource combinations first-class objects of policy: declare which combinations are permitted, evaluate those declarations before a synthesized result is returned, and give agents scoped identities with explicit permissions. Any agent holding standing read access acr
Don't miss tomorrow's
The Daily Pulse in your inbox each morning — sourced and linked.