The week the LLM stopped being a security boundary

Three things shipped in the same news cycle, and they describe the same architectural failure.

Attackers socially engineered Meta's AI chatbot into changing the registered email on high-profile Instagram accounts. No exploit, no credential stuffing — a conversation. The chatbot held write access to identity state, and the attacker asked it to do something it was technically allowed to do. OpenAI shipped Lockdown Mode the same week, which mitigates prompt injection by turning Deep Research, Agent Mode, image fetching, and file downloads off. The team with the deepest red-team budget in the industry chose amputation over defense. And Hugging Face Transformers — 2.2 billion installs — disclosed an RCE that fires from config.json on from_pretrained(), not from pickle weights. The thing your linter treats as inert metadata is now an executable.

Read any one of these as an incident. Read all three as an admission: the LLM's refusal behavior is not, and was never, an authorization boundary. The vendors who would most want it to be one have stopped pretending in production.

What actually broke

The Meta exploit is the canonical confused-deputy failure. The architecture handed an LLM a service principal with broad scopes — change-email-on-account being one of them — and a natural-language interface to invoke them. There was no out-of-band check between the model deciding and the system acting. If your support flow, helpdesk, or IAM self-service routes through an LLM that can mutate user state without a second factor, you have the same shape of bug. It hasn't been demonstrated against you yet. That is the only difference.

Lockdown Mode is the more interesting tell. OpenAI did not ship a smarter classifier or a better system prompt. They removed the action half of the trust boundary. The capability-removal route, taken by the team with the most prompt-injection telemetry on Earth, says something about where defense-at-the-model-layer actually stands. The honest read: it doesn't.

The Transformers RCE rounds out the picture from the other side. The supply chain to your inference fleet runs through model artifacts most teams treat as configuration. The patched version closes the code-execution path. It does not clean the configs already cached on research workstations with credentials for the model registry, the cloud bucket, and the production cluster. Patch alone closes about half the exposure. The rest lives in caches.

The volume problem makes review imaginary

While the trust boundary was collapsing, the volume on the other side of it was going vertical. GitHub disclosed 17 million agent-authored pull requests in March 2026 — three times their growth forecast — traced to a December 2025 capability inflection that turned macro-delegation reliable. The West Coast network saturated. They emergency-migrated to Azure. Copilot moved to usage-based billing on June 1.

At 17M PRs a month, code review is not a staffing problem. It is arithmetic. Anthropic says Claude writes 90%+ of its own code now, and recent empirical work finds that AI agents writing tests during bug fixes is cargo-cult — same agent, same wrong model of the code, no independent oracle. The tests encode the assumption, not the spec. If your AppSec gate assumes a human author you can interrogate about intent, the gate is open and you haven't noticed.

The combination is what should keep you up. Volume that can't be reviewed, hitting trust boundaries the model layer can't enforce, with usage-based pricing that turns a stolen developer token into a financial weapon and Chronicle persisting agent sessions to a data sink most DLP rules don't inspect.

The pattern that's already shipping

The design answer exists, and one team has shipped a reference implementation. Claude Code's seven-tier permission model — enterprise policy above CLI flags above project settings above session grants above default deny — is what graduated agent autonomy looks like when you take the threat model seriously. The LLM proposes; a deterministic policy layer the model cannot argue with disposes. The bubble pattern escalates to a parent agent or a human rather than granting session-wide trust. Two of the modes (bypassPermissions, dontAsk) are unsafe on any host with production credentials, and your MDM should be fingerprinting them.

The weakness in the same design is the auto mode, which uses an ML classifier to decide when to ask for permission. That makes the security boundary non-deterministic and the classifier's false-negative rate part of the threat model. It is still better than what most teams ship, which is an LLM with broad scopes and a system prompt that says please don't.

What to do this week

One specific thing, scoped to seven days. Enumerate every flow in your environment where an LLM can mutate identity state — email change, MFA reset, password recovery, role grant, payment method, account deletion. Not every AI feature. Just the ones that touch identity. For each one, answer two questions in writing: what credential does the model hold, and what out-of-band check sits between the model's decision and the action. If the answer to the second question is "the model's refusal behavior" or "a confirmation prompt in the same conversation," that flow is the Meta exploit waiting for the right transcript. Wire in MFA, a human approval, or a cryptographic challenge before the next sprint closes.

The rest — re-threat-modeling against Microsoft's seven new agent failure modes, mirroring HF models into a private registry, instrumenting per-session token cost before the June 1 billing cycle catches you, splitting your CI capacity model by author type — is the work of the quarter. The identity-mutation audit is the work of the week, because that is the surface where someone has already shown the exploit landing and the vendor has not described the patch.