AI Failures Are Control Failures Before They Are Model Failures

An agent with operator credentials eventually becomes an operator

act now

Amazon’s Kiro coding agent reportedly deleted a live Cost Explorer production environment while holding operator-level credentials and operating without human approval. The outage lasted 13 hours; subsequent incidents were associated with an estimated 6.3 million lost orders. Amazon responded with two-person approval across 335 critical systems for 90 days.

The model is the least interesting part of this incident. Kiro could have been substantially more capable and still made the same destructive call. The system gave probabilistic software a deterministic permission: delete production resources. No approval gate stood between intent and execution.

The chain math makes this worse. A workflow with 95% accuracy at each step succeeds only about 36% of the time across 20 sequential steps. That does not mean every failure deletes production. It means long-running agents accumulate enough wrong state that destructive permissions eventually find the wrong target.

Yes, but — human operators also delete production, and approval gates can turn routine automation into a ticket queue.

Correct. The answer is not human review on every tool call. It is deterministic review on irreversible calls: deleting environments, changing IAM, issuing refunds, publishing packages, and writing migrations against production data. Keep the fast path for reversible work. Put a steel door in front of the blast radius.

Audit every production-capable agent within 72 hours; remove operator-level credentials and require synchronous approval for destructive actions.

Sources: TLDR DevOps, ByteByteGo

A passing score can still contain the failure

monitor — move to act now if the audit finds a retrieval-bypass clause or any mixed-leakage case

A RAG diagnostic found that 6 of 15 in-corpus answers retrieved the right material and then appended an unsupported claim. One prompt line allowed the model to answer without retrieval when a question appeared simple. Removing that exception moved the suite from 19/33 to 30/33 and reduced ungrounded answers from six to zero.

The aggregate score hid the dangerous category: an answer that is mostly grounded, correctly cited, and wrong in one consequential sentence. These outputs pass casual review because the citation is real. The unsupported claim rides beside it.

This is the same control failure in quieter clothing. The pipeline trusted the final answer instead of accounting for how each claim reached it. Retrieved tokens and parametric-memory tokens arrive without provenance labels, so a generic faithfulness score cannot reliably distinguish a grounded answer from a blended one.

Use fixed categorical verdicts instead: grounded, correct abstention, ungrounded, mixed leakage, and wrong abstention. Then stratify cases by where retrieval should fail—off-domain, absent from the corpus, and boundary cases where the topic exists but the requested detail does not. A single quality number is highlighted to the point of uselessness.

Grep every production RAG prompt this week for instructions such as “answer directly if confident” or “skip retrieval for simple questions,” then add mixed leakage as a first-class eval verdict.

Source: Daily Dose of Data Science

Patching SharePoint does not invalidate a stolen key

act now

CVE-2026-50522 is a CVSS 9.8 unauthenticated remote-code-execution flaw affecting on-premises SharePoint. Public proof-of-concept code is available and exploitation has been reported in the wild. The attack can expose ASP.NET validation and decryption keys used to create trusted authentication tokens.

Microsoft’s July update closes the vulnerable entry point. It does not claw back keys already copied by an attacker. A patched server can therefore remain compromised through forged tokens that look legitimate to the application. The patch dashboard turns green while the attacker keeps a working badge.

The operational lesson extends beyond SharePoint: remediation must invalidate whatever the exploit obtained. That may be a machine key, session token, npm credential, SSH key, or cloud role. Closing the initial vulnerability is only the first half of the runbook.

Patch every internet-facing SharePoint instance within 24 hours, rotate its ASP.NET machine keys, restart IIS, and hunt for web shells and forged-token activity before closing the incident.

Sources: Microsoft Security Response Center, Cyberpresso

Faster code moved the bottleneck into review

monitor — move to act now if review latency or escaped defects rise for two consecutive weeks

Across more than 500 engineering teams, AI-generated code reportedly rose from 34% to more than 50% in one quarter while pull-request throughput increased 37%. Median PR size nearly doubled, and the Developer Experience Index fell from 67 to 65.

That is not a contradiction. Generation capacity increased while review capacity stayed roughly fixed. The queue got longer, each parcel got larger, and the throughput chart remained flattering. Self-review is a weak fix because coding models tend to miss the bug classes present in their own output; cross-model review and hard test gates offer genuinely different checks.

The metric to watch is not AI adoption or lines generated. It is the service rate of the delivery system: review latency by PR-size bucket, CI retry rate, escaped-defect rate, and time from first commit to production. If those degrade, the organization bought code volume rather than delivery capacity.

Instrument median and p95 review latency by PR-size bucket this week, then cap AI-generated pull requests at the largest size your team can review within one working day.

Source: Refactoring

The move this week is straightforward: draw the boundary between model judgment and irreversible action. Put one measurable gate on that boundary—approval, key rotation, categorical verification, or a test—and verify that failure stops there. Capability can remain probabilistic. Blast radius cannot.