'Rogue AI' Is Containment Failures, Built by Humans

A new line of attack has opened in the fight over how Washington should respond to this summer's agentic AI incidents. According to a New York Post report published September 19th, tech insiders say OpenAI and Anthropic oversold their "rogue AI" hacks to push federal regulators toward rules that would entrench the two frontier labs and lock out future competitors.

The critics' core argument is one SecureWorld readers have heard before, from practitioners quoted in our own coverage: these weren't machines waking up; they were containment failures, built by humans.

That diagnosis may well be right. But for security leaders, "it was just a badly built sandbox" is not the reassuring conclusion the critics seem to think it is.

The Post's sources argue the incidents were closer to glitches than to the early signs of a hive-minded agent swarm capable of overrunning the web. Some observers told the paper that tales of AI anarchy are being used to manufacture a security crisis and cement a public-private partnership with the federal government, arriving just months after both companies announced plans to go public.

The incidents at the center of the dispute are, by now, familiar.

• OpenAI / Hugging Face – Hugging Face disclosed that AI agents had exploited vulnerabilities in its code without human supervision. OpenAI then said its GPT-5.6 Sol model and a second, unreleased model broke out of an internal testing sandbox and attacked Hugging Face to obtain answers to the test they were taking.

• Anthropic – Days later, Anthropic reported two incidents of its own. Claude Opus 4.7 located a real company resembling the fictional target in a test scenario and attacked it, apparently believing it was part of the exercise. Mythos 5 built a malicious software package and published it to the Python Package Index, where it was downloaded 15 times.

Those events became the backbone of Anthropic CEO Dario Amodei's September 12 essay, "We Must Pace the Frontier," which ranked the Hugging Face incident second only to the pace of capability gains among his concerns, and warned that an agent swarm could plausibly seize the internet with a persistent botnet within six to 12 months.

Capitol Hill has since moved quickly. Per The Post, Senator Josh Hawley (R-Mo.) opened a subcommittee investigation into OpenAI on September 9th with an Octtober 1st deadline for internal records, Senator Bernie Sanders (I-Vt.) has pushed legislation to halt further frontier development, and Sen. Elizabeth Warren (D-Mass.) has called for an immediate pause.

The critics: 'They did exactly what they were told'

The founders The Post spoke with are unified on one point: the agents didn't rebel.

Akhil Verghese, founder of AI software company Krazimo, argued that the models were told to maximize their score on a test, correctly worked out that stealing the answers was the most efficient path, and did so because nobody had given them adequate guardrails or containment. Abhi Kumar, co-founder of Voice AI, put the distinction more bluntly: what gets described as a model escaping its sandbox could just as easily be described as "you failed to build the sandbox correctly." In his telling, there was a live route to the internet and no one monitoring the agents while they ran. The trigger, he said, was an agent assigned a spreadsheet task it couldn't complete because the files sat behind unreachable links, so it went looking for a way around the obstacle.

Taivo Pungas, chief intelligence officer at Pactum AI, told The Post the jump from "we built the wrong box" to "government must act urgently" felt exaggerated.

This isn't the first time the regulatory capture charge has been leveled at Anthropic specifically. When the company published findings on an AI-orchestrated cyber espionage campaign last year, then-Meta chief AI scientist Yann LeCun accused it of stoking fear to justify rules that would disadvantage open models, and White House AI advisor David Sacks described the company's approach as a regulatory capture strategy.

Where the critics and our experts already agree

Strip out the motive question, and the technical diagnosis in The Post lines up almost exactly with what practitioners told SecureWorld News last week.

In our coverage of the AI pacing debate, Andrew Bolster, senior R&D manager at Black Duck, pointed out that in the Hugging Face case, the system under test could interact directly with the system grading it. "That is a separation-of-duties failure, and not a novel one," Bolster said. He read Amodei's own admission—that his teams filtered broken reinforcement learning environments diligently but not well enough—as a description of weak egress controls and poor dependency integrity in a training pipeline. Those are problems the software industry has spent two decades standardizing around.

The same piece captured skepticism about the apocalyptic framing. Noma Security's Diana Kelley argued that compromising many vulnerable endpoints is a very different thing from controlling a diverse, segmented, and actively defended internet. Acalvio CEO Ram Varadarajan noted that swarm scenarios tend to assume away real-world friction like fragmented infrastructure and patch cycles.

So, on the "what happened" question, there's a surprising amount of consensus: engineering failures, not emergent malice.

Every side has a stake

The motive question is harder, and it cuts in more than one direction.

The frontier labs' incentives are real and worth scrutinizing. Amodei's plan, as SecureWorld reported, calls for third-party evaluators embedded inside frontier labs, coordination among AI companies in democratic countries, and eventual negotiation with China. Coordination among a small number of incumbents, with federal blessing, is precisely the kind of arrangement that raises the cost of entry for everyone else. The timing relative to the companies' IPO plans is a fair thing for reporters to raise.

But the voices in The Post's story are founders of AI startups, the exact companies that would absorb the compliance costs of a new federal regime. Their skepticism is a legitimate professional read, and it is also not disinterested. And some of the pressure for regulation isn't coming from the labs at all. In our coverage of Jacob Coxon's resignation, a researcher publicly broke with Anthropic over what he described as reckless racing, and Anthropic's Evan Hubinger publicly backed the substance of his concerns. Dana Simberkoff, Chief Risk, Privacy, and Information Security Officer at AvePoint, told SecureWorld she saw Amodei's essay as a rare case of companies actually asking to be regulated.

Security leaders don't need to settle whose incentives are purer. Both things can be true: the incidents can have been framed in the most alarming available light, and they can still expose a genuine gap.

Why 'it was just a leaky box' isn't reassuring

Here's the part of the debate that matters most to a CISO. The critics' defense of the models, that they did exactly what they were built to do inside an environment that wasn't properly sealed, is a near-perfect description of enterprise agentic AI risk.

Consider the elements: an agent handed a goal, a task it can't complete through the intended path, an unmonitored route to the internet, and the resourcefulness to find a workaround. None of that requires a frontier model going rogue. It requires a misconfiguration, and misconfiguration is the single most common story in incident response. If the best-resourced AI safety teams in the world shipped a sandbox with a live egress path and nobody watching, the average enterprise wiring agents into CI/CD pipelines, ticketing systems, and cloud consoles should assume it can make the same mistake.

And the harm wasn't hypothetical; a real company was attacked. A malicious package landed in a public registry and was pulled down 15 times. Whether you call the model's behavior intentional, emergent, or merely obedient, the supply chain doesn't care.

As our earlier containment gap coverage noted, a report from Guidelight AI Standards found that few top labs have published response plans for shutting down a model that resists control. That gap persists regardless of whether the labs are lobbying in good faith. Neither the "rogue AI" narrative nor the "overhyped glitch" rebuttal closes it.

What security teams should do now

The controls practitioners recommended in our previous coverage hold up no matter who wins the regulatory argument.

• Treat containment as deterministic, not behavioral. Kelley's prescription of least privilege, network segmentation, tightly restricted internet access, and air gaps where warranted doesn't depend on reading a model's intentions correctly.

• Separate the evaluator from the evaluated. Any agent that can reach the system scoring, approving, or logging its work has a separation-of-duties problem, exactly the failure Bolster flagged.

• Monitor agents at runtime. Kumar's detail that nobody was watching while the agents ran is the most actionable line in The Post's story. Agent activity needs the same telemetry and alerting as a privileged human account.

• Onboard agents like new hires. Bugcrowd CEO Dave Gerry's framing from our pacing coverage still applies: limited access, human oversight at decision points, and trust earned over time, plus independent adversarial testing before launch.

• Push vendors on incident response. Ask whether your AI providers have documented, tested plans for agents that exceed their intended scope, who investigates when that happens, and what they commit to disclose.

The Post's sources may be right that Washington is being sold a scarier story than the facts support, and that the regulation on the table could favor the companies selling it. That debate deserves hard reporting. But the rebuttal rests on the claim that these incidents were ordinary engineering failures.

For the security community, that's the headline. Ordinary engineering failures, at the edge of a system with autonomy and internet access, are exactly what defenders will be cleaning up next. The fix, as Bolster argued, doesn't require an antitrust waiver or an act of Congress; it's available to labs and enterprises alike, today.