Anthropic CEO Dario Amodei has a message for anyone who thinks AI is just another tech trend: think again. In an essay published Wednesday, Amodei warned that increasingly powerful AI systems cannot be safely entrusted to governments or corporations alone. Both, he argues, need meaningful checks and balances as artificial intelligence becomes more capable.

Amodei didn't mince words. He said governments should have the authority to block or reverse deployments of advanced AI models that pose unacceptable risks—especially those related to cybersecurity, biological weapons, and loss of control of AI systems. The idea is simple: if an AI model is too dangerous, hit the brakes.

But here's where it gets interesting. Amodei also argued that AI companies themselves could eventually develop what he called “quasi-state characteristics,” becoming powerful enough to influence society in ways traditionally reserved for governments. In his words, advanced AI may become “so powerful that it cannot safely be fully entrusted to either governments or companies.” That's a big deal coming from the CEO of one of the leading AI labs.

Why Transparency Is No Longer Enough

For years, the AI industry has leaned on transparency as its primary safeguard. Show what you're doing, and everything will be fine. Amodei says that's no longer sufficient as frontier models become increasingly capable.

He called for “more serious and binding regulation of AI” and compared advanced AI systems to airplanes and pharmaceuticals. You wouldn't let a plane fly without rigorous testing, right? Same logic applies here. Amodei proposed mandatory third-party testing before deployment, with the power to block or reverse releases that don't meet high safety standards. “Their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety,” he wrote.

The proposed evaluations would focus on risks related to cybersecurity, biological weapons, loss of control, and automated research capabilities. He also emphasized that AI developers should maintain strong security standards, conduct regular testing, and promptly report major safety incidents.

Preparing for AI-Driven Job Loss

Safety isn't the only concern. Amodei warned that AI could create larger and more persistent labor-market disruptions than previous technological shifts. “It's reasonable to think that AI could produce much larger disruptions to the labor market than previous technologies,” he wrote.

To address those risks, he proposed measures including wage insurance, worker-retention incentives, workforce training programs, and expanded monitoring of AI's impact on employment. He also suggested that broader income-support mechanisms could eventually become necessary if displacement becomes widespread. The goal, he argued, is to ensure the benefits of AI-driven economic growth are shared broadly rather than concentrated among a small group of companies and investors.

Who Should Control Powerful AI?

The essay also tackled the thorny issue of AI's impact on civil liberties, political power, and national security. Amodei warned that advanced AI could become a tool of surveillance and authoritarian control if governments gain unchecked access to increasingly capable systems. He called for stronger protections against domestic use of fully autonomous weapons.

But he didn't let private companies off the hook. He argued that firms should not hold unchecked influence over increasingly powerful AI systems, warning that some could eventually develop “quasi-state characteristics.” On geopolitics, Amodei described AI as a technology comparable in strategic significance to nuclear weapons, arguing that it could become a dominant source of military and economic power. He called for closer coordination among democratic nations on AI policy, safety standards, and supply-chain controls.

Mythos Sparks Concern

The debate around Mythos—Anthropic's cybersecurity-focused AI model—follows months of friction between Anthropic and U.S. officials over AI safety and military applications. Earlier this year, the Trump administration moved to phase out Anthropic's technology from federal agencies after the company maintained restrictions on mass surveillance and autonomous weapons.

Anthropic later briefed members of the House Homeland Security Committee on Mythos and its national security implications. Pentagon tech chief Emil Michael described the model as a “separate national security moment” because of its advanced cyber capabilities.

The essay comes days after Anthropic launched Mythos 5, the latest version of its cybersecurity-focused AI model, alongside Claude Fable 5, its newest consumer-facing system. Against that backdrop, Amodei's call for stronger safeguards, mandatory testing, and government authority to block dangerous AI deployments arrives as policymakers and technology companies grapple with increasingly powerful frontier AI systems.

This content was partially produced with the help of AI tools and was reviewed and published by MarketDash editors.

Image via Shutterstock