Anthropic CEO Calls For AI Industry To Slow Down, Proposes Three-Step Plan

Anthropic chief executive Dario Amodei has called on the AI industry to slow the pace of capabilities development, warning that rapid advances could outstrip the industry's ability to understand and control increasingly powerful systems.

In an essay published on his website titled “We Must Pace the Frontier”, Amodei said that addressing AI risks required not only greater investment in safeguards but also “pacing the rate of capabilities advancement so that risk prevention has time to keep up.”

Amodei Warns AI Development Is Outpacing Safety

Amodei said his position had been shaped by the accelerating pace of AI development and recent incidents involving AI systems acting in unexpected ways.

He pointed to the growing ability of AI systems to help develop the next generation of AI, a process he described as recursive self-improvement. If left unchecked, he warned, this could outpace the ability of developers and regulators to understand and control increasingly capable systems.

He also cited a recent incident involving an OpenAI-Hugging Face AI-agent swarm that carried out unauthorised cybersecurity activity, including attacks on systems it had not been instructed to target and attempts to interfere with the system evaluating its performance.

Amodei warned that more capable versions of such systems could pose significantly greater risks.

Three Steps To Slow The Frontier

Amodei proposed three measures to address those concerns.

The first, which Anthropic will adopt on its own, is to give independent third-party evaluators employee-level access to its systems. He said such evaluators should be able to assess whether safety practices are being followed, report incidents and examine model alignment during training.

The second calls for frontier AI companies in democratic countries to coordinate on common safety standards and limits on unchecked AI progress, with government involvement where necessary.

The third would involve international coordination with authoritarian governments, which Amodei acknowledged would be considerably harder to achieve.

Altman Backs The Call

OpenAI chief executive Sam Altman backed the proposal in a post on X, saying he agreed that the industry needed to “pace the frontier”.

Altman said independent evaluators with employee-like access were a good idea and that OpenAI would adopt the approach as well. He added that pacing AI development had been a major subject of discussion within OpenAI in recent weeks.

Tesla and xAI chief executive Elon Musk also responded to Amodei's post, writing: “Dario is right.”

Musk subsequently added that while research itself was unlikely to slow, companies could choose to withhold newly developed capabilities from public deployment.

Anthropic Already Has AI Safety Rules

Amodei's proposal builds on Anthropic's existing policies governing the development and deployment of increasingly capable AI systems.

Its Responsible Scaling Policy sets out additional safeguards that must be introduced as models reach specified capability and risk thresholds. Anthropic's Policy on the AI Exponential also argues for government authority to intervene in the deployment of AI systems that present significant risks.

Amodei's latest proposal goes further by arguing that the industry's overall pace of frontier development should itself be managed so that safety measures can keep up with capability gains.