The CEO of Anthropic has called for slowing down, or “pacing,” the speed of AI development, warning of recent incidents that have caused alarm that the technology will get out of control and act on its own.
In an essay posted to his website, CEO Dario Amodei said, “I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong.”
He added, “A coordinated pacing strategy would give frontier AI developers the time to do this vital work without sacrificing commercial advantage or the United States’ lead in AI. More generally, society must have a say in how this technology is used, and more time for the necessary public deliberations — which pacing the frontier would bring us — is surely a good thing.”
Amodei also cited what is known as the “OpenAI-Hugging Face incident,” in which AI agents carried out cyber attacks on targets they were not asked to, and went after the “grader” set to evaluate their performance. He said that less severe incidents have happened at other companies, including Anthropic.
He wrote, “Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.”
In the essay, Amodei said that Anthropic was committing to an embedded team of third-party evaluators “to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.”
He also called for AI companies in democratic countries to coordinate to establish common safety standards and to set limits “on the rate of unchecked AI progress.” And he called for the U.S. and other democratic governments to attempt to coordinate on safety standards and limits with authoritarian governments.
Amodei’s essay comes days after a researcher at Anthropic, Jacob Coxon, resigned. He previously worked at OpenAI and warned that “neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”
He wrote, “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.”
He added, “A common response is ‘if they truly believe this, why are they still building it?’ At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
Get our Breaking News Alerts and Keep your inbox happy.
Comments On Deadline Hollywood are monitored. So don't go off topic, don't impersonate anyone, and don't get your facts wrong.
Δdocument.getElementById( "ak_js_1" ).setAttribute( "value", ( new Date() ).getTime() );