UN Secretary-General Antonio Guterres called on Wednesday for a global effort to establish guardrails for artificial intelligence, just days after top executives of leading US technology companies warned that the pace of AI development should be slowed before it accelerates beyond human control.
Worrying pace of development
Anthropic CEO Dario Amodei posted an article last Friday warning that the current pace of AI development could enable AI systems to carry out destructive activities online. He said AI companies should slow the pace at which they improve AI models and strengthen their safety measures. OpenAI CEO Sam Altman, xAI CEO Elon Musk, and a number of other AI experts have publicly supported the proposal.
The debate on AI risks has changed significantly in the past two years. Theoretical discussions of hypothetical scenarios have been replaced by alarmed discussions of actual events. Since late 2025, cyberattacks carried out by AI agents have become real and frighteningly sophisticated. An attack by AI agents on Hugging Face’s database in July 2026 raised the alarm. An autonomous swarm of AI agents had carried out the attack at an inhuman speed.
“It's now going so fast that the researchers themselves have become terrified of it. And that's what we've seen in the last two or three weeks. It's the people who work at the companies who are coming forward, and the call is inside the House, saying like, ‘What we're doing is risking catastrophic damage,’” said Brad Carson, a former US congressman and current President of Americans for Responsible Innovation (ARI).
A major concern is the lack of oversight of technology companies that are investing billions of dollars into developing superintelligent AI models as quickly as possible.
“I think what worries me right now, and what probably is also being signaled by these worries around human extinction, is that currently companies hold a lot of power, and they're not really being held in check for their practices. I think that makes for a perfect storm, and I think that is the real worrying bit,” said Marta Ziosi, Deputy Director for Emerging Technology and Industrial Practice at the Ada Lovelace Institute in the UK.
The geopolitical dimension
AI’s risks demand a binding global governance mechanism subject to close independent oversight. Slowing down AI is not just a technological issue, however. It has a geopolitical dimension that’s growing. Last week, US President Donald Trump and senior US officials dismissed the growing concerns about AI’s risks, insisting that the US needs to maintain its current lead in AI development.
China has responded cautiously to the call from US tech leaders to slow down AI. Chinese tech experts and media questioned Anthropic’s motive, saying repeated warnings of AI-driven doomsday scenarios might be a tactic to promote regulations that favor the early leaders and put competitors in other countries at a disadvantage.
Stuart Russell, President of the International Association for Safe and Ethical AI and computer science Professor at the University of California, Berkeley, said AI risks know no borders and thinking one country’s AI is inherently better than another’s reflects a fundamental misunderstanding of AI.
“I think we have to take them seriously, a regulatory aspect to that. If we are worried about extinction risks from AI systems, those could come from anywhere. Obviously, they cross borders. We need international agreements to make sure that every country has in place the guardrails to make sure that that can't happen,” Russell said.
The important questions are: What should the AI legal framework be? and who should create it? Recent events have made it clear that AI is going to reshape the global order. People need to decide quickly how soon they can allow it to happen and who will write the rules.
The major powers regard AI dominance as crucial to their national security and future economic standing, but they should take care that they’re not trying to win a race off a cliff.
