AI is not the same as nuclear weapons. Can the US and China cooperate?

Earlier in September 2026, Anthropic CEO Dario Amodei published a 3,800-word essay on his blog called “We Must Pace the Frontier”. His calls to “slow the pace at which we improve the capabilities of AI models” followed the high-profile resignation of an Anthropic employee over his concerns about AI safety.

Others scoffed at the idea of slowing the pace, rejecting the premise that AI could cause human extinction. Nvidia’s Jensen Huang said that “scaring people is unnecessary” and “irresponsible” and “there is 0% chance that’s going to be the end of the world”. Former Google CEO Eric Schmidt acknowledged risk but argued that slowing AI development merely cedes advantages to China. 

As this discord over the nature of the risk and the prospects for coordination suggests, even leaders within the technology industry disagree not only about the dangers posed by advanced AI, but also about whether constraining its development is wise or feasible.

Difficult trust among rivals

The tension is even sharper once China enters the picture since efforts to “pace” AI require coordination across competing firms and countries. Several ingredients would be necessary for some kind of broader international agreement on AI: convergence that AI does present risks best managed across countries, but also trust that rivals will forgo potential advantages by engaging in restraint.

Neither condition is easy to satisfy. Both the US and China may have an interest in avoiding catastrophe, but they have also professed the value of artificial intelligence (AI) in their country’s economic competitiveness and national security, which creates opportunity costs to slowing development, not least if the other cannot be trusted to do the same.

Advocates of US-China cooperation on AI have invoked Cold War nuclear cooperation between the US and the Soviet Union. For example, in 2023, Henry Kissinger and Graham Allison observed that “even adversaries racing to develop and deploy an unprecedented technology that could kill hundreds of millions of people nonetheless discovered islands of shared interests”.

The Cold War analogy, however tempting, is also misleading. A step toward meaningful nuclear agreement finally emerged in 1963 with the Limited Test Ban Treaty but only after a gravely manifested harm with the two detonated bombs in 1945, 18 years of Cold War escalation, a Cuban missile crisis in 1962 that led to the brink of nuclear apocalypse and a dramatic lowering of ambitions in scope, leaving the underground tests that were near impossible to verify off the table and including just atmospheric, underwater and space nuclear tests. 

Not state to state

Second, nuclear weapons were the province of states. No private contractor or individual could develop or host a nuclear arsenal. Governments controlled those decisions and their arsenals. By contrast, the engine of AI development resides in private firms that have commercial and investment incentives and proprietary technologies they are motivated to protect. Private firms are not insulated from government control. For example, OpenAI restricts access to its biological models to vetted organisations operating under specified governance and security requirements. AI cooperation across states is thereby more complicated because it means coordinating with each other but also with private firms whose commercial incentives are varied and may not align with the state.

Third, the geopolitical incentives are different. During the Cold War, nuclear competition operated with a mutually assured destruction dynamic. President Reagan said in 1985 that he and his counterpart had agreed that “a nuclear war cannot be won and must never be fought”. AI is strategically different. Xi Jinping has referred to AI as a “rare, historic opportunity” and an engine of economic growth. The US Treasury has written that AI investment accounted for 30% of GDP growth in 2025. Both countries see AI as valuable for progress, whereas nuclear weapons were costly and played no central role in their economies.

Hopes for risk reduction

That traditional arms control is difficult for AI does not mean that no steps can be taken. OpenAI, for example, has called for common standards for measuring frontier capabilities, shared incident-reporting protocols, and secure channels for exchanging information about emerging risks, while leaving regulatory authority with national governments. These proposals do not “pace” AI in the way Amodei had implied, but more closely resemble early logics of Cold War risk mitigation in which both sides had incentives to avoid accidents and miscalculation and installed hotlines that could improve communication.

Other feasible measures tend to fall into risk reduction rather than slowing or pacing AI. An AI incident hotline could help China and the US learn about serious safety events, authenticated crisis communications could reduce the risk of deepfakes triggering escalation, and conversations around autonomy on the battlefield can help shape norms even if they do not proscribe certain uses or behaviours. 

None of these would require the deep trust associated with “unilateral disarmament”, which in this case might mean slowing the pace of AI, relinquishing the prospect of economic gains from AI, or abandoning the geopolitical competition. The ambition underlying the cooperation is narrower than either the nuclear test bans or, later, cuts to the arsenals. On the other hand, these measures are not trivial and gesture toward an important lesson of the Cold War, which is that two sides can be fierce rivals but have mutual incentives to limit the costs of competition.

Related: Will AI drink China’s farms dry? | Time to ‘pace’ AI development?