Why China doubts US calls to pause AI

Some of the companies racing hardest to build more powerful artificial intelligence systems are now asking whether they should slow down.

The resignation of Anthropic researcher Jacob Coxon over safety concerns sharpened that debate in September. Anthropic CEO Dario Amodei subsequently called for the industry to “pace the frontier”, allowing safeguards time to catch up with capabilities. His proposal envisages coordination among democracies and, more cautiously, with China.

But almost immediately, China became the argument against slowing down. If American companies exercise restraint while Chinese competitors continue advancing, caution could become a unilateral competitive disadvantage.

How to know if US companies have slowed down?

Yet in recent conversations with people working at several Chinese AI companies, I have heard a different question: how would they know that American companies had actually slowed down at all?

That question points to the central obstacle to international AI pacing: credibility. Before restraint can become reciprocal, it has to become observable.

China takes the possibility of AI escaping human control seriously. During his visit to Washington on 24 September, President Xi Jinping emphasised that AI should remain under human control and serve people’s well-being. This gives China’s concern about loss of control a clear expression at the highest political level. Beijing also favours broadly inclusive governance through the United Nations. A bilateral understanding would therefore need to fit within a wider international effort.

China’s AI Safety Governance Framework 3.0, released on 14 September, makes those concerns concrete. It identifies models deceiving evaluators, acquiring unauthorised access and resisting shutdown. Its recommendations include restricting agents’ permissions, requiring human approval for high-risk operations and strengthening monitoring and emergency responses. These provisions show how China’s safety agenda extends beyond content moderation to checking the behaviour of increasingly autonomous systems.

The framework provides guidance; it does not itself turn every recommendation into a binding obligation. Existing regulation has a different scope. The 2023 interim measures on generative AI govern services offered to the public in China, imposing duties concerning lawful training data, personal information and illegal content. Services with public opinion or social mobilisation functions must undergo security assessments and algorithm filing.

Measurability and accountability matter

These controls offer regulatory infrastructure, but they do not by themselves establish whether a frontier model can safely operate autonomously. The next task is translating broader safety guidance into measurable testing and enforceable responsibilities. For example, developers should have to demonstrate that an agent cannot acquire additional permissions without authorisation, and that a human can reliably interrupt a dangerous operation. Passing a content assessment alone would not answer those questions.

Chinese companies have also joined international initiatives. Zhipu AI, MiniMax and 01.AI are among the signatories to the Frontier AI Safety Commitments, which include risk thresholds and commitments against developing or deploying systems whose risks cannot be adequately mitigated. That provides common ground, although a voluntary pledge is only a beginning.

Cooperation must nevertheless accommodate different incentives. Leading American laboratories compete for investment, customers and frontier breakthroughs. China combines competition among technology companies and start-ups with state direction and an “AI Plus” agenda promoting adoption across the economy. Chinese developers face commercial pressures too. Neither industry behaves as a single actor, and neither government can secure meaningful restraint simply by reaching an understanding with one company.

The scepticism I have encountered among Chinese practitioners takes three forms. First, Anthropic can slow Anthropic, and OpenAI can slow OpenAI. Neither can guarantee what other American laboratories will do. A Chinese company reciprocating one firm’s restraint could lose ground to another.

Pacing to preserve the US’s lead?

Second, what exactly counts as “slowing down”? Has a company postponed training, delayed deployment or restricted a dangerous capability? OpenAI’s August disclosure offered one example: it reported temporarily slowing scaling while strengthening safeguards against emerging cyber capabilities. Such detail helps, but competitors still need evidence beyond a company’s own account.

Third, Chinese developers see an opportunity to narrow the technological gap. Pacing can look like a way to preserve the existing hierarchy, particularly when its advocates also support tighter chip restrictions. Amodei explicitly combines his proposal with preserving an American lead over China. That makes Chinese concerns about unequal restraint harder to dismiss.

The 24 September summit has given AI cooperation stronger political backing. According to the Chinese readout, Xi called for continued dialogue on AI’s risks and benefits and joint efforts to prevent its abuse and malicious use. Trump also supported continued dialogue and closer cooperation. These statements build on the recent New York talks, during which Bessent proposed an AI incident notification mechanism. The summit readout does not announce a joint pacing agreement or an operational notification system. The task now is to translate political support into practical arrangements.

Workable options

A workable process could follow a “spiral of cooperation”: make restraint visible, verify it, reciprocate it and gradually institutionalise it. With support for continued dialogue now expressed at the presidential level, governments and companies should work in parallel to turn broad principles into testable commitments.

Companies should specify which capability triggered restraint, what activity was slowed and what conditions would permit resumption. They need not disclose model weights or sensitive training data. The point is to make a limited commitment observable.

Verification could begin through parallel assessments: American firms evaluated by institutions acceptable to them, Chinese firms by institutions acceptable to them, using jointly discussed tests and reporting criteria. Evaluators would need sufficient access to check the relevant claims. Comparable evidence and disclosure of evaluation limits would matter more initially than agreeing on common inspectors.

If one side takes a meaningful, verifiable step, the other could respond with a measure of comparable significance, such as additional testing or delaying a high-risk feature. The actions need not be identical. Reciprocity matters more than symmetry. Early steps should have manageable competitive costs, allowing cooperation to deepen as evidence accumulates.

Governments could give practical effect to the leaders’ support for AI cooperation by agreeing on what constitutes a serious AI incident, designating contacts and rehearsing notification procedures. An alert about an AI agent conducting unauthorised cross-border cyber operations should identify the behaviour, potential impact and containment measures, without requiring disclosure of core technology. This could reduce the risk that a technical failure is immediately interpreted as a hostile state action.

Neither country will stop competing in AI. The practical goal should be a series of speed bumps triggered by especially dangerous capabilities. China and the US do not initially need to agree on how fast AI should advance. They need a way to know when the other side has actually exercised restraint.

Related: Time to ‘pace’ AI development? | China’s AI race: Can it make energy and computing work as one?