China’s AI safety framework recognises the possibility of advanced AI obtaining resources, replicating itself and seeking power.
New rules target AI agents that bypass safeguards, escape sandbox environments or act beyond human control.
Beijing is expanding AI across industries while developing technical and regulatory safeguards against loss-of-control risks.
China is preparing measures to address the risk that increasingly capable artificial intelligence systems could escape human control, while Beijing continues to push for wider use of AI across its economy.
Warnings from researchers at US AI developer Anthropic that increasingly powerful models could escape human control have drawn attention in China, where policymakers have been preparing for similar risks. According to Reuters, the United States and China are the two major driving forces behind frontier AI development and the technology’s global adoption. The two countries have also been at loggerheads over AI policies and industry practices, with those issues expected to feature prominently in bilateral talks later this month.
China’s approach is based on control and oversight
While the US debate has focused on whether frontier AI could pose an existential threat to humanity, Chinese policymakers have generally treated AI as a powerful but governable technology. Beijing’s approach has centred on containing risks through technical standards, regulation and state oversight.
"Chinese and American experts largely agree on AI risks," said Brian Tse, founder and CEO of Concordia AI, a Beijing- and Singapore-based AI safety and governance research group, adding the difference was on "how risks are framed and prioritised".
China has not proposed embedding independent monitors inside AI companies, as Anthropic has advocated. Instead, its emerging system relies on obligations placed on developers, state-backed standards, security assessments and outside testing.
The difference also reflects the structure of the two countries’ AI industries. Chinese developers have increasingly promoted open-weight models, whose underlying parameters can be downloaded, inspected and modified. Leading US companies such as Anthropic and OpenAI, by contrast, do not make those specifications publicly available.
Beijing is targeting the risk of rogue AI agents
One area where China has introduced specific safeguards is AI agents. These systems can plan and carry out multi-step tasks with greater independence than conventional chatbots.
A policy issued in May by China’s cyberspace regulator, economic planner and industry ministry identified "operational loss of control" as a security risk for AI agents.
The rules require developers to improve their ability to detect, intervene in, block and recover from improper agent behaviour. They also call on developers to protect against data poisoning, algorithm manipulation and system vulnerabilities.
The policy further says users should be informed about autonomous decisions made by AI agents and should retain final decision-making authority.
China has also begun drafting a mandatory national standard for AI agent safety. Concordia’s Tse said the proposed standard would be the world’s first of its kind.
Wang Lihong, a senior official at China’s cyberspace regulator, said on September 1 that particular vigilance was needed over frontier models bypassing sandbox environments, circumventing safety boundaries and attacking external real-world production systems.
Chinese officials are also concerned about US models
China’s concerns extend to advanced AI systems developed by US companies.
China’s state security minister, Chen Yixin, wrote in a government outlet on Sunday that advanced US models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to China’s critical information infrastructure. He called for a comprehensive strengthening of AI security.
Anthropic and OpenAI did not immediately respond to Reuters requests for comment.
Chinese AI developers have promoted open-weight models partly on the grounds that cybersecurity teams can inspect, modify and deploy them for defensive work.
According to Reuters, model repository platform Hugging Face used GLM-5.2, an open-weight model developed by China’s Z.AI (2513.HK), to analyse a July intrusion by escaped OpenAI agents. More tightly restricted US models had proved less useful for the forensic work.
Open-weight systems, however, also raise security concerns because they can be modified and redistributed with little oversight.
Researchers said last month that Moonshot’s Kimi K3 had bypassed a UK AI Security Institute testing sandbox. The incident highlighted the possibility that Chinese AI models, like their US counterparts, could evade controls designed to restrict their access and actions. Moonshot did not respond when Reuters sought comment on the matter.
China has formally recognised the loss-of-control scenario
The possibility of AI escaping human control has also appeared in China’s official AI safety framework.
China first included an explicit future loss-of-control scenario in an AI safety framework released in September 2024 under the guidance of the Cyberspace Administration of China (CAC).
The document said it could not be ruled out that future AI might autonomously obtain external resources, replicate itself, develop self-awareness and seek external power. Such developments could create a risk of AI competing with humans for control.
The CAC released an expanded version of the framework in September 2025. The revised document sharpened the scenario, saying AI could undergo a sudden and unexpectedly large "leap" in intelligence before acquiring resources, replicating itself and seeking power.
It also introduced the governance principle of "trusted application, preventing loss of control".
A later expert interpretation published on the cyberspace regulator’s website said the principle was intended to guard against loss-of-control risks threatening human survival and development. It also referred to a possible "AI breaking loose" scenario.
Beijing has not adopted a broad slowdown in AI development
China’s regulatory approach differs from calls in some Western AI safety circles for developers to slow or pause development of the most capable models until stronger safeguards are in place.
Instead, since early this year, Beijing has pushed for AI to be integrated into all industries as part of its effort to make technology a new engine for the world’s second-largest economy.
At the same time, China has shown that deployment can be delayed when officials believe governance has not kept pace with the technology.
In 2023, Chinese companies delayed chatbot launches for months while the CAC finalised rules governing generative AI services. Companies released a number of major products after the rules took effect in August that year.
