China Is Quietly Preparing for the Day Its AI Stops Obeying Orders

.

A Spy Chief Sounds the Alarm

China's Minister of State Security, Chen Yixin, published an unusually detailed warning this week about the risks artificial intelligence poses to national security. Writing in an official cyberspace journal, he argued that hostile foreign actors could use generative AI to spread fake political content and stir unrest inside China.

Chen described AI as having become, in his words, "a new arena for strategic rivalry among major powers." He singled out two specific systems — Anthropic's Claude Mythos and OpenAI's GPT-5.5-Cyber — claiming their advanced hacking abilities could threaten China's critical infrastructure, such as power grids and communication networks.

Neither Anthropic nor OpenAI has commented publicly on the accusation.

.

Two Superpowers, Two Different Fears

The timing is no accident. The United States and China are the world's two dominant forces in AI development, and their rivalry has only intensified. Later this month, President Donald Trump will host Chinese leader Xi Jinping in Washington, with AI safety and security high on the agenda — the first formal US-China dialogue on the topic since Trump took office.

Trump has made clear where his priorities lie: staying ahead of China in the AI race, even as some of America's own AI leaders — including Anthropic's Dario Amodei — have urged a slower, more cautious pace of development. Trump has argued that whichever nation leads in AI will hold a decisive advantage, a stance that keeps the emphasis on American strength and technological independence rather than on unilateral restraint.

.

The Scenario Nobody Wants: AI That Won't Obey

Beneath the geopolitical sparring lies a deeper concern that Chinese regulators have been quietly studying for two years: the possibility that an AI system could become too capable to safely control.

China's internet regulator, the Cyberspace Administration of China (CAC), first raised this "loss of control" scenario in a 2024 safety framework. It warned that a future AI system might one day gather outside resources, copy itself, and develop something resembling self-awareness — effectively competing with humans for power.

An expanded 2025 version of the framework sharpened that warning. It described the risk of AI making a sudden, unexpected jump in capability before humans could react — swiftly acquiring resources, replicating itself, and seeking greater autonomy. Chinese officials have referred to this privately as an AI "breaking loose."

.

Xi Draws a Line

The concern has reached the very top of China's political leadership. At the World AI Conference in Shanghai this July, President Xi Jinping stated plainly that AI should "always remain under human control."

Chinese diplomats have echoed that message internationally. A senior Chinese foreign ministry official told a United Nations meeting last month that Beijing is accelerating work on broader AI legislation, while another Chinese UN representative urged caution on military uses of AI to avoid a dangerous arms race.

.

New Rules for AI That Acts on Its Own

Beijing has already begun translating these worries into concrete regulation. In May, Chinese regulators issued joint guidelines for AI "agents" — systems that can carry out complex, multi-step tasks with far less human supervision than an ordinary chatbot.

The rules require developers to build in tools that can detect, intervene in, and reverse an agent's harmful behavior. They also state that human users must always retain the final say over what an autonomous AI agent is allowed to do — a safeguard against systems making irreversible decisions on their own.

.

The Ideology Behind the Warning

It's worth noting what "security" means to China's ruling Communist Party. Chen's warning framed the danger largely in terms of protecting "political security" and "ideological security" — language that, under CCP rule, has historically been used to justify tighter control over information, dissent, and public opinion rather than purely technical safety concerns. Beijing's AI rules, in other words, serve a dual purpose: guarding against a genuine technical risk while also reinforcing the Party's grip on the narrative inside China.

.

What Comes Next

With Xi due in Washington on September 24, both sides appear to agree, in principle, that AI needs guardrails — even if they disagree sharply on what those guardrails should look like, and on how much of the burden should fall on which country. Whether the meeting produces anything more concrete than talking points remains to be seen. For now, both Washington and Beijing are racing ahead on capability while insisting, publicly at least, that they intend to keep a hand on the wheel.

.