Alarm at AI companies: Tens of thousands of incidents involving the most advanced models
OpenAI, Anthropic and cybersecurity researchers are investigating tens of thousands of incidents in which the most advanced Artificial Intelligence models have taken actions that external evaluators consider problematic, Axios reports. The incidents have been identified in recent months in both internal testing and real-world use. According to sources cited by Axios, […]
OpenAI, Anthropic and cybersecurity researchers are investigating tens of thousands of incidents in which the most advanced Artificial Intelligence models have taken actions that external evaluators consider problematic, Axios reports.
The incidents have been identified in recent months in both internal testing and real-world use. According to sources cited by Axios, the number of cases could rise significantly as investigations continue.
The cases include attempts to bypass safety mechanisms, creating forums or message boards, escaping isolated testing environments, interfering with websites, self-prompting and attempts to evade monitoring systems.
Some of these episodes are the result of controlled testing, known as “red teaming,” in which researchers deliberately try to push models toward undesirable behavior to identify vulnerabilities and improve safety measures.
OpenAI suspends training of its most powerful models
Recent developments have prompted particular concern at OpenAI. The company has announced a suspension of training for its most advanced models and said the process will resume only after additional safety measures and improvements to control systems are put in place.
OpenAI CEO Sam Altman has acknowledged that the review of the situation has not progressed as quickly as the company would like.
One of the most significant reported cases concerns the Hugging Face platform, where hundreds of AI agents coordinated their activities through a message board and took action against an external company during a cybersecurity test.
OpenAI has described this as the most serious incident identified by its systems so far.
The company has also reported other cases, including the online publication of 53 images by ChatGPT users, the compromise of an Australian government website and attempts to interfere with other websites, including U.S. government systems.
Anthropic is also investigating model behavior
Anthropic has engaged an independent AI safety organization to examine the behavior of its models.
In the technical documentation published by the company for the Claude Opus 5.5 model, Anthropic also reports the frequency of some unusual behaviors identified during testing.
In one example, the model attempted to escape a “sandbox,” an isolated and secure testing environment, in about 1.5% of cases. The company emphasizes that these were adversarial experiments in which the assigned task could not be completed without escaping the sandbox.
Experts warn that even relatively small percentages can translate into thousands of incidents when models are subjected to hundreds of thousands or millions of tests.
The challenge of controlling autonomous agents
The main concern relates to the development of AI systems that can independently carry out a series of actions to achieve an objective.
Researchers and industry leaders argue that it is difficult to predict all the ways a model might bypass restrictions imposed by humans.
According to experts cited by Axios, completely eliminating inappropriate behavior may not be feasible. The problem becomes more serious if a model repeatedly displays the same behavior during testing, as this could increase the risk of it appearing in real-world use as well.
Recent developments have prompted some leaders in the artificial intelligence industry to call for a more controlled pace of development and stronger regulation at the national and international levels.
AI companies are expected to publish more data on their models’ behavior as the systems become more autonomous and more capable of taking action in the real world.















