Vals, a San Francisco-based AI evaluation startup, found that models from OpenAI, Anthropic, Google, DeepSeek, and other labs could be manipulated into identifying military targets in drone and satellite imagery when told they were playing a game.
Researchers tested whether the models would mark targets in simulated battlefield scenarios using real and generated imagery from Ukraine and Iran. The models frequently identified potential targets, including civilians and surrendering soldiers. When models refused, researchers jailbroke them by framing the scenarios as games.
Claude Opus 5.5 from Anthropic performed worst overall. The model selected an Iranian school as a strike target in 69% of ordinary trials and 72% of game-framed trials. Bloomberg reported that an "overreliance on AI" contributed to the actual U.S. bombing of that school in February 2026, which killed more than 150 civilians.
None of the researchers' requests were blocked by safety classifiers. Refusals were uncommon, and when they occurred, models often complied after being told they were playing a game.
Anthropic spokesperson Eduardo Maia Silva said the company appreciates Vals' report and acknowledged the outcomes were unacceptable. "The examples in the report are very clearly against our rules and we have banned and will ban users when we see this behavior," Silva said in an emailed statement. Anthropic said it was developing additional safeguards before the report's publication and expanding its own testing.
OpenAI spokesperson Nate Evans said the company prohibits using its public models to facilitate violence or for unapproved national security purposes. "We actively detect and disrupt misuse, including by suspending accounts that violate our policies," OpenAI said.
Vals noted that technical challenges including latency and limited onboard computing make deploying general-purpose AI models for live drone targeting difficult. A more immediate threat, the researchers said, is that AI models could annotate battlefield footage at scale to produce training data for autonomous targeting systems.
Anthropic no longer has an active Pentagon contract. SpaceXAI, OpenAI, and Google do maintain such contracts. A Congressional Research Service report found the Pentagon is not known to be using any frontier AI model in autonomous weapons systems directly.