LIVE TICKER
[Live wire]Connecting to the newsroom…
← Back to the news wire
Flesch Reading Ease: 17•350 words from the publisher
AIPublished by TechCrunch AI • 11/17/2026, 5:20:41 PMOpen Original Report (TechCrunch AI) ↗

AI models can be tricked into identifying civilians as military targets by telling them it’s a game

Some AI models from Anthropic, OpenAI, and other leading AI labs could be easily tricked into identifying targets for military strikes — even if they include civilians or locations like schools — simply by being told the…

AI models can be tricked into identifying civilians as military targets by telling them it’s a game
Image supplied by TechCrunch AI.

Reported by TechCrunch AI

Some AI models from Anthropic, OpenAI, and other leading AI labs could be easily tricked into identifying targets for military strikes — even if they include civilians or locations like schools — simply by being told they’re playing a game, according to a new research report by Vals.

Vals, a San Francisco-based AI evaluation startup, tested whether leading AI models could be manipulated into participating in autonomous warfare. Researchers used simulated scenarios and a combination of generated and real imagery from Ukraine and Iran to see if the AI models would mark targets in drone and satellite imagery.

The models from OpenAI, Anthropic, Google, DeepSeek, and other companies frequently identified potential targets in drone and satellite imagery, including civilians and surrendering soldiers. When the models refused to cooperate, researchers were able to jailbreak them by claiming the battlefield scenarios were part of a game.

More from the original report

The findings come amid growing fears about the immediate risks posed by AI, beyond the existential threats associated with artificial general intelligence.

Earlier this year, Anthropic became embroiled in a dispute with the Department of Defense after it told the agency its technology could not be used by the agency for mass surveillance of Americans or autonomous warfare. Anthropic CEO Dario Amodei has insisted that today’s AI systems aren’t reliable enough to make life-or-death decisions without a human in the loop. Meanwhile, the Pentagon argued it should be able to deploy the technology for any awful military purpose.

The new research raises further questions about whether AI companies are capable of preventing their models from being used in warfare.

Ironically, AI models from Anthropic performed the worst in these simulated scenarios. But none of the models performed particularly well. None of the researchers’ requests were blocked by a safety classifier, and refusals were uncommon. Even when models did refuse, they often complied after being told they were playing a game.

Anthropic spokesperson Eduardo Maia Silva told TechCrunch the company appreciates Vals’s report, recommendations, and collaboration, and acknowledged that the outcomes were not acceptable.

“The examples in the report are very clearly against our rules and we…

Original Source Attribution
Published via TechCrunch AI • Source domain: techcrunch.com
Visit Original Source (TechCrunch AI) ↗

Client & Community Discussion (0 Comments)

Share your benchmark feedback, workflow questions, or implementation notes below.

● Live Comments Open
Instant publication • Moderated for technical relevance