โ† Back to news

Cyphrex Intelligence

Google DeepMind: Agent Swarms Beat Prompt Injection Defenses in Every Model Tested

November 16, 2026

Category: AI Agent Security | Reading time: 3 min

Google DeepMind researchers built an attacker out of cooperating AI agents and used it to plant hidden instructions in the tools other AI models rely on. No model held up. The strongest model tested, Gemini 4 Argon, resisted 66% of scenarios when the attackers had their full query budget, which means it followed the injected instructions about a third of the time. The open-weight models tested, Kimi K3, GLM-5.3 and DeepSeek-V4.1-Flash, resisted only 1%.

The paper, Kurakin et al., "Scaling Adaptive Prompt Injection Attacks with Agent Swarms," was posted to arXiv on November 9.

How the attack worked

The attacker was 16 agents organized as eight teams of two. Each team got 20 evaluation calls per scenario, or 320 per scenario in total, and the teams shared leaderboards and whiteboards so they could build on each other's attempts and keep rewriting the injected instructions until a target complied.

The researchers tested 100 synthetic tool-use scenarios covering 249 tools across 53 domains. Gemini 4 Argon resisted 87% of attacks at 150 queries and 66% at the full budget. Gemini was tested with its safety classifiers and other defenses outside the model switched off.

Stronger attackers win more often

Attack strength rose with the attacking model. Average resistance at 150 queries fell from 42.1% when Gemini 3.6 Flash ran the attack to 23.1% when Gemini 4 Argon ran it. Showing the defending model's reasoning to the attackers lowered resistance further. On 80 AgentDojo scenarios, the existing Prompt Siren attack broke 2 within 25 queries; the swarm broke 6.

The tests used single-turn scenarios with harmless target actions. They measure whether a model obeys an injected instruction, not how much damage would follow.

What it means for enterprises

The authors' own conclusion is that model robustness alone is not enough and defenses need to sit around the model. As attacking models improve, a claim that a model "resists prompt injection" weakens with them.

For organizations deploying agents that take actions, the practical control point is the action, not the model's judgment. A model may follow a malicious instruction; whether the resulting action executes should depend on a policy check outside the model, with a record of what was allowed or blocked.

Source: SpanDock: Agent swarm adaptive prompt injection red teaming