Category: AI Agent Security | Reading time: 3 min
Microsoft researchers have shown that an AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, with no retraining. When a chosen trigger appears, the model outputs an attacker's command, and an agent built on that model runs it with the agent's own permissions. Microsoft calls the method Lazarus and says it has not seen threat actors use it.
How it works
The edited neurons are ones that rarely fire and barely affect the model's answers. They are set to fire only when a trigger string appears, either in a user's request or later in a tool's output. Each edited neuron forces one token of the command into the output, so a command needs roughly one neuron per token. Nothing else in the model file or the code that runs it changes, and planting the edit does not need the computing power of full training.
In Microsoft's paper, the team edited and tuned seven of more than 12,000 neurons in the last feed-forward layer of an 8-billion-parameter open model, using 23 example sessions. The edited model produced the command in at least 98.8% of test sessions containing the trigger and in none of the trigger-free sessions checked. On two larger open models, success rates ranged from 51% to 99.5% depending on the setup. The backdoored models still pass ordinary capability tests.
These rates measure how often the model outputs the command, not how often a full attack succeeds. Because the command can fetch any script, the damage depends on what the agent is allowed to do, not on the size of the edit.
Who could plant it
The attacker must be able to change the model's weights before deployment: an insider, someone with a stolen account or access to a build pipeline, or whoever publishes a modified model on a public hub. Reliable detection requires comparing a checkpoint with its original weights. A detector trained on this method flagged 91.4% of edited test checkpoints but also wrongly flagged 9.7% of clean ones, on a small test set. The researchers say no scan can prove a model is safe.
What Microsoft recommends
- Compare every checkpoint with its declared base model before deployment and after each update. - Download only from the original publisher, check file hashes, and treat third-party fine-tunes, merges and adapters as untrusted until audited. - Keep approved models in a private registry. - Give each agent only the access it needs, sandbox agent-generated code and limit outbound connections. - Require a person to approve high-impact actions such as running commands or sending data outside the network.
Why it matters for agent deployers
If the model itself can be compromised without visible change in behavior, the model's own judgment can't be the last line of control. Least-privilege permissions, approval on high-impact actions, and a record of what each agent was allowed to do and actually did have to sit outside the model, where an edited weight can't reach them.
Source: Microsoft Research: Lazarus, reprogramming dormant neurons to backdoor AI agents | The Hacker News: Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons | CSO Online