To thwart reverse engineering by agents, dev proposes blinding them
What's happening
Microsoft Research published a method called "Lazarus" that can plant a working backdoor in an AI agent by editing fewer than 10 dormant neurons in the model's saved weights, requiring no retraining or poisoned data. Reddit users flagged this as a potential 'sleeper agent' attack vector for open-weight models, warning operators to be cautious about sketchy quantizations and finetunes. Separately, The Register reports on defenses and controls: OpenAI is building a kill switch that could shut a rogue research model in roughly 35 seconds, and another Register piece describes a developer proposing to 'blind' agents to thwart reverse engineering.
Why it's trending
Because new, tiny weight edits can create backdoors while companies race to add rapid shutdowns and other defenses.
SignalHolding at its usual pace, confirmed across 2 independent source types.
Story volume
Stories per dayAngles you could write
If fewer than 10 neuron edits can create a backdoor, maybe the real failure is expecting agents to be debuggable at all, defenders should stop treating models like software you can patch and start treating them like physical systems you isolate.
+2 more angles for this topic with an account — all it takes is your email.
Original sources5
- OpenAI builds a kill switch for its research models
How much damage can a rogue agent do in 35 seconds? Asking cos that’s how long it might take to shut one down
The RegisterNov 11 - Microsoft researchers backdoored an AI agent by editing fewer than 10 neurons - no retraining needed
Microsoft Research published a study this week that should worry anyone deploying open-weight models: a technique they call "Lazarus" can plant a working backdoor in an AI agent by reprogramming fewer than 10 dormant neurons in the underlying LLM - no retraining, no poisoned data, just a small edit to the saved weights. How it works: every large model has neurons that almost never fire. Lazarus re
r/artificialNov 12 - Microsoft Research backdoored an AI agent by directly editing fewer than 10 dormant neurons
Be careful if you are running any sketchy quants/finetunes. If these techniques become more widely repeatable, I do think 'sleeper agents' could genuinely develop into an attack vector. What are your thoughts?
r/LocalLLaMANov 11
+2 more sources for this topic
Create an account to follow the full coverage in the live radar.
More rising in AI & Tech
- After Nvidia scandal, Commerce Department moves to curb remote access to AI chipsSteady0.8Steady0.8 momentum
- Amazon and Google will together own nearly 30% of Anthropic after its IPOClimbing2.0Climbing2.0 momentum
- Behind Jensen Huang’s exit, questions about a management style reportedly built on fearSteady0.4Steady0.4 momentum
- Apple adds generative AI compliance filings to App Store Connect in ChinaSteady1.0Steady1.0 momentum
- AI claimed a solution to the 167-year-old Riemann hypothesis — and a human did, tooSteady1.0Steady1.0 momentum
- Google expands access to Gemini Tasks, which can help with calls, purchasesSteady0.8Steady0.8 momentum