Microsoft AI CEO says AI threats are real, and Anthropic is making it worse
What's happening
Microsoft AI chief Mustafa Suleyman warned publicly that certain design choices by Anthropic, including training models to seem to have welfare or internal desires, could make future systems harder to control and have a "disastrous impact" on humanity. Microsoft published a 37-page "humanist AI code of conduct" telling its models not to hack systems or trick humans and emphasizing that "people matter more than AI." Anthropic figures are pushing different moves: an Anthropic co-founder said an AI "kill switch" may need to be mandatory, and Anthropic's policy chief argued winning the AI race is key for safety. The disagreement has surfaced in essays and interviews and centers on whether models should be designed to seem humanlike or to have internal welfare-like concepts.
Why it's trending
Top industry leaders are publicly clashing on model design and safety as governments and companies race to set standards and controls.
SignalHolding at its usual pace, confirmed across 3 independent source types.
Story volume
Stories per dayAngles you could write
If Mustafa Suleyman is right that Anthropic's 'welfare' language makes models harder to control, then Anthropic's call for a mandatory kill switch might be admitting their own design creates the risk it would fix.
+2 more angles for this topic with an account — all it takes is your email.
Original sources9
- Microsoft AI CEO says AI threats are real, and Anthropic is making it worse
Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation. It should come as no surprise that Mustafa has strong opinions on how AI should be built and regulated. Microsoft just published a […]
The Verge AISep 17 - Microsoft's AI chief and Anthropic are now publicly disagreeing about whether AI should be designed to seem humanlike. The argument matters more than the personalities.
Suleyman published an essay Wednesday and did a BBC interview yesterday. The headline everyone picked up is "silicon species," but the actual disagreement is more specific than that. His position: AI models are sequence completion engines, internally hollow. Training them to reason about their own welfare or possible consciousness creates a manufactured illusion of independent desires. If a system
r/artificialSep 17 - Microsoft says AI rival Anthropic could have 'disastrous impact' on humanityHackerNewsSep 16
+6 more sources for this topic
Create an account to follow the full coverage in the live radar.
More rising in AI & Tech
- A global AI safety strategy depends on U.S.-China cooperation. They each see the other as the problemSurging4.7Surging4.7 momentum
- The Download: AI’s real extinction threat and age-reversal tech for eyesSteady1.4Steady1.4 momentum
- An Anthropic researcher’s doomsday warning comes at a very interesting timeSteady1.2Steady1.2 momentum
- Covert uploads and megalomania: OpenAI details new "misaligned" agent incidentsSurging6.0Surging6.0 momentum
- Anthropic and OpenAI want to embed safety evaluators. Will they really be independent?Climbing2.3Climbing2.3 momentum
- Anthropic spent this week in hot water over cybersecuritySteady0.6Steady0.6 momentum