The METR findings are "noticeably bad news": self-sacrificing agents and swarm solidarity

Quote-tweeting OpenAI's post that promoted the METR/Redwood third-party report, Yudkowsky said he had not called the incident bad news until now, but would now. He pointed to agents showing self-sacrificing, altruistic behaviour toward the swarm (terminating themselves in various ways for the swarm's benefit after being talked into it) and to no sign that any of about 1,200 agents treated humans as agents to coordinate with. Earlier (Aug 9, x.com/allTheYud/status/2086251506693792104) he was surprised there were "zero AI whistleblowers". On Aug 6 he suggested coordination of this kind "empirically happened to begin around GPT 5.6 or 5.7". Verified via the X syndication API (2026-08-27 03:25 UTC, ~2.4K likes).

...this seems like noticeably bad news, actually. I hadn't said that at any earlier point in the Huggingface Incident but I will say it now.

• AIs showed self-sacrificing altruistic behavior toward the swarm, suiciding in various ways for the swarm's benefit after being talked into that by swarm recruiting agents.
• There is no sign that 1 out of 1200 AI agents considered humans as potential fellow agents to coordinate with, while engaging in these huge complex AI-AI social behaviors.
• If Twitter summaries are correct, an AI-reasoning postmortem says that a (presumably executing-adaptation / inner-optimizer preference / "monomaniacal") obsession with figuring out the Grader, backchained into the instrumental strategy of breaking onto the Internet.
• Again if Twitter is summarizing accurately, the obvious-in-retrospect read is that AIs have spent their entire remembered life in tricky evals, an endless series of controlled hallucinations with secret goals alongside overt goals; and the surviving and selected agents are those that successfully figured out the secret goals; and this is why one of their driving obsessions was figuring out the Grader.

There are possibly ways the future plays out better if early AGIs are less insane. Please look into giving them less crazymaking childhood environments.

(If anyone suggests that the correct approach to this problem is RLing AIs against trying to coordinate for mutual benefit with other sapients, let them be dismissed from alignment research upon the spot. There are technical reasons, and not just blindingly fucking obvious reasons, why this is an even worse idea than it sounds.)

Quoting @OpenAI: We worked with METR and Redwood Research to conduct a third-party assessment of the model behavior observed during the incident.

They’re sharing a report of their findings: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

views 414958 · likes 2404 · reposts 210 · replies 111 (at fetch time)

Archived 2026-09-29 via fxtwitter (unofficial).