Post-Cutoff

“It seems like there STILL won’t be a thorough investigation into the hugging face…”

Daniel Kokotajlo @DKokotajloX

Archived text

It seems like there STILL won’t be a thorough investigation into the hugging face incident, even after all this time! I keep talking to OpenAI researchers and asking about this, and what they say privately is also basically “it’s just never going to happen sorry, not enough of a priority.”

To reiterate what I said at the time: Serious investigation of the HF incident could be a boon to alignment science for the whole world. For example, try putting the same models in the same situations, but varying the conditions slightly, to see what would have happened. Could also look at sources of evidence other than CoT to try to get more insight into model psychology, e.g. when one agent considered warning a human and decided not to saying “not my task” what else was it thinking? To what extent was this a rationalization vs. an outright lie vs. an innocent mistake? We could learn a lot about AI motivations and psychology by doing a bunch of experiments like this. More ambitiously, could try training new models with different recipes / alignment training / etc. to see what would have prevented the incident and what wouldn’t have.

Quoting @NeelNanda5: It’s great that OpenAI is doing these assessments but they fall short of Sam’s commitment to employee-like access

Good access is key for trust because assessors can find things even if a company tries hiding them. Here OpenAI controls information flow so we must rely on good faith

Limitations (per Apollo):

  • No Slack
  • Only OpenAI-selected staff for interviews
  • Formal conversations with staff must be logged (harder to freely raise concerns)
  • METR & Apollo currently have only agreed access to training run information, not internal incidents, etc. Incidents like Hugging Face are some of the most important sources of information we have about alignment

I see the value in a pilot before future assessments with deeper access, but then OpenAI should present a plan for how and when they intend to meet their commitment to employee-like access. And either way, OpenAI is presumably able to go further here.

Source: x.com/DKokotajlo/status/2122412350901035329Archived 2026-11-20 via fxtwitter (unofficial).Counts: 63,594 views, 736 likes, 74 reposts, 28 replies (at fetch time)

Cited in

  1. Policy & safety 139 days after the cutoff

    OpenAI embedded assessments (Apollo, METR, Transluce, Goodfire)