“One question to consider with respect to these embedded assessments is whether these…”
Nathan Calvin @_NathanCalvinX
Archived text
One question to consider with respect to these embedded assessments is whether these evaluators would have had a sufficient amount of access to detect incidents like the Artifactory outage stemming from the unauthorized exploit gym message board (which pre-dated the HF incident in July).
Based on what I have seen here, I think the answer is no, unless the company chose to deliberately make the evaluator aware of such an incident (which to be clear, is what I would expect companies would want by default!).
Quoting @NeelNanda5: It’s great that OpenAI is doing these assessments but they fall short of Sam’s commitment to employee-like access
Good access is key for trust because assessors can find things even if a company tries hiding them. Here OpenAI controls information flow so we must rely on good faith
Limitations (per Apollo):
- No Slack
- Only OpenAI-selected staff for interviews
- Formal conversations with staff must be logged (harder to freely raise concerns)
- METR & Apollo currently have only agreed access to training run information, not internal incidents, etc. Incidents like Hugging Face are some of the most important sources of information we have about alignment
I see the value in a pilot before future assessments with deeper access, but then OpenAI should present a plan for how and when they intend to meet their commitment to employee-like access. And either way, OpenAI is presumably able to go further here.