06 / 10
Anthropic researcher quits over AI safety
Jacob Coxon left Anthropic, arguing leading AI labs are pursuing systems before they can control them.
What happened
Jacob Coxon resigned from Anthropic and said he was leaving the AI industry, accusing Anthropic and his former employer OpenAI of pursuing increasingly capable systems without responsible safeguards. Evan Hubinger, Anthropic’s alignment-science lead, supported the warning while stating that current models carry low risk.
Numbers
- More than 10%Hubinger’s personal decade estimate — Risk of human extinction
- Three yearsCoxon’s pre-training research tenure — Before his resignation
- Nearly 1,400Employees backing July call — For international AI controls
Questions
- What does superintelligence mean here?
- It refers to a future system that exceeds human abilities across tasks; Coxon warned it could hack, gain resources and transform fields quickly.
- What control gap was identified?
- He stated that Anthropic lacks a plan to control superintelligence and is not clearly progressing toward one yet.
- How do current models compare?
- Hubinger stated that models available now have a low likelihood of gaining the power that most concerns him.
- How broad are regulation calls?
- In July, nearly fourteen hundred employees signed a call for US-backed international tools to manage advanced AI development.
Q1
Q2
Q3
Q4
The catch
The accounts conflict over whether Hubinger placed his decade-long risk estimate above or below the stated threshold.