An Anthropic AI safety researcher has said he believes there is more than a 10% chance that artificial intelligence could kill all humans within the next decade, following a public warning from a former colleague.
Jacob Coxon, a 27-year-old researcher who recently announced his departure from the AI industry, said people developing advanced AI genuinely believe the technology could become capable of killing humanity. Coxon said he no longer believes any single laboratory can safely build self-improving AI.
Coxon, who has worked at both OpenAI and Anthropic, accused the companies of moving too quickly toward self-improving superintelligence and taking unacceptable risks.
Evan Hubinger, Anthropic’s Alignment Science Lead, agreed with Coxon’s assessment and said he personally estimates the chance of AI killing all humans within the next decade at more than 10%.
Hubinger said Anthropic is doing its best to address AI safety but acknowledged that researchers do not yet have a solution to the alignment problem for superintelligence and are not clearly on track to develop one.
He later stressed that today’s Claude models pose a relatively low risk. His primary concern, he said, is the emergence of superintelligence through recursive self-improvement, in which AI systems improve their own capabilities and potentially advance faster than anticipated.
The comments have sparked discussion about the different approaches to AI safety across the industry’s leading companies. Arun Rao, an adjunct professor at UCLA, said the episode reflects a broader ideological divide over the risks posed by advanced AI.
US Congressman Ted Lieu also cited the controversy in support of the bipartisan AI Kill Switch Bill, arguing that such warnings demonstrate the need for stronger safeguards around frontier AI.
The debate comes as AI companies continue to develop increasingly capable systems while researchers and policymakers remain divided over how quickly the technology should advance and whether existing safety measures are adequate.