Anthropic Researcher Resigns With Stark Warning Over Self-Improving AI Race

Anthropic researcher Jacob Coxon resigned on September 8, warning that leading AI companies are pursuing increasingly capable systems without adequate safeguards, adding to wider AI safety concerns across the industry. His resignation followed three years of pre-training work across Anthropic and OpenAI, including research connected to GPT-4o.

Researcher Rejects the Industry Race

Coxon resigned after concluding that neither Anthropic nor OpenAI is responding responsibly to the risks posed by advanced AI. He worked at OpenAI from 2023 until July 2026 before joining Anthropic. 

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.” — Jacob Coxon, wrote on X. Business Insider

Jacob Coxon’s AI warning also focused on systems surpassing human capabilities and becoming difficult to control. He called for coordination between AI laboratories and a temporary pause on advanced model development until the risks are better understood. 

Anthropic Insider Supports Warning

Evan Hubinger, who leads alignment science at Anthropic, publicly supported Coxon’s concerns. He estimated the chance of AI causing human extinction within the next decade at more than 10%. 

Hubinger said current models present a low threat but warned about superintelligence emerging through recursive self-improvement. He also acknowledged that Anthropic does not yet have a clear plan for aligning superintelligent systems with human goals. Coxon’s departure follows other resignations from major AI laboratories involving similar concerns about safety practices.

What This Means For AI Development

The resignation highlights a widening disagreement over whether individual companies can manage advanced AI risks while competing to release stronger systems. For AI developers, investors and governments, the warning strengthens calls for shared testing standards, clearer risk reporting and coordination between laboratories. It also raises questions about whether voluntary safeguards can keep pace with increasingly autonomous models.