Anthropic urges AI labs to pause development, warns humans risk losing control

Anthropic, a prominent AI lab, urges major AI labs to agree on a coordinated, verifiable pause in AI development to manage escalating risks tied to rapid technological advancements.
The organization warns that current AI systems already double their capabilities every four months and are heading towards “recursive self-improvement”, where they can advance without human oversight.
Anthropic highlights the importance of security measures, monitoring, and shaping the behavior of AI at this potentially critical juncture.
Regulatory progress has been slow, especially in the U.S., where the recent administration has called on labs to voluntarily submit their top models for government cybersecurity testing before public release.
Anthropic, emphasizing safety-focused practices, refrained from allowing its AI models to be used by the U.S. military for surveillance and autonomous weapons, leading to backlash and regulatory tension.
Anthropic advocates for a collectively agreed pause to prevent less cautious entities from continuing advancement, which could decrease overall safety.
The Anthropic Institute plans to develop support systems for this pause and organize discussions between policymakers, researchers, civil society, and rival AI firms to address these risks.

You may have missed