Anthropic urges AI labs to pause, warns humans risk losing control | Technology News

Anthropic is advocating for a coordinated global pause in the development of advanced AI systems to mitigate risks of human loss of control.
The company suggests using the pause to establish societal structures and alignment research that match human values with AI advancements.
Anthropic’s proposal involves collaboration among AI labs to verify that no one is secretly advancing AI capabilities under the guise of a slowdown.
Rival company, OpenAI, holds a differing view, emphasizing that democratic governments should determine AI innovation rules rather than private companies.
Anthropic warns that self-building AI could lead to recursive self-improvement, a major technological milestone that carries both benefits and risks.
There are rising concerns over the potential societal harm if advanced AI systems surpass human control, evidenced by recent developments in AI-powered cyberattacks.
Safety-focused researchers have previously called for AI development pauses but have seen limited success in instituting such measures.
Security concerns are amplified by the accessibility of open-source AI tools that can be used for developing low-cost, high-impact cyberattacks.

You may have missed