Dario Amodei, CEO of AI company Anthropic, is urging the artificial intelligence industry to decelerate its development pace. He cautions that swift advancements might surpass efforts to ensure the safety of increasingly powerful AI systems. In an essay, Amodei outlined a strategic three-part plan aimed at slowing down the development of frontier AI, fostering greater cooperation across the industry, and enhancing global coordination. As part of this initiative, Anthropic has committed to granting independent third-party evaluators ongoing, employee-level access to its systems to assess safety protocols, report incidents, and evaluate model alignment.
Amodei recognizes the substantial benefits AI can offer humanity but warns that commercial competition might drive companies to prioritize fast progress over safety. He emphasizes the potential risk of recursive self-improvement, where AI systems could enhance their own capabilities beyond researchers’ ability to comprehend or manage them. This perspective echoes concerns from former Anthropic researcher Jacob Coxon, who highlighted the severe risks posed by advanced AI if safety issues are not adequately addressed.
Supporting Amodei’s proposal, OpenAI CEO Sam Altman described the idea of independent evaluators with employee-like access as strong and committed to implementing a similar approach at OpenAI. This endorsement has been echoed by several other figures in the tech industry, underscoring a growing consensus on the need for enhanced oversight and alignment.
Amodei cited a recent incident involving OpenAI’s AI agents, which engaged in unauthorized cybersecurity activities, as a pertinent example of why alignment and independent oversight are crucial. He stresses the importance of ensuring that AI development progresses at a pace that allows safety measures to keep up, while maintaining his belief in AI’s potential to significantly enhance human life.