Anthropic urges ‘pause’ or ‘slowdown’ of AI development after Leo’s encyclical

Less than two weeks after Pope Leo XIV published an encyclical warning artificial intelligence (AI) companies against constructing “a new Tower of Babel,” the multibillion-dollar AI company Anthropic is calling for a global pause or slowdown in development.

Anthropic co-founder Jack Clark and Anthropic Institute head Marina Favaro published a blog on June 4 warning about a risk of “humans losing control over AI systems” as its own system Claude is reaching the potential to autonomously design its own successor without any human contributions.

“This is called recursive self-improvement,” they wrote. “We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for.”

The blog post did not mention the encyclical, but a separate Anthropic co-founder, Chris Olah, met with Leo and sat alongside the pope when the encyclical Magnifica Humanitas was revealed on May 25. Anthropic has engaged in outreach to the Vatican and other religious leaders to help address ethical questions related to AI development.

In the blog post, Anthropic leaders explained that its AI system is taking over a large portion of writing code that designs AI — with its workload growing eightfold every quarter. AI will “become much more capable in coming years,” they wrote, and “these trends have huge implications.”

“If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important,” they wrote.

Although Clark and Favaro acknowledged AI has not reached this level yet and they cannot say for certain it will, they wrote: “We do not have good intuitions for what this world would look like” if this occurs, and AI capabilities “eclipse those of humans.”

Anthropic’s leaders wrote that AI companies should come together to either pause or slow down development “to give ourselves more time to deal with its immense implications.” However, this would require global international cooperation among countries and AI companies because “if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe,” they wrote.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” they added.

Anthropic intends to engage with policymakers, researchers, and other members of the public to discuss these concerns. The company will publish a document based on what comes out of the conversations.

‘Disarming’ AI

Charles Camosy, a moral theologian at The Catholic University of America who has worked with Anthropic on ethical questions, told EWTN News that Anthropic’s statements appear in line with Leo’s desire to “disarm” AI, which the pontiff explained as not halting innovation but “preventing it from dominating humanity.”

He said Anthropic recognizes the speed of development as “such a problem we all need to slow down here.” Such a pause would allow society to “think about what AI should or should not do in the culture,” he said.

Camosy pointed to concerns about “outsourcing” teaching, tutoring, parenting, care for the sick, and other human interactions to AI, possibly “undermining the things that … make our humanity magnificent.”

He recognized that fierce AI competition among nations and companies “creates a significant roadblock” to global cooperation for slowing everything down, but said: “I’ve been astonished by how many different kinds of people are interested in this encyclical.”

“Many people were kind of waiting for someone to fill the moral space,” Camosy said and suggested the Church help lead a global movement that demands ethical AI, and he encouraged the Holy Father to consider a trip to Silicon Valley.

“To many people that sounds hopefully naive,” he said. “But I don’t see another choice here.”

Source: https://www.ewtnnews.com/world/us/anthropic-ai-warning