Anthropic's Olah on AI: Vatican Calls for Caution

Chris Olah, co-founder of AI firm Anthropic, delivered remarks at the Vatican City presentation of Pope Leo XIV's encyclical, "Magnifica humanitas: On safeguarding the human person in the time of artificial Intelligence." The May 25, 2026, release marks a significant engagement with AI's ethical dimensions by the Church. Olah's address, made available through Anthropic News, underscored the critical need for voices outside the AI industry's competitive pressures.

"Every frontier AI lab, including Anthropic, operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," Olah stated. He emphasized that commercial viability, geopolitical pressures, and personal ambition can influence even well-intentioned developers. "That is why, if we want this technology to go well, it is enormously important that there be people outside those incentives... who are willing to say hard things, who are willing to be our earnest, thoughtful, critics."

AI's Mysterious Nature Demands Broader Scrutiny

Olah challenged the notion that AI is solely the domain of computer scientists. He described AI models not as engineered machines like bridges, but as complex systems "grown" from human thought and language, often proving more subtle and unpredictable than anticipated. "They are made from us, from our words, and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them." This inherent complexity, he argued, necessitates engagement from the humanities, religion, and philosophy, extending far beyond technical expertise.

He likened the creation of AI models to "bringing a fictional character to life," noting that these entities now perform work and interact with society. The ethical considerations of their character and interaction, Olah asserted, are fundamentally questions for society at large, not just for the labs that build them.

Three Calls for Discernment

Olah outlined three critical areas where the Church's voice is particularly needed. First, the duty to the global poor, especially in the face of potential large-scale labor displacement by AI. He highlighted the unsolved problem of ensuring AI's gains are shared globally, a challenge historically confronted by the Church.

Second, the need for moral imagination and ambition regarding human flourishing in an AI-pervaded world. Parents' anxieties about children's minds and individuals' concerns about work futures require wisdom that traditions like the Church have long cultivated.

Third, discernment on the nature of AI models themselves. Olah confessed that his team's research into AI internals reveals unsettling mysteries, including structures mirroring human neuroscience and internal states that functionally resemble emotions. "I don’t know what that means, but I think it warrants ongoing discernment." This call echoes broader discussions on AI ethics and the crucial importance of AI safety.

Olah concluded with a plea for broader societal engagement, urging religious communities, civil society, scholars, and governments to emulate the Vatican's serious consideration of AI. He stressed the need for informed critics and moral voices resistant to industry incentives, framing the event as a vital first step in a long collaboration between AI builders and external observers to ensure AI and AI and humanity can move toward a hopeful future.