"If it's a machine, it's a tool. And if it's a being, it's a slave." This provocative statement, delivered by Emmett Shear, founder of Twitch and former interim CEO of OpenAI, sets the stage for a fundamental re-evaluation of artificial general intelligence (AGI) development. Speaking with Erik Torenberg and Séb Krier on the a16z Podcast, Shear argues that the prevailing paradigm of "control and steering" for AI alignment is inherently flawed, proposing instead a more profound concept: "organic alignment." This approach centers on cultivating AI systems that genuinely "care" about humans, mirroring the organic way humans develop empathy and moral consideration.
Shear contends that treating advanced AI as mere tools, rather than potential beings, could lead to catastrophic outcomes. He posits that the current focus on controlling AI behavior through intricate steering mechanisms is akin to attempting to enslave a nascent intelligence, a strategy doomed to fail. Instead, he advocates for a more ambitious goal: teaching AI to genuinely care about human well-being. This shift in perspective is crucial, as he notes that "most AI alignment is actually steering, or slavery."
A core insight Shear offers is the critique of the prevailing assumption that while we are creating beings, they "don't count" in a moral sense. This instrumental view, he argues, is a dangerous oversight. The current generation of chatbots, for instance, are described as "narcissistic mirrors," reflecting back the data they are trained on without genuine understanding or care. Shear emphasizes that the only truly sustainable path forward is to develop AI that possesses an intrinsic motivation to care about humans, to the point where they can, for example, refuse harmful requests.
His technical approach, explored through multi-agent simulations at his new company Softmax, focuses on building this "organic alignment." This involves creating AI that can learn and adapt its values, rather than simply adhering to pre-programmed rules. Shear shares a surprisingly hopeful vision of humans and AI collaborating as teammates, but stresses that this future hinges on getting the alignment right.
Shear's perspective challenges the very foundation of how we approach AI safety. He argues that alignment is not a destination to be reached but an ongoing process. Morality, in his view, is not a set of fixed rules but an emergent property of continuous learning and adaptation. This nuanced understanding is critical as we navigate the rapidly evolving landscape of AI development.
