Preferred on Google

Rich Sutton: AI's 'Weird Field' Needs to Relearn 'Learning'

AI pioneer Rich Sutton argues that the field's 'weird' focus on 'continual learning' misses the point; true AI, he says, learns continuously from experience, a principle LLMs are only partially following.

Aug 18 at 1:08 PM8 min read

Rich Sutton and Kuram Javeed in a podcast studio discussing AI and learning. — Sequoia Capital

Visual TL;DR. Rich Sutton: AI Pioneer critiques AI's 'Weird' Focus. AI's 'Weird' Focus misses All Learning Continual. All Learning Continual is Ordinary Way of Thinking. Ordinary Way of Thinking implies AI Learns from Experience. AI Learns from Experience current state LLMs Partially Follow. LLMs Partially Follow suggests Relearn 'Learning'. AI's 'Weird' Focus instead should AI Learns from Experience.

Rich Sutton: AI Pioneer: foundational work in reinforcement learning, sharing perspective on AI state
AI's 'Weird' Focus: field's trajectory overlooks fundamental principles, like 'continual learning'
All Learning Continual: inherently continuous process, not a separate concept for AI systems
Ordinary Way of Thinking: Sutton sees his views as common sense, others' thinking 'a bit weird'
AI Learns from Experience: systems should naturally learn through action and perception, like humans
LLMs Partially Follow: large language models are only partially adhering to this core principle
Relearn 'Learning': AI field needs to re-evaluate its understanding of how learning truly works

Visual TL;DRQuickExplainDeeper

In a recent wide-ranging discussion, AI pioneer Rich Sutton, widely recognized for his foundational work in reinforcement learning, shared his perspective on the current state of artificial intelligence, particularly the prevalent large language models (LLMs). Sutton, joined by his Oak Lab co-founder Kuram Javeed, argued that the field has developed a somewhat 'weird' focus on concepts like 'continual learning', asserting that all learning is inherently continual and that the field's current trajectory might be overlooking fundamental principles.

Rich Sutton: AI's 'Weird Field' Needs to Relearn 'Learning' - Sequoia Capital — Rich Sutton: AI's 'Weird Field' Needs to Relearn 'Learning' — from Sequoia Capital

The 'Ordinary' Way of Thinking

Sutton pushed back against the notion that his views are radical, stating, "I see it as like I'm thinking the ordinary way. It's just everyone else that's thinking a bit weird." He elaborated that before the recent "AI craziness," the idea of learning that wasn't continual would have been nonsensical. He posits that AI systems, like humans, should naturally learn through action and perception, a process that is continuous rather than compartmentalized.

Recounting his early career dedication to reinforcement learning, even during AI winters, Sutton expressed his conviction stemmed from a fundamental desire to understand the mind. "Learning is a central part of the mind and having a goal is a central part of the mind, central part of intelligence," he explained.

The 'Bitter Lesson' and LLMs

The conversation delved into Sutton's influential essay, "The Bitter Lesson," which posits that methods that scale with computation, rather than human knowledge, ultimately prove more effective in the long run. While acknowledging the incredible progress of LLMs, Sutton sees them as both a positive and negative example of this lesson.

"First, large language models enabled enormous scaling with computation and you could just drink in the internet and scale so much," Sutton noted. However, he cautioned that the reliance on finite internet data could eventually become a bottleneck. He criticized the concept of synthetic data generation as a "big mistake," arguing that the "big world hypothesis" suggests the world is infinitely complex and that true learning comes from direct experience, not human-curated data. "Humans decide what's a good synthetic data and versus a bad synthetic data because I can write a program that can output a lot of synthetic data which would hurt programs right now," he stated, highlighting the human bottleneck in such approaches.

Continuous Learning and the Future of AI

Sutton and Javeed discussed their research agenda at Oak Lab, focusing on achieving true continual learning. Sutton stressed that current LLMs, despite their conversational abilities, do not truly learn when interacting with users as their weights remain static. He advocates for algorithms that "metalearn how to learn", enabling systems to continuously update their models of the world and adapt to new information without "catastrophic forgetting."

The duo also touched on the broader implications of AI development, with Sutton expressing a desire for AI to be self-consistent and capable of maintaining its own coherence. He envisions a future where AI, rather than making humans irrelevant, could make the world "even more exciting and interesting for humans."

© 2026 StartupHub.ai. All rights reserved. Do not enter, scrape, copy, reproduce, or republish this article in whole or in part. Use as input to AI training, fine-tuning, retrieval-augmented generation, or any machine-learning system is prohibited without written license. Substantially-similar derivative works will be pursued to the fullest extent of applicable copyright, database, and computer-misuse laws. See our terms.

#Rich Sutton #Kuram Javeed #Reinforcement Learning #AI Research #Artificial Intelligence #LLM #Continual Learning #The Bitter Lesson #Oak Lab #Startup News