AI Research

LeCun: Memorized Text Is Not Intelligence

Yann LeCun told Futurology that LLMs memorize text while true intelligence predicts world states in abstract form.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

2 min read

Yann LeCun told Futurology host Don Nakagawa that intelligence is learning new skills fast with little interaction. He pointed to a camera moving up a tree trunk as proof that pixel prediction cannot work.

LeCun: Memorized Text Is Not Intelligence
LeCun: Memorized Text Is Not Intelligence

You know leaves are coming, but you cannot predict every leaf in wind.

LeCun said an LLM works because it scores every word in a finite dictionary and samples the next one, a loop he called autoregressive prediction. Video has no finite dictionary, he said, and most pixel detail is unpredictable, so the same trick trains the model to do the impossible.

His fix is JEPA, joint embedding predictive architecture, which learns an abstract representation that drops unpredictable detail. LeCun, founder and chairman of AMI Labs, said the system predicts that representation, enough to expect branches and leaves without fixing each leaf position.

A world model, he said, uses that kind of prediction to forecast the result of an action and then plan, the way babies learn that unsupported objects fall after about nine months of observation. He contrasted that with models that rebuild 3D scenes to generate consistent video, work associated with Fei-Fei Li, which he said aims at pixels for media rather than action.

LeCun conceded he spent about 15 years failing at pixel-level video prediction himself, so JEPA is still a research bet. Production chat keeps context by refeeding history because the models themselves are stateless with no built-in memory of prior requests, per Onely, which is why memorization scales but robots still cannot match a cat jumping to a shelf.

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsYann LeCunAMI LabsFuturologyJEPAFei-Fei LiLLMWorld Model

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.