Jeff Dean: AI is a 'compression problem'

Jeff Dean, Chief Scientist at Google, shared his insights on the rapid evolution of artificial intelligence, future predictions, and advice for aspiring founders during a recent Startup School session. Dean, a pivotal figure behind foundational technologies like MapReduce, Bigtable, TensorFlow, and TPUs, discussed the current state of AI and its trajectory.

AI's Progress: Beyond Models to Systems

Reflecting on his past prediction that AI is now at the level of a junior engineer, Dean stated, "I feel like the models have been getting a lot better at sort of agent-based longer running coding tasks and it seems pretty clear that they are now actually pretty capable." He admitted to underestimating the pace at which AI capabilities would grow, particularly in complex tasks and agent-based systems across various domains.

The full discussion can be found on YC's YouTube channel.

Looking ahead to 2027, Dean predicts a significant increase in the automation of ML systems themselves. "I think you will see a lot more automation of ML systems themselves," he stated, envisioning systems that improve their own capabilities through automated experimentation, breaking down problems into sub-problems, and iterating in a closed loop to refine performance. He believes this approach will extend beyond ML to other scientific and engineering fields where measurable objectives can be defined.

The 'Fits the Memory' Moment of 2026: Hardware Specialization

Drawing a parallel to Google Search's historical shift from hard drives to RAM, Dean identified a similar inflection point for AI hardware in 2026. He emphasized the growing importance of high-performance and low-energy inference hardware. "I think everyone is now realizing that inference is the key to making you know these agent-based systems be available to more and more people and that latency is really important," Dean explained. He highlighted that specialization of hardware is a key strategy for achieving greater energy efficiency and lower latency compared to general-purpose devices like GPUs or TPUs.

Rethinking Assumptions: The Power of Long-Running Agents and Energy Metrics

Dean challenged a common assumption that AI systems are limited to short-term tasks. He asserted that agent-based systems are increasingly capable of running for days or weeks to solve highly complex problems. He also touched upon the shift in focus within the tech industry, noting that energy has become a primary unit of measurement for calculations, with data movement being a thousand times more costly in terms of energy than computation itself. This reality, he noted, shapes many decisions in building AI systems and explains practices like batching, which is crucial for hardware efficiency but can be detrimental to low-latency applications.

The Future of AI: Beyond Parameter Count

Dean observed that AI progress is no longer solely defined by bigger models with more parameters or data. Instead, advancements are increasingly driven by the surrounding systems, including retrieval tools, memory, and agent capabilities, often consolidated under the umbrella of "context engineering." He elaborated, "The model is really only one piece of what you're trying to do, which is build an overall system that can solve really interesting problems." This involves models that know how to use tools, retrieve information, maintain a history, and integrate information within the context of a specific problem. He emphasized that this context is clearer to the model than the vast, mixed data it was trained on.

Finding Startup Opportunities in the AI Era

When asked about where startups can win in the current AI landscape, Dean advised founders to focus on areas where general models currently struggle or fail entirely. "If they're completely failing, that's probably a good sign," he noted. He suggested looking for opportunities that involve access to specific types of data that general models might not have, such as personal information organization. Alternatively, he pointed to highly complex problems that could benefit from specialized models trained on niche datasets, citing AlphaFold as a successful example in protein folding. Dean encouraged founders to "pick something you're super excited about and want to build and you think would be useful in the world."

The Scarce Skill: Taste and Critical Thinking

Dean stressed that as AI becomes more capable of assisting with coding and other tasks, the most scarce skill will be human "taste", the ability to discern which problems are worth solving and to steer AI-assisted computation effectively. He advised founders to revisit fundamental assumptions, even in established fields like chip design, to foster new ideas and breakthroughs.