Data is the Real AI Bottleneck, Say Flapping Airplanes Founders

In a compelling presentation at AI Ascent, Ben and Asher Spector, the founders of Flapping Airplanes, argued that data is the true bottleneck in the current AI landscape. They posited that the future of artificial intelligence lies not just in scaling compute power, but in achieving greater data efficiency, a paradigm shift that requires new systems and approaches.

Meet Flapping Airplanes' Founders

Ben Spector, a product founder with a cohort valued over $50 billion and experience with projects like Thunderkittens and Megakernels, dropped out of his PhD eight months before completion to co-found Flapping Airplanes. Asher Spector, his brother, brings a background in Cursor, Mercor, and Meta, along with a US debate championship title. He is currently the oldest person at their company at 26.

The Thesis: The Future is Data-Efficient

The Spector brothers presented a clear thesis: "The future is data-efficient." They highlighted that while current large language models (LLMs) excel at high-data tasks like search and coding, commanding vast markets valued in the trillions of dollars, there's a significant opportunity in domains with less readily available data. They illustrated this with a chart showing the massive amounts of data used by current LLMs, contrasting it with the potential for models that achieve similar or better results with significantly less data.

This drive for data efficiency is crucial because, as they pointed out, collecting frontier-quality data is inherently complicated. Unlike the relatively homogeneous compute market, the data market is fragmented and subject to regulations, terms of use, and privacy concerns. This makes acquiring and processing data for new AI algorithms a significant challenge.

The economic implications are substantial: a model that is 1000 times more data-efficient is also 1000 times easier to deploy into the economy. This efficiency is key to democratizing AI, moving beyond the current state where only a few companies can afford to train state-of-the-art models. They showed a visual representation of the world today, where only a few entities can train AI models, versus a data-efficient future where many more could participate.

The Approach: New Systems for New Algorithms

Flapping Airplanes' approach centers on building systems that enable these more data-efficient algorithms. They explained that while current frameworks like PyTorch are excellent at synthesizing single-threaded programming models and expressing parallel operations, they struggle to efficiently capture the nuances of more data-efficient algorithms. The Spectors showed a diagram illustrating how current frameworks map well to existing GPU capabilities, but there's a gap for novel, more data-efficient computations.

"We design data-efficient algorithms (we won't talk about these)," Asher Spector stated, noting that these are proprietary. "We build systems to enable these (we will talk about this)." Their focus is on creating new primitives for interacting with hardware, allowing developers to express and execute algorithms that are currently difficult or impossible to implement efficiently with existing tools.

They referenced historical advancements like DistBelief, Transformers, and FlashAttention as examples of how new primitives for hardware interaction have led to significant algorithmic breakthroughs. Their work aims to continue this trend by developing systems that can fully exploit the capabilities of modern hardware, particularly GPUs, in novel ways.

As a sneak peek, they showcased a visualization of their internal virtual machine, which allows them to efficiently manage and utilize GPU resources for their specialized workloads. This internal tooling is designed to support the development of their data-efficient algorithms, enabling them to "do weird stuff" with GPUs that is not typically possible with standard frameworks.

The Spectors concluded by emphasizing that their team comprises individuals with diverse and creative backgrounds, including a Clash of Clans world champion, a current high school student, an IMO perfect score, and 1T model enjoyers. They invited anyone who finds this mission exciting to connect with them, highlighting that they are looking for creative individuals who can help drive the next wave of AI innovation.