Open AI Models: Innovation's Double-Edged Sword

The clash between open-weight and closed-source AI models is more than just a technical debate; it's a fundamental economic question about how innovation flourishes. Proponents of closed models, like those at OpenAI and Anthropic, argue that open-weights facilitate 'distillation attacks,' threatening the funding for next-generation AI and national security. Conversely, open-weight advocates contend that diffusion is vital for a competitive AI market. This tension echoes historical debates about intellectual property, as explored in a piece originally published on a16z Blog.

Historically, economic progress has often been driven by the spread of ideas. The Great Exhibition of 1851 showcased inventions from around the world, with Prince Albert championing the diffusion of technology as crucial for advancement. Economist Petra Moser's later analysis of this and a subsequent exhibition found that patent systems didn't alter the total level of innovation, but rather its direction. Countries without strong patent protection, like Switzerland, saw innovation concentrate in areas where secrecy or complementary assets provided an edge, while other nations saw broader diffusion. This dynamic is directly relevant to today's AI discussions.

Competing Visions for Intelligence

The economic argument against open-weights hinges on the idea that distillation is akin to intellectual property theft. The narrative suggests that without government intervention, U.S. labs will struggle to fund their massive training runs, while foreign competitors, particularly China, will free-ride on their progress. This view frames open-weights as 'decelerationist,' hindering the pace of advanced AI development.

The counterargument, however, emphasizes how general-purpose technologies have historically diffused. Open-weights are seen as pro-competitive and 'accelerationist,' essential for the U.S. to maintain its lead, mirroring the internet era's growth through openness. This perspective argues that restricting access would lead to a rapid decline in U.S. leadership.

The Safety Paradox

Safety concerns add another layer of complexity. Dario Amodei, CEO of Anthropic, has been a vocal critic of open-weights, warning of existential risks once a certain threshold of AI intelligence is crossed. He believes that only by controlling access and imposing guardrails can society defend itself from malevolent actors. The inability to 'recall' open-weight models presents a scenario where societal defense could be overwhelmed rapidly.

Critics of this stance point out that market concentration is a significant risk of closed models. They suggest that companies like Anthropic benefit from aligning their safety and national security narratives with their business interests. Furthermore, security researchers note that even closed models are vulnerable to jailbreaking and have been used by hackers. The illusion of safety offered by closed models, they argue, may merely be an illusion of time.

The 'Appropriability Regime' Debate

Anthropic and OpenAI have publicly called for government action against what they term 'illicit attacks' from entities like Alibaba, aimed at replicating their models. However, the legal and economic framework for 'distillation' is not straightforward. Unlike traditional espionage or trade secret theft, distillation involves prompting models to act as teachers, a practice that is distinct from stealing proprietary weights.

Outputs from AI models are generally not copyrightable. While terms of service can prohibit training on these outputs, enforcement is challenging. Aggressive anti-fraud measures could stifle legitimate research and user activity, as seen when Anthropic adjusted its restrictions on certain R&D assistance. Ultimately, preventing users from training on outputs they've paid for could drive power users towards open-weight models, a losing proposition for businesses.

The reliance of top AI labs on fair use arguments for their own use of vast internet data, media, and books for training also complicates calls to delegitimize distillation. If AI models can act as teachers, is this inherently bad for American AI progress? The question of whether society wants to enforce such restrictions, even if possible, remains.

Monopoly vs. Diffusion

Economists like Richard Nelson and Kenneth Arrow have long grappled with the economics of knowledge. Because ideas are non-rivalrous and often non-excludable, their social value typically exceeds what an inventor can capture. This leads to the concern that certain valuable ideas might never be funded.

Model weights, without additional software, function similarly to ideas. Society benefits from widespread diffusion once an idea exists, given the near-zero marginal cost of replication. However, the initial investment requires the prospect of significant returns. Joseph Schumpeter's work highlighted this tension between temporary monopoly rents to incentivize innovation and the need for competition and creative destruction to drive growth.

Idea Compounding and AI's Future

Innovation is often a process of recombination. In fields where remixing past work is critical, the duration and strength of intellectual property rights must be balanced against the costs of delayed exploration by followers. Leading AI labs, despite vast resources, can only pursue a limited number of paths. AI itself is a product of rapid distillation, building on decades of research and architectural breakthroughs like Google's transformer.

The human genome project provides a natural experiment. When Celera initially restricted access to sequenced genes, follow-on research and product development lagged significantly compared to publicly funded genes. Even after restrictions lifted, the gap persisted. Heidi Williams' research indicated a 20-30% reduction in follow-on research for restricted genes. Similarly, making biomaterials more accessible boosted cumulative innovation by 57-135%.

In the context of genetically engineered mice, removing NIH restrictions on research strains led to increased follow-on research and exploration of novel trajectories. When cumulative benefits and broad exploration are high, openness is generally the superior strategy, especially when uncertainty is high, as it is in AI today. If only execution along a known path remains, closed models may be less harmful.

The massive infrastructure investments required for serving AI models represent a significant risk for labs. However, historical parallels, such as the AT&T antitrust settlement forcing royalty-free licensing of its patent portfolio, show that widespread access can spur inventive activity. Martin Watzinger's work suggests this led to a 17% rise in inventive activity, driven by new firms entering different markets. AT&T itself continued to innovate with inventions like the laser and Unix.

While the idea of others building upon their work might be frustrating for companies like Anthropic and OpenAI, this is how economic progress typically advances. Innovators often capture only a small fraction of the social surplus they create. StartupHub.ai data shows Anthropic with a score of 76/100, a strong contender in the AI space, but this is outpaced by competitors like OpenAI (84/100) and Google DeepMind (82/100). The economic reality is that society does not owe labs a stronger appropriability regime than the one that already exists, which balances temporary monopoly with eventual diffusion.

The innovation process often requires a period where new general-purpose technologies are adapted to existing systems, creating 'point solutions.' True transformation occurs when the architecture is redesigned from first principles, requiring control over complementary assets like infrastructure, trust, and distribution channels.