The landscape of artificial intelligence is currently experiencing a profound bifurcation, presenting developers and strategists with a critical decision: commit to open-source or closed-source solutions. This foundational choice, explored by Lauren McHugh Olende, Program Director at IBM, in her presentation, dictates everything from flexibility and customization to cost and deployment strategy across the entire AI stack. Researchers from Harvard Business School estimate the value of all open-source software, whose source code is publicly available and distributed freely, to be a staggering $8.8 trillion. Within AI specifically, many of the most exciting new features from commercial AI tools are rapidly recreated as open-source implementations, made by and distributed freely among the AI community.
Lauren McHugh Olende’s presentation delineates the key components of the AI stack, models, data, orchestration, and applications, and examines the trade-offs inherent in choosing open versus closed solutions at each layer. This exploration provides crucial insight for leaders aiming to architect robust and adaptable AI systems. The foundational decision of integrating open or closed AI into one's stack is arguably one of the most important a developer will make.
The central point of the AI stack is the model. Open-source models range from large language models (LLMs) to fine-tuned versions optimized for specific tasks like question-answering or specialized domains such as legal or biomedical imaging. Utilizing an open-source model necessitates implementing one's own inference engine to run these models, with popular options including Ollama for local execution or vLLM and TensorRT-LLM for server-side deployment. Conversely, closed models are typically accessed via an API, abstracting away the complexities of the inference engine, optimization, and underlying infrastructure. This offers convenience but sacrifices granular control.
The next critical layer is data, where the fundamental elements, data sources, connectors, conversion, Retrieval-Augmented Generation (RAG) pipelines, and vector databases, are similar for both open and closed approaches. The divergence lies in access and control. Open-source data tools and frameworks provide the ability to customize and adapt solutions to specific needs, given that the source code is freely available. This grants users complete control over deployment, whether on-premise or in a public cloud, ensuring data privacy and sovereignty. Closed solutions, on the other hand, are often part of commercial offerings, providing pre-built integrations but offering significantly less control over where private data resides or how it is processed.
Orchestration defines how an AI system breaks down complex problems into smaller, manageable tasks, encompassing reasoning, planning, execution of tool calls, and iterative reviewing to refine responses. For open-source solutions, developers leverage open-source agent frameworks like LangChain or LlamaIndex, which offer extensive customization over the agent's structure and behavior. This level of control allows for highly specialized and adaptable AI agents. In contrast, closed-source stacks provide commercial platforms that manage agentic tasks and orchestration through an API. While simpler to implement, these often involve an oversimplified structure, limiting the ability to customize the agent's logic to the same degree as an open-source framework.