Firework CEO: Post-Training is Key to Unique AI Business

In a recent talk, Lin Qiao, CEO and co-founder of Firework, a specialized intelligence platform, emphasized the critical role of post-training in building durable AI-powered businesses. Qiao highlighted how the collapsing of resource requirements in application development, driven by advancements in AI, is shifting companies from relying on off-the-shelf APIs to building more deeply differentiated products.

Owning Intelligence, Not Renting

Qiao argued that in the current AI landscape, where open models are gaining traction, the depth of an alliance between companies stems from a shared belief in the industry's potential. She stated, "Every single company exists for a reason because they focus on solving a unique problem in a special way. And that means they carry their own judgment, taste, and determination, conviction into that product." Building on this, she believes that companies must bake their unique judgment and customer understanding into the intelligence they build, rather than relying solely on generic APIs.

Owning one's intelligence, according to Qiao, starts with data. This includes curating high-quality production data and generating synthetic data to enrich existing datasets. The next step involves using this data to build models and own the weights, employing a collection of techniques tailored to specific problems. Finally, serving these models effectively, through A/B testing and iterative loops, is crucial for success.

The Post-Training Progression

Qiao outlined a progression for AI development, starting with simple prompting, then moving to Retrieval Augmented Generation (RAG) to ground AI in a company's own data. This is followed by supervised fine-tuning (SFT) or LoRA for teaching format and behavior, preference tuning (like DPO) for personalizing taste and tone, and finally reinforcement learning (RL) for optimizing on a reward signal. She noted that while SFT transfers new knowledge, RL sharpens behavior, and that most teams eventually incorporate RL.

Avoiding Common Pitfalls

Qiao also highlighted six common ways teams burn time and money in post-training:

• Data quantity over quality: Pristine examples are more valuable than noisy ones.
• No eval before training: If you can't measure improvement, you can't ship it.
• Sloppy RL environments: Imperfect training can lead to shortcuts that break in production.
• Training-serving drift: Small gaps in compounds can lead to significant drift over time.
• Wrong tool for the job: Fine-tuning is for facts, RL is for behavior.
• Building the car vs. tuning the engine: Focus on data and reward signals first.

Real-World Successes

Qiao showcased several examples of companies successfully leveraging post-training on Fireworks' platform. Cursor, a coding assistant, has continuously released new models, competing at the frontier of quality. Doximity, a healthcare AI company, topped the Stanford-Harvard clinical safety benchmark. Factory, a security-focused company, tuned a model that excelled in risk detection.

These examples, Qiao emphasized, demonstrate that by focusing on custom data and leveraging open models, companies can build specialized intelligence that outperforms even closed frontier models in quality, cost, and speed.