Artificial Intelligence

Applied Vertical AI: From Trading to Drug Discovery

Ayush Bhardwaj of Allos AI outlines a 7-step process for building applied vertical AI, stressing the importance of proprietary data and domain expertise.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

4 min read

Ayush Bhardwaj speaking at a podium at the AI Engineer World's Fair.
Credit: AI Engineer
Visual TL;DR
Vertical AIContext
AI built for a single industry, simulating job functions of people within that sector
From the article 5 mentionsAyush Bhardwaj, who has experience in applied AI for both hedge funds and a pharma-tech startup, shared his insights on building and iterating applied vertical AI at a recent AI Engineer World's Fair event.
Data ScarcityDriver
common challenge in specialized fields, requiring human-in-the-loop for data generation
From the article 7 mentionsThe scarcity of crucial data, particularly negative results in pharma or proprietary trade theses in finance, was identified as a major challenge.
7-Step RecipeCore
Ayush Bhardwaj outlines a structured process for building applied vertical AI solutions
From the articleThis observation led him to abstract a common seven-step recipe for developing these specialized AI agents.
Human LoopEffect
iterative process where human feedback refines models and generates more training data
From the article 5 mentionsBhardwaj concluded by emphasizing that while 89% of enterprise AI agents may never reach production, those that do succeed often keep a domain expert in the loop.
Domain ExpertiseCore
critical for understanding industry nuances and building effective, specialized AI models
From the article 3 mentionsBhardwaj highlighted that while the technical aspects of building AI agents (tools, prompts, observability) are becoming commoditized, the true competitive advantage lies in domain expertise and proprietary data.
Proprietary DataCore
essential for creating a 'moat' and competitive advantage in vertical AI applications
From the article 7 mentionsIdentify the data: Bhardwaj stressed that proprietary data is the true moat.
Industry-Specific SolutionsOutcome
AI tailored for sectors like trading or drug discovery, addressing unique challenges
From the article 3 mentionsHe contrasted this with general-purpose AI like Google Translate, highlighting that vertical AI solutions, such as those used for drug discovery at his former company Allos AI, are highly specific in their application.
Competitive MoatOutcome
proprietary data and domain expertise create a defensible advantage for vertical AI
From the article 3 mentionsBhardwaj noted that companies often keep this data private, as it represents their competitive edge.
Contents(4)

In the rapidly evolving world of artificial intelligence, the focus is shifting towards specialized applications tailored for specific industries. Ayush Bhardwaj, who has experience in applied AI for both hedge funds and a pharma-tech startup, shared his insights on building and iterating applied vertical AI at a recent AI Engineer World's Fair event.

Applied Vertical AI: From Trading to Drug Discovery - AI Engineer
Applied Vertical AI: From Trading to Drug Discovery, AI Engineer

The Essence of Applied Vertical AI

Bhardwaj defined applied vertical AI as AI built for a single industry, aiming to simulate the job functions of people within that sector. He contrasted this with general-purpose AI like Google Translate, highlighting that vertical AI solutions, such as those used for drug discovery at his former company Allos AI, are highly specific in their application.

He noted that despite the vastly different timelines and risk tolerances between the fast-paced hedge fund world (where speed and 'mostly right' is acceptable) and the pharma industry (where accuracy over 15 years is paramount), the core process of building applied AI remained surprisingly similar. This observation led him to abstract a common seven-step recipe for developing these specialized AI agents.

The Seven-Step Recipe for Applied Vertical AI

Bhardwaj outlined the key steps involved in building and iterating applied vertical AI:

  1. Formulate the problem: Emphasizing the need for a narrow focus, he advised against trying to solve too many problems at once. Specificity is key, such as identifying a particular market and industry, and then defining a precise task for the AI agent.
  2. Identify the data: Bhardwaj stressed that proprietary data is the true moat. While public data is accessible, a company's unique, curated datasets are what will differentiate its AI solutions. He suggested that leveraging existing unstructured data within an organization and converting it to structured data using LLMs is a feasible approach.
  3. Write the prompts: The prompts should be designed to mirror how a human expert would solve the problem, breaking down the task into logical steps.
  4. Observe: Implementing observability is critical to track the AI's actions, understand its outputs, and debug effectively.
  5. Hire the user: Perhaps the most critical step, Bhardwaj asserted the necessity of bringing domain experts into the AI development process. These experts are vital for data curation, prompt refinement, and ultimately, for judging the AI's performance.
  6. Iterate: The process is iterative, requiring continuous refinement based on observed performance and expert feedback.
  7. Ship: Once the AI agent is performing effectively and demonstrating ROI, it's time to deploy it.

The 'Moat' and the Role of Domain Expertise

Bhardwaj highlighted that while the technical aspects of building AI agents (tools, prompts, observability) are becoming commoditized, the true competitive advantage lies in domain expertise and proprietary data. He pointed out that many vertical AI projects fail because they cannot accurately assess the performance of their models without this specialized knowledge.

He shared a personal anecdote about struggling to evaluate the output of his AI agents in finance and biology, realizing that an LLM could not replace the nuanced judgment of a human expert who understands concepts like 'alpha' or biological significance.

Data Scarcity and the Human Loop

The scarcity of crucial data, particularly negative results in pharma or proprietary trade theses in finance, was identified as a major challenge. Bhardwaj noted that companies often keep this data private, as it represents their competitive edge. He also touched upon the importance of turning expert judgment into reusable signals through methods like supervised fine-tuning, reinforcement learning from human feedback, rubrics as rewards, and error analysis.

Bhardwaj concluded by emphasizing that while 89% of enterprise AI agents may never reach production, those that do succeed often keep a domain expert in the loop. This human-in-the-loop approach, or more accurately 'AI in the loop' for specialized industries, is essential for ensuring AI models can reason from causation rather than just correlation, ultimately leading to more effective and valuable solutions.

TopicsAyush BhardwajAllos AIApplied AIVertical AIAI DevelopmentDomain ExpertiseProprietary DataMachine Learning

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.