# LLM Reliability: Control Flow Over Prompting _Ornella Bahidika and Joel Allou from Microsoft discuss 'harness engineering' for AI agents, advocating for code-driven control flow over LLM-driven decision-making to ensure reliability._ **Published:** 2026-07-20 **Source:** https://www.startuphub.ai/ai-news/artificial-intelligence/2026/llm-reliability-control-flow-over-prompting --- In the rapidly evolving world of AI agents, achieving consistent and reliable performance is a significant challenge. Ornella Bahidika and Joel Allou, presenting a concept they call '[harness engineering](/ai-news/artificial-intelligence/2026/google-deepmind-s-philipp-schmid-on-testing-ai-agent-skills),' argue that the common approach of 'prompting harder' is insufficient for building dependable multi-step agents. Instead, they advocate for a shift in architectural thinking, where the control flow of an agent is managed by code, effectively acting as a director to the LLM's talent. LLM Agent ReliabilityDriver multi-step AI agents often fail midway, skip steps, or enter loopsFrom the article 4 mentionsInstead, they advocate for a shift in architectural thinking, where the control flow of an agent is managed by code, effectively acting as a director to the LLM's talent.Prompting HarderDrivercommon approach of trying to improve LLM performance with more detailed promptsFrom the article 2 mentionsOrnella Bahidika and Joel Allou, presenting a concept they call 'harness engineering,' argue that the common approach of 'prompting harder' is insufficient for building dependable multi-step agents.LLM State ManagementDriverLLMs struggle to track progress through tasks, lacking internal state awarenessFrom the article 3 mentionsThey propose that the fundamental issue is not the LLM's inherent reasoning but its lack of state management capabilities.Harness EngineeringCorearchitectural shift where code manages agent control flow, directing the LLMFrom the article 6 mentionsOrnella Bahidika and Joel Allou, presenting a concept they call 'harness engineering,' argue that the common approach of 'prompting harder' is insufficient for building dependable multi-step agents.Code-Driven ControlCoreexplicit code dictates agent steps and decisions, acting as a directorFrom the article 6 mentionsThis approach ensures that the LLM is never in control of the overall process, preventing the common failure modes of agents that rely solely on LLM-driven logic.Ace Voice TutorContextlive AI voice tutor project demonstrating harness engineering principles in practiceFrom the article 2 mentionsThe core of their argument centers on a project called Ace, a live AI voice tutor.enablesReliable AI AgentsOutcomeachieving consistent and dependable performance for complex multi-step tasksFrom the article 9+ mentionsIn the rapidly evolving world of AI agents, achieving consistent and reliable performance is a significant challenge. The core of their argument centers on a project called Ace, a live AI voice tutor. Bahidika and Allou highlight a common problem encountered when deploying complex agents: they fail midway through tasks, skip steps, or enter loops, issues often hidden in polished demos. They propose that the fundamental issue is not the LLM's inherent reasoning but its lack of state management capabilities. LLMs are excellent at generating responses for a given prompt, but they struggle to track their progress through a sequence of operations. ## The 'Harness' as a Director To address this, Bahidika and Allou describe a lesson as a 'small state machine.' This state machine defines the distinct steps of a lesson, such as introduction, teaching, checking comprehension, grading, advancing, and wrapping up. Each step sends the LLM a 'narrow contract', a specific instruction to perform a single task and return a result. The 'harness,' which is the surrounding code, then validates the LLM's output, advances the state machine, and determines the next action. This approach ensures that the LLM is never in control of the overall process, preventing the common failure modes of agents that rely solely on LLM-driven logic. Allou elaborated on this by contrasting the use of large, powerful models like Anthropic's Opus with smaller, more efficient models. He explained that by implementing a 'harness,' they could utilize a model like Anthropic's Haiku, which is less computationally intensive and faster, while still achieving the desired reliability and performance for their AI tutor. This is because the harness handles the complex decision-making and state tracking, allowing the LLM to focus on its core strength: generating natural language responses. ## Key Decisions Belong to Code The presentation emphasized that critical decisions within an AI agent's workflow should not be left to the LLM. These include determining when a task is complete, assessing if a user has achieved a learning objective, and deciding what the next step should be. Bahidika and Allou assert that these are 'code' problems, not 'prompting' problems. By engineering these control flows outside the LLM, developers can build more predictable and reliable systems. The video showcased logs demonstrating this 'harnessing' in action, illustrating how specific steps in the AI tutor's lesson were managed by the harness, including providing input for what to speak about, drawing on a whiteboard, clearing queues, and ending the lesson. This meticulous control ensures that even with novel scenarios, the agent can proceed reliably. ## The 'Coin Flip' Rule A key takeaway, summarized as 'When reliability is a coin flip, take control flow out of the model,' highlights the practical advice for developers. If an agent's performance is inconsistent, the solution is not to endlessly refine prompts but to externalize the control logic. The core message is to 'Let the model talk, but you keep the wheel.' This means the LLM can generate content and respond to inputs, but the surrounding code dictates the sequence, conditions, and overall execution path, thereby ensuring reliability and efficiency. This approach is presented as broadly applicable, extending beyond voice tutors to coding agents, operational runbooks, and onboarding flows, anywhere that reliable, multi-step execution is required. --- Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.