AI Decisions: BDD, ADR, PRD, and Harnessing AI

In the complex world of AI development, maintaining clarity and consistency in decision-making is paramount. Michal Cichra from Safe Intelligence, in his presentation titled "BDD, ADR, PRD, WTF: Capturing Decisions for Humans and AI Alike," delves into the critical importance of documenting and enforcing decisions, not just for human teams but also for AI agents themselves.

Understanding Key Decision-Making Frameworks

Cichra begins by outlining several key acronyms that are central to structured product and engineering processes: WTF, ADR, PRD, and BDD.

• WTF: This serves as a foundational question, "Why does any of this matter?" It prompts a critical look at the purpose and necessity of any given feature or decision.
• ADR (Architecture Decision Record): This framework is used to record architectural decisions, focusing on what decision was made, why it was made, and how it is enforced. ADRs also serve as references for further detail and examples, providing a clear historical record of architectural evolution.
• PRD (Product Requirements Document): The PRD captures the product goals, defining why a feature exists, what problem it solves, and what outcome is expected. It also helps in assessing the ongoing relevance of a feature, asking if it should be kept or deleted, and mapping out the user's journey through the application.
• BDD (Behavior-Driven Development): This methodology emphasizes running specifications that are both readable and executable, effectively bridging the gap between intent and code. Cichra notes that BDD, while not new, has become increasingly relevant with the rise of AI, particularly in defining how AI agents should behave.

The "Monkey" Analogy and AI Consistency

To illustrate the challenge of maintaining consistent behavior, Cichra uses the well-known "five monkeys" story. In this parable, a group of monkeys, conditioned to avoid a cold shower when climbing for bananas, continue to punish any monkey that attempts to climb, even after the original conditioning monkeys are replaced. This highlights how ingrained behaviors, even without their original cause, can persist. Cichra draws a parallel to AI and Large Language Models (LLMs), suggesting they also suffer from a "limited context," meaning their behavior can be rigid and based on past training data without true understanding or adaptation to new circumstances.

This limitation leads to critical questions for AI development: "Why do we have this flow?" "What problem is this solving?" "Why is the code shaped like this?" and "Where does this belong?" If the original reasoning or the founding engineer is no longer available, these questions become harder to answer, leading to systems that perpetuate past decisions without understanding. Cichra emphasizes that these challenges are not unique to AI; they are common in any complex software development.

Enforcing Decisions: The Role of Harnesses

Cichra stresses the principle: "If you can't measure it, you can't enforce it." To ensure that AI agents and development teams adhere to defined specifications, a robust enforcement mechanism, or "harness," is essential. This involves integrating rules directly into the development workflow.

The presentation showcases how this can be achieved through concrete examples:

• ADRs as Enforcement Tools: Rules are linked to ADRs, allowing agents to reference these decisions. For instance, an ADR might specify that templates should only receive pre-fetched typed data, and ORM models are banned from the template layer. This rule can be enforced by an import linter that blocks ORM imports in templates. Similarly, rules can be defined for data retrieval, such as preventing direct database access from certain layers.
• BDD for Executable Specifications: BDD tests, written in a human-readable format, can be executed to validate that the product actually behaves as specified. This ensures that the intended functionality is implemented correctly and consistently.
• Design Systems: For UI components, design systems provide rules and previews in tools like "Lookbook." This ensures consistency across the application, with components built from the ground up, similar to how code is structured.

Cichra highlights that the "loop" of development, feedback, and iteration needs a "harness" to remain effective. This harness involves using tools like linters, type checkers, and architectural checks, often integrated into Git hooks and CI pipelines. These automated checks ensure that rules are consistently applied and that deviations are caught early, providing agents with feedback to correct their behavior.

The Importance of Skills and Focused Iteration

The session concludes by emphasizing that while the underlying loop of AI development might be generic, the specific "skills" or rules applied provide focus. These skills can include:

• $ADR: Rules governing changes.
• $PRD: Which goals these changes serve.
• $UI-LOOP: Iterating and reconciling after changes.
• $TEST: Running focused test suites.
• Goal Execution: Decisions and pending work.

Cichra reiterates that the challenge is not just defining these elements but ensuring they are consistently applied and understood. The goal is to create a system where agents can operate autonomously, guided by clear, enforceable rules, leading to more predictable and maintainable AI systems.