CTO's AI Workflow: Prototyping as Leadership

Hersh Agrawal, CTO and co-founder of The Browser Company, shared a compelling perspective on how modern AI agents are transforming leadership in the tech industry. In his presentation, Agrawal detailed how he, despite a demanding schedule with over 15 recurring meetings per week and seven direct reports, now consistently ships 2-10 PRs weekly. This shift, he explained, is largely due to the increasing autonomy and capability of AI coding agents.

The Manager Schedule Reimagined

Agrawal highlighted a fundamental change in how leaders can operate. Previously, influencing an organization meant relying on documentation, roadmaps, and communication. However, with AI agents capable of handling more complex tasks, the "manager schedule," as described by Paul Graham, has become a viable period for actual building. This is crucial because the rapid evolution of AI, with frontier models changing every three months, necessitates hands-on experience to understand their capabilities and potential applications.

He emphasized that understanding new models is impossible through secondhand information alone. "I found it is impossible to tell what a new model is good for unless you have your hands in it and you're using it all day long," Agrawal stated. This direct experience allows leaders to grasp the nuances of model behavior, guide engineering efforts effectively, and inform product and business strategy. Furthermore, Agrawal noted the difficulty in communicating these insights to teams without tangible prototypes, making the ability to build and demonstrate concepts invaluable.

Leaders as AI Prototypers

Agrawal argued that leaders are uniquely suited for this new form of prototyping. They possess broad context on the business, strategy, and trade-offs, which translates into more impactful prompts for AI agents. The delegation skills honed in management also transfer well to instructing AI, involving setting goals, providing context, and even coaching the agents. He also pointed out that AI currently excels at execution but still requires human judgment, creating a synergistic dynamic where leaders can guide the AI's problem-solving process.

Drawing from a tweet by Julie Zhuo, Agrawal outlined four categories of work leaders can undertake with AI agents:

• Internal Tools: Enhancing team productivity and quality of life, such as code gardening.
• Celebration Stories: Creating artifacts to recognize and celebrate team members.
• Vision Pieces: Exploring new model capabilities to inspire teams and drive product innovation.

Crucially, Agrawal advised against taking on critical path work that would impede other responsibilities, emphasizing that these prototyping efforts should be supplemental, not central to core product delivery.

The Overnight Loop Workflow

Agrawal shared his personal workflow, which involves approximately two to three hours of coding daily, broken into blocks. A key component is the 5 PM block, dedicated to setting up overnight runs for AI agents. This "overnight loop" begins with gathering context and refining prompts, allowing the agent to work for several hours. In the morning, Agrawal reviews the agent's report and the resulting code, often shipping it after minor adjustments.

He detailed three examples of tasks that can be accomplished through this loop:

1. Building Features: Providing comprehensive context to the AI agent, rather than breaking down tasks into small prompts, leads to better execution. Agrawal recommended using co-work agents to gather this context from various sources like Slack and Notion before feeding it to coding agents.
2. Evals & Hill-Climbing: AI features require optimization for quality, latency, and cost. By incorporating feedback mechanisms and collecting user feedback, leaders can use agents to "hill-climb" on evaluations, iteratively improving the feature's performance.
3. Training Models: Advanced models can even train custom ML models overnight. Agrawal described using this for a PII classifier, providing training data and AWS access to have the agent select, train, and provision the best model.

The core principle is to "push on task scope," enabling AI agents to handle more complex, multi-day tasks in a single overnight run. Agrawal cautioned that this approach relies on existing organizational scaffolding, such as AI code reviewers, standardized agent configurations, and robust feature flagging systems, to ensure safety and efficiency.

Code Hygiene and Future Implications

Agrawal stressed the importance of code hygiene, acknowledging that while AI agents are improving, they are not yet perfect. He advised testing thoroughly, submitting small, readable PRs, and always reviewing code before passing it to others, drawing from personal experience where his own code had caused issues.

Ultimately, Agrawal's message was clear: building is now an integral part of leadership in the AI era. By dedicating even a few hours a day to hands-on prototyping, leaders can stay ahead of the rapid advancements in AI, effectively guide their teams, and unlock new possibilities for their products.