# Tejal Patwardhan: Stop Underestimating AI Models _Tejal Patwardhan of OpenAI discusses the evolution of AI evaluation, the concept of 'capability overhang,' and the need for realistic, real-world benchmarks._ **Published:** 2026-06-16 **Source:** https://www.startuphub.ai/ai-news/artificial-intelligence/2026/tejal-patwardhan-stop-underestimating-ai-models --- In the latest episode of The OpenAI Podcast, host Andrew M. interviews Tejal Patwardhan, a researcher on OpenAI's alignment team. Patwardhan, who joined the organization in Fall 2023, discusses the critical need to stop underestimating the capabilities of AI models and the importance of developing relevant, real-world benchmarks to measure their progress. Tejal PatwardhanCore From the article 7 mentionsIn the latest episode of The OpenAI Podcast, host Andrew M. interviews Tejal Patwardhan, a researcher on OpenAI's alignment team.discussesAI Models Evolving FastDriverAI models develop capabilities faster than humans can measureleads toCapability OverhangContextGap between AI skills and human understanding/adoptionFrom the articlePatwardhan introduces the concept of "capability overhang," a phenomenon where AI models develop capabilities significantly faster than humans can measure or adopt them.causesCurrent Benchmarks LimitedDriverMathematical benchmarks fall short of real-world nuancesFrom the articleThe goal is to ensure that the benchmarks not only measure current capabilities but also anticipate future advancements and potential applications.highlights need forNeed Realistic BenchmarksDriverDevelop relevant, real-world benchmarks for progress measurementFrom the article 2 mentionsPatwardhan, who joined the organization in Fall 2023, discusses the critical need to stop underestimating the capabilities of AI models and the importance of developing relevant, real-world benchmarks to measure their progress.Stop Underestimating AIOutcomeAccurate evaluation prevents underestimation of AI capabilitiesFrom the articlePatwardhan, who joined the organization in Fall 2023, discusses the critical need to stop underestimating the capabilities of AI models and the importance of developing relevant, real-world benchmarks to measure their progress.Continuous EvaluationContextImportance of ongoing assessment and adaptation of AIFrom the article 5 mentionsPatwardhan stresses the importance of a continuous feedback loop, where new benchmarks and evaluation techniques are developed and refined in parallel with model advancements. ## Understanding "Capability Overhang" Patwardhan introduces the concept of "capability overhang," a phenomenon where AI models develop capabilities significantly faster than humans can measure or adopt them. This gap creates a situation where models might possess advanced skills that are not yet fully understood or integrated into practical applications. She highlights that while mathematical benchmarks offer a starting point for evaluation, they often fall short of capturing the nuanced performance of AI in real-world scenarios. The full discussion can be found on **OpenAI Youtube**'s YouTube channel. ![](https://img.youtube.com/vi/CFqjjKp9Y-Q/maxresdefault.jpg) Why Tejal Patwardhan stopped underestimating the models - Episode 21, from OpenAI Youtube ## The Evolution of AI Benchmarking The conversation delves into the evolution of AI benchmarking, noting how early evaluations often relied on simplified tasks. As models have become more sophisticated, the need for more complex and relevant benchmarks has become apparent. Patwardhan explains that simply measuring performance on tasks like basic math problems is no longer sufficient. Instead, evaluators must consider how models perform in more intricate domains, such as scientific reasoning, coding, and understanding real-world contexts. She elaborates on the challenges of creating these more sophisticated evaluations, emphasizing that they need to be both accurate and adaptable. The goal is to ensure that the benchmarks not only measure current capabilities but also anticipate future advancements and potential applications. ## From Theory to Practice: Real-World Relevance A key point raised by Patwardhan is the transition from theoretical capabilities to practical, real-world utility. She points out that even if a model can perform a task exceptionally well in a controlled environment, its true value is only realized when it can be reliably and safely applied in various real-world situations. This transition often involves overcoming significant hurdles, including ethical considerations, safety protocols, and the potential for unintended consequences. Patwardhan shares her experience in developing and measuring these real-world capabilities. She notes that the initial stages of her work at OpenAI involved focusing on the preparations for future AI models, aiming to understand their potential impact and ensure their alignment with human values. This includes developing new ways to evaluate models that go beyond traditional metrics and capture a more holistic understanding of their performance. ## The Importance of Continuous Evaluation and Adaptation The discussion underscores the dynamic nature of AI development. As models continue to improve at an unprecedented pace, the methods for evaluating them must also evolve. Patwardhan stresses the importance of a continuous feedback loop, where new benchmarks and evaluation techniques are developed and refined in parallel with model advancements. This iterative process is essential for identifying potential risks, ensuring safety, and ultimately building AI systems that are beneficial to humanity. She concludes by emphasizing that the field is still in its early stages, and much work remains to be done in understanding and guiding the development of advanced AI. The focus, she suggests, should be on creating robust, adaptable, and ethically sound evaluation frameworks that can keep pace with the rapid progress of AI research and development. --- Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.