Joe Reeve, working in the growth organization at ElevenLabs, has developed an intriguing application that blurs the lines between history, technology, and interactive storytelling. Titled "Statue Scanner," the app allows users to point their phone's camera at any statue, monument, or painting, and then "talk" to the characters within using AI. This innovative tool leverages advanced AI capabilities to identify the subject of the statue, conduct deep research into its historical context, and then generate a voice for it using the ElevenLabs Voice Design API. The result is a unique and engaging way for people to interact with art and history.
Related startups
From "How to Talk to Statues" to a Viral Hit
Reeve shared his experience building this application, noting that the initial concept came about on a Sunday afternoon. He managed to build a functional prototype in just two hours using Cursor, a code editor. After publishing a "one-shot prompt" and creating a video demonstration, the app quickly gained traction online. The video garnered over 1.5 million impressions, with many people expressing interest in collaborating. The app's ability to bring static objects to life through AI-powered conversation resonated widely, sparking conversations about the intersection of AI and culture.
The Technology Behind the Magic
The "Statue Scanner" app's functionality is built on a sophisticated pipeline of AI technologies. The process begins with the user taking a photo of a statue. This image is then processed by a multi-modal Large Language Model (LLM), which identifies the specific statue or monument. Following identification, the app performs "deep research" into the statue's historical context, gathering information about the person or subject it represents. This research is crucial for generating an accurate and contextually relevant voice. Finally, the app utilizes the Voice Design API to create a unique voice for the statue, and an "ElevenAgent" is created to facilitate the conversation, allowing users to "talk" to the historical figures.
Reeve highlighted the Voice Design API as a particularly powerful tool that is often underutilized. This API allows for the creation of highly expressive and professional-sounding voices, which are essential for bringing historical figures to life convincingly. The ability to generate custom voices with specific characteristics opens up a vast array of possibilities for interactive storytelling and educational experiences.
