Joe Reeve: Talking to Statues with AI

Joe Reeve, working in the growth organization at ElevenLabs, has developed an intriguing application that blurs the lines between history, technology, and interactive storytelling. Titled "Statue Scanner," the app allows users to point their phone's camera at any statue, monument, or painting, and then "talk" to the characters within using AI. This innovative tool leverages advanced AI capabilities to identify the subject of the statue, conduct deep research into its historical context, and then generate a voice for it using the ElevenLabs Voice Design API. The result is a unique and engaging way for people to interact with art and history.

From "How to Talk to Statues" to a Viral Hit

Reeve shared his experience building this application, noting that the initial concept came about on a Sunday afternoon. He managed to build a functional prototype in just two hours using Cursor, a code editor. After publishing a "one-shot prompt" and creating a video demonstration, the app quickly gained traction online. The video garnered over 1.5 million impressions, with many people expressing interest in collaborating. The app's ability to bring static objects to life through AI-powered conversation resonated widely, sparking conversations about the intersection of AI and culture.

The Technology Behind the Magic

The "Statue Scanner" app's functionality is built on a sophisticated pipeline of AI technologies. The process begins with the user taking a photo of a statue. This image is then processed by a multi-modal Large Language Model (LLM), which identifies the specific statue or monument. Following identification, the app performs "deep research" into the statue's historical context, gathering information about the person or subject it represents. This research is crucial for generating an accurate and contextually relevant voice. Finally, the app utilizes the Voice Design API to create a unique voice for the statue, and an "ElevenAgent" is created to facilitate the conversation, allowing users to "talk" to the historical figures.

Reeve highlighted the Voice Design API as a particularly powerful tool that is often underutilized. This API allows for the creation of highly expressive and professional-sounding voices, which are essential for bringing historical figures to life convincingly. The ability to generate custom voices with specific characteristics opens up a vast array of possibilities for interactive storytelling and educational experiences.

The Road to "ElevenHacks" and Beyond

The success of the "Statue Scanner" app, evidenced by its rapid virality and significant impressions, led to the creation of "ElevenHacks." This initiative encourages developers to build innovative projects using ElevenLabs' APIs, fostering a community of creators pushing the boundaries of AI. The platform hosts weekly challenges, offering prizes for the most creative and impactful applications.

Reeve's project not only showcases the capabilities of ElevenLabs' technology but also points towards a future where AI plays a significant role in how we learn about and interact with history and art. The ability to have a "conversation" with historical figures, even if simulated, offers a novel way to engage with the past, making it more accessible and relatable for a wider audience.