AI UX is Broken, Not the Model

In the rapidly evolving world of artificial intelligence, the user experience (UX) of AI-powered applications is often a point of friction. Mike Christensen, a staff engineer at Ably, delivered a compelling presentation at AI Engineer Europe titled "Why Your AI UX Is Broken (and It's Not the Model's Fault)." Christensen argued that the common perception of AI UX issues stemming from flawed models is often a misdiagnosis, with the real culprits lying in the fundamental architecture and implementation of these systems.

The Default Architecture: HTTP Streaming

Christensen began by outlining the most common architectural pattern for AI chat applications: HTTP streaming. In this model, a client (like a browser) sends a message to an agent (server-side), which then prompts a large language model (LLM). The LLM generates an event stream, which is then sent back to the client via a single point-to-point connection. While this setup is straightforward to implement, Christensen highlighted its inherent limitations.

The primary issue with this default approach, according to Christensen, is that it creates a brittle and often frustrating user experience. When network connections falter or users switch between devices, the experience breaks down. He elaborated, "The default architecture is fundamentally oriented on the idea of a single client, a single connection, to a single agent." This one-to-one mapping creates significant challenges when dealing with the dynamic nature of user interaction in modern applications.

The Problems with HTTP Streaming

Christensen identified three key problems arising from the reliance on HTTP streaming for AI UX:

• Resilient Delivery: Streams built on this model often falter when disconnections occur. Unlike traditional web applications where a user might refresh a page and pick up where they left off, AI streams are typically lost, forcing users to restart conversations or tasks.
• Continuity Across Surfaces: Users expect a seamless experience as they move between devices or applications. The point-to-point HTTP connection makes it difficult to maintain this continuity, leading to fragmented and disjointed interactions.
• Live Control: Users need to see what's happening and have the ability to control the AI's output. The limitations of HTTP streaming can make it challenging to provide real-time feedback or allow users to interrupt or steer the AI's generation process effectively.

He illustrated the fragility of this model with an example: if a user's connection drops mid-stream, the entire interaction is often lost. The agent might continue generating tokens, but with no client to receive them, the work is wasted, and the user is left with an incomplete or broken experience. This lack of resilience is a major contributor to the perceived "brokenness" of many AI UX implementations.

The Solution: Decoupling and Durable Sessions

To address these issues, Christensen advocated for a more robust architectural approach that decouples the agent layer from the client layer. He introduced the concept of "durable sessions" or "channels" as a more effective solution.

In this model, a durable session acts as a stateful layer between agents and users. Agents write events to this session, and clients subscribe to it. This decoupling offers several advantages:

• Resilient Delivery: Events are stored in the durable session, allowing clients to reconnect and resume their experience from where they left off, even after network interruptions.
• Continuity Across Surfaces: Clients can subscribe to the same durable session from different devices, ensuring a consistent and uninterrupted user experience.
• Live Control: By managing state in a central session, the system can more effectively handle requests to interrupt or steer the AI's output, providing users with greater control.

Christensen showed a diagram illustrating how this works: agents publish messages to a channel, and clients subscribe to that channel. If a client disconnects, it can simply re-subscribe and pick up the stream from the durable session, ensuring that no progress is lost.

Ably's AI Transport: A Real-World Implementation

To facilitate these more robust AI UX patterns, Ably has developed "AI Transport," a drop-in session layer for AI applications. This layer is designed to build durable, steerable, and multi-device experiences without requiring developers to rebuild their entire stack.

Ably's solution leverages channels for publish-subscribe messaging, allowing agents and clients to communicate asynchronously and reliably. This approach enables granular control over the AI's output, such as the ability to cancel a response or to receive detailed progress updates. The demo showcased how this can lead to a much smoother and more intuitive user interaction, even in scenarios with multiple concurrent users or frequent connection changes.

Christensen highlighted that by using Ably's channels, developers can move away from the problematic single point-to-point HTTP streaming model towards a more scalable and resilient architecture. This allows for features like real-time progress updates, the ability to cancel ongoing AI tasks, and seamless transitions across devices, all of which are critical for a positive AI UX.

The Future of AI UX

The core message from Christensen's presentation is clear: the perceived flaws in AI UX are often not inherent to the AI models themselves, but rather a consequence of outdated or insufficient architectural choices. By adopting more resilient and flexible communication patterns, such as those offered by durable sessions and publish-subscribe channels, developers can significantly improve the quality and responsiveness of their AI applications, ultimately leading to better user experiences.