Netflix's Graph Query Engine

Netflix has detailed the intricate serving layer behind its Real-Time Distributed Graph (RDG), a system designed to power real-time insights across the streaming giant's vast operations. This third installment in a series on the RDG, published on the Netflix Tech Blog, focuses on how the company tackles the challenge of querying a constantly evolving, multi-billion edge graph with sub-100-millisecond responses.

The core problem Netflix faced was enabling a wide range of internal teams to ask complex questions of the graph, from high-volume security lookups to deep dives into user viewing habits. This required a serving layer that could efficiently handle queries with vastly different structures. For instance, a "shallow and wide" query like "Which devices has this account used in the last 30 days?" demands rapid retrieval and filtering of potentially hundreds of edges from a single node. Conversely, a "deep and narrow" query, such as tracing a specific show’s viewing history across multiple profiles, requires chaining multiple hops sequentially, where network latency can quickly become a bottleneck.

Balancing Breadth and Depth

To address these opposing demands, Netflix adopted a breadth-first traversal strategy over depth-first. Breadth-first allows the system to process an entire level of the graph in parallel before moving to the next. This is crucial for avoiding the latency penalties of sequential network calls inherent in deep-narrow queries. While this approach can increase memory usage by holding an entire graph level in memory, Netflix manages this by bounding each hop with per-edge-type limits, ensuring even wide levels remain manageable.

The system’s foundation is built on an async-first architecture. Instead of dedicating a thread to each query, which would lead to massive thread bloat waiting for I/O, Netflix uses a small pool of threads that handle thousands of concurrent requests by never blocking. When a storage call is in flight, the thread is free to work on other tasks, picking up results asynchronously as they arrive. This is fundamental to achieving low latency under heavy load.

Smart Caching and Opt-In Features

Recognizing that not all graph data changes at the same rate, Netflix employs selective caching. Stable data, like account plan types or content metadata, is stored in a distributed cache (EVCache) with carefully tuned Time-To-Live (TTL) values. This strategy, refined through iteration, aims for 70-80% hit rates on frequently accessed, slow-changing nodes. Data that changes too rapidly is excluded from the cache to prevent staleness.

Enrichments, which involve fetching additional metadata from external services, are opt-in. This prevents queries from incurring unnecessary overhead by fetching data they don't need. If an enrichment service is slow or unavailable, the system degrades gracefully, returning the core graph data without the missing metadata. This approach prioritizes availability and speed.

Architecture and Query Flow

The serving layer is structured into three main components. The Graph Query Service acts as the entry point, validating requests and passing them to the execution engine. The execution engine orchestrates the breadth-first traversal, applying filters and limits asynchronously. The Storage Abstraction Layer interfaces with the underlying key-value store, handling streaming for large datasets and managing caching. The Enrichment Layer fetches external data on demand.

A typical query begins with a gRPC request being parsed into an execution plan. The engine then traverses the graph level by level, reading from storage efficiently. For the example query tracing "Stranger Things" viewing history, the engine first identifies all profiles associated with an account (Hop 1), then fetches the "started_watching" edges for each profile, filtering for the specific content (Hop 2). Parallel execution across multiple profiles and smart filtering ensure that only relevant data is processed. Caching further accelerates subsequent, identical queries. This multi-stage process, optimized at every step, allows for complex graph traversals to be answered in under 100 milliseconds.

This sophisticated query layer is a testament to Netflix’s deep engineering expertise. With platforms like Netflix, Netflix (NASDAQ:NFLX), consistently pushing the boundaries of data processing and user experience, it’s no surprise they maintain a strong presence in the industry. StartupHub.ai data shows Netflix with a score of 81/100, significantly outperforming competitors like Binge (43/100) and Crunchyroll (63/100) in areas of innovation and platform capability.