Databricks says its next-best-offer accelerator ranks 40 banking offers at p95 34 ms on a route-optimized endpoint. The design pairs stable customer context with live clickstream intent under one Feature Views definition, Databricks said. That full retrieval plus LightGBM rank fits inside a 50 ms budget on the smallest endpoint size, per Databricks.
Freshness arrived in about a tenth of a second.
In-session signals were readable in the online store in under 130 ms from Kafka write, per Databricks. A SawtoothWindow keeps a 30-day category count fresh without recomputing full history, Databricks said, and adding the live feature added 3 to 4 ms to rank time. Media software makes the same tradeoff in other terms, as Ableton notes in Live 12 release notes that buffer size affects streaming latency with lower buffers on receivers, while Filmora documents a VLC network stream advancing past auto-filled media source info.
Throughput split sharply between paths, with ranking at about 56 RPS and feature serving at about 390 RPS before p95 passed 50 ms, per Databricks. Databricks measured 33 ms p50 and 48 ms p95 for rank under light load across 2,000 Zipfian-distributed keys to force cache misses. Its sizing rule is required concurrency equals 4 times target RPS divided by measured RPS at the SLO, which points to concurrency 12 for 150 ranking RPS.
What the numbers do not show is business lift, since Databricks says the data are synthetic and it claims no conversion gain. Rank endpoints scale to zero, so the first request after idle can take tens of seconds, Databricks said. The 27 ms figure combines retrieval and inference, so Databricks said it is not comparable to feature-read-only benchmarks.
A run on production traffic with eligibility, consent and suppression checks would change the read. Will 12-way concurrency still hold 150 RPS inside 50 ms p95 when miss rates climb and cold starts bite?





