Artificial Intelligence

Databricks ranks 40 offers in 34ms

Databricks says its accelerator ranks 40 banking offers at p95 34 ms, with streaming intent live in a tenth of a second.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

2 min read

Databricks Feature Views next-best-offer ranking latency chart
Credit: Databricks

Databricks says its next-best-offer accelerator ranks 40 banking offers at p95 34 ms on a route-optimized endpoint. The design pairs stable customer context with live clickstream intent under one Feature Views definition, Databricks said. That full retrieval plus LightGBM rank fits inside a 50 ms budget on the smallest endpoint size, per Databricks.

Freshness arrived in about a tenth of a second.

In-session signals were readable in the online store in under 130 ms from Kafka write, per Databricks. A SawtoothWindow keeps a 30-day category count fresh without recomputing full history, Databricks said, and adding the live feature added 3 to 4 ms to rank time. Media software makes the same tradeoff in other terms, as Ableton notes in Live 12 release notes that buffer size affects streaming latency with lower buffers on receivers, while Filmora documents a VLC network stream advancing past auto-filled media source info.

Throughput split sharply between paths, with ranking at about 56 RPS and feature serving at about 390 RPS before p95 passed 50 ms, per Databricks. Databricks measured 33 ms p50 and 48 ms p95 for rank under light load across 2,000 Zipfian-distributed keys to force cache misses. Its sizing rule is required concurrency equals 4 times target RPS divided by measured RPS at the SLO, which points to concurrency 12 for 150 ranking RPS.

What the numbers do not show is business lift, since Databricks says the data are synthetic and it claims no conversion gain. Rank endpoints scale to zero, so the first request after idle can take tens of seconds, Databricks said. The 27 ms figure combines retrieval and inference, so Databricks said it is not comparable to feature-read-only benchmarks.

A run on production traffic with eligibility, consent and suppression checks would change the read. Will 12-way concurrency still hold 150 RPS inside 50 ms p95 when miss rates climb and cold starts bite?

Mentioned in this story

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsDatabricksFeature ViewsLightGBMUnity CatalogModel ServingSawtoothWindowAbletonVLC

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.