Sizing Lakebase Search: lessons from a real app and 4 million passages

來源原標題

Sizing Lakebase Search: lessons from a real app and 4 million passages

機構

Databricks

原始標籤

Lakebase

原始連結

https://www.databricks.com/blog/sizing-lakebase-search-lessons-real-app-and-4-million-passages

摘要原文

Slide Hub is an internal Databricks app. Field engineers use it to find and reuse slides from about 700 internal presentations: event talks, product decks, and architecture decks. A search bar finds single slides by meaning and by keyword, and a Model Context Protocol (MCP) server gives AI agents the same search. The app stores its slides, votes, search telemetry, and change history in Lakebase Postgres . We use Slide Hub as the baseline to benchmark Lakebase Postgres. It is a real app with a real use case and real users, so its numbers show how the database behaves under an actual workload. This post is for developers and architects who build apps on Databricks and want search inside the app's own Lakebase Postgres database. It assumes that you know SQL and the basics of PostgreSQL. If you plan a search bar or an agent tool on Lakebase Postgres, the numbers below help you choose the compute size before you go to production. This post shows how Slide Hub runs search inside Lakebase Postgres, what limits it under load, what it costs, and how Lakebase Search behaves from 1M to 4M rows of a public corpus. Lakebase Search became generally available in September. It is a set of Postgres extensions , so it runs next to the application tables. The app does not copy data to a separate search system. The developer does not maintain a second transaction path. Two CREATE EXTENSION statements enable the components. lakebase_vector stores embeddings , vectors of numbers that represent the meaning of a text, and supports the lakebase_ann index for approximate nearest neighbor (ANN) search. lakebase_text supports the lakebase_bm25 index for BM25, a standard method that ranks text by the query words it contains. lakebase_tokenizer is optional. The ANN index uses IVF partitioning, which groups similar vectors so a query reads only a few groups, and RaBitQ quantization, which compresses each vector to about 1 bit per dimension. BM25 scores text with the <@> operator. Slide Hub’s 23k slide rows sit in the same serverless Lakebase Postgres instance as the votes, telemetry, change history, previews, and both indexes. The app reads the data and the indexes inside one connection and one permission model. Slide Hub runs hybrid search . It combines vector search , which finds slides with a similar meaning to the query, and BM25 keyword search , which finds slides that contain the query words. Each method returns a short list of its best matches, the candidates . One SQL statement carries the whole ranking policy. It ranks each candidate list, fuses the two rankings, applies the app filters, and joins the votes. Search and app logic share one connection, one set of grants, and one commit. Slide Hub takes 200 candidates from each method. It fuses the two lists with Reciprocal Rank Fusion (RRF) at k=60: a slide scores 1/(60 + its rank) in each list, and the two scores add. A similarity gate of 0.70 rejects vector matches that are too far from the query to be useful for this embedding model. Title tiers add weight when the query matches a slide title. Notes receive weight 0.5. Scope filters restrict the result to assets the user has permission to see. A search request uses two separate computes, and either one can set the limit. Lakebase Postgres compute runs the database. Lakebase Postgres sizes it in Compute Units (CU) . One CU allocates about 2 GB of RAM to the database, with its share of CPU and local SSD. A compute autoscales between a minimum and a maximum size from 0.5 CU to 64 CU, and the maximum can be at most 16 CU above the minimum. Fixed-size computes go up to 112 CU, and computes up to 32 CU can scale to zero .Postgres stores tables and indexes in 8 KB pages . Lakebase Postgres keeps recently read pages in a Local File Cache (LFC) on the compute. The cache grows with the CU size and can use up to 75% of the compute's memory. A page that is not in the cache comes from storage, which takes longer. Databricks Apps compute runs the Slide Hub web server. Its size is fixed per app. A MEDIUM app has up to 2 vCPU and 6 GB of memory, and a LARGE app has up to 4 vCPU and 12 GB. Slide Hub runs one app instance. Each Python worker process in it holds a pool of 8 database connections, so 2 workers open at most 16 connections and 4 workers open at most 32. For each request, the app embeds the query text, sends one SQL statement to Lakebase Postgrse, and returns the result. To find the limit, the test changes one side at a time. It pins Lakebase Postgres at one CU size and changes the app, or it keeps the app and changes the CU size. A load generator simulates users that send search requests, and it raises the number of users in steps. The list shows requests per second and the median latency (p50) at 500 users for each Lakebase Postgres size and app size. Up to 4 CU, Lakebase is the limit. A LARGE app with twice the workers and connections adds nothing at 1 CU. At 1 CU the search statement averaged 146 ms and read 14.9M pages from storage, with a file cache of 1.5 GB. At 8 CU it averaged 34 ms, with a cache of 12 GB, and throughput rose about 50%. From 8 CU, the app is the limit. 16 CU with the same MEDIUM app adds only 4%, and its database connections sit mostly idle. A LARGE app with 4 workers at 16 CU reaches at least 662 req/s, and there the load generator was the limit. Size the query first. Then scale Lakebase Postgres until the app workers are the limit, and then scale the app. Takeaway: Find the limit before you scale. Below 8 CU, add Lakebase Postgres CU. From 8 CU, give the app more workers. The connection pool is still necessary. At 50 users the app reached 45 req/s and 2.4 s p95 without the pool, and 85 req/s and 0.7 s p95 with it. The embedding cache made no measurable difference under load. The candidate limit sets the number of pages a query touches. On a cold cache, that number sets the latency. Slide Hub shipped with 1,000 candidates per method. The docs pattern uses 40. The candidate-limit experiment changed only candidate_limit and measured 20 public queries in three states. Cold is the first query after the compute scaled to zero, with an empty cache. First-touch is the first run of a query on an awake compute. Repeat is the same query run again, with its pages in the cache. Pages touched counts the pages a query reads from any source. Top-10 overlap is the share of the top 10 results that match the result at 1,000 candidates. At 200 candidates, 90.5% of the top 10 stays the same, and the reported number of matches is 81% of the number at 1,000 candidates. Users rarely page past 200 results, so Slide Hub now ships 200. Measured end to end from a job in the same cloud region as the app, a cold search at 200 candidates took 2.9 s, and a repeat search took 40 ms. The cold time has two parts. The connection waits about 1 s while the endpoint wakes up. Then each query reads its pages from storage until the cache warms. The first query on a new connection also pays about 75 ms of planning. Takeaway: Keep the candidate pool small. 200 candidates keep 90.5% of the top 10 results, read about a third of the pages of 1,000 candidates, and halve the cold latency. The scale tests use MIRACL English , a passage corpus built from Wikipedia. The full corpus has 32,893,221 passages. We embedded it with gte-large-en-v1.5 , a 1024-dimension open model, the same size as the Slide Hub embeddings. We self-hosted the model on one A10G GPU at 379 passages per second. The whole corpus took 2.95 GPU-hours, against $48.87 for the same work on a model-serving endpoint. The GPU vectors matched the endpoint vectors (mean cosine 1.0000, minimum 0.9978). We built three subsets: 1M, 2M, and 4M passages. The lakebase_ann index grew with the corpus: 5.8 GB at 1M, 11.5 GB at 2M, 23.0 GB at 4M. A binary COPY loaded all 4M rows in 19 minutes at 8 CU. Lakebase Postgres holds about 1.6 GB of file cache per CU. Scale raises one question: what happens when the index is larger than the cache?