UC Berkeley’s Sky Computing Lab built LEANN, an open-source vector index that stores almost no embeddings. The LEANN vector index recomputes embeddings at query time instead of saving them to disk. The repo benchmark shows a 60 million chunk Wikipedia index dropping from 201 GB to 6 GB. The research paper won the Best Research Paper award at MLSys 2026. So, it brings semantic search and RAG over personal data, on a laptop.
TL;DR
- Size: 6 GB for 60M chunks (repo); 4 GB in the paper’s 76 GB Wikipedia setup. Default embedder: Contriever (110M params). It is an index, not an LLM, so no context length applies.
- Runs on: laptops and workstations. The paper tested an RTX 4090 box with 32 GB RAM and an M1 Ultra Mac. A CPU-only Linux install exists, with no published CPU latency.
- Performance: matches HNSW downstream accuracy at 90% recall, using under 5% of raw-data storage.
- Best: 25.5% NQ accuracy, identical to HNSW, using 4 GB instead of 188 GB.
- Worst: HotpotQA retrieval takes 44.80 s on the Mac, 50.1% of total pipeline time.
What is LEANN?
LEANN is a low-storage approximate nearest neighbor (ANN) index built on top of FAISS. It targets one problem: vector indexes are often bigger than the data they index. The research paper shows a 76 GB corpus needs 173 GB of embeddings plus a 15 GB HNSW graph. That more than doubles the raw data.
LEANN keeps a pruned graph, drops the full vectors, and rebuilds them on demand. The code is MIT licensed and installs with uv pip install leann.
How does LEANN recompute embeddings without stalling search?
Graph indexes like HNSW visit only a small set of nodes per query. LEANN exploits that. It stores the graph plus a compact product quantization (PQ) table. At query time, it re-embeds only nodes on the search path.
3 techniques keep this practical:
- Two-level search: cheap PQ distances rank neighbors first. Only top candidates get exact recomputation.
- Dynamic batching: candidates from several hops are grouped, for example 64 at once, for GPU efficiency.
- High-degree preserving pruning: most nodes keep few edges, while the top 2% “hub” nodes keep more.
In ablations, two-level search gives an average 1.4x speedup. Adding batching raises it to 1.8x. Embedding recomputation still accounts for about 76% of query latency.