CEDAR: Compressed-Object Parallel Retrieval for Scalable AI Systems

Retrieval-augmented AI systems maintain external memory through parallel pipelines that shard corpora, batch embeddings, build ANN indexes, refresh updates, and serve concurrent queries. Although corpora are often stored and transferred in compressed form, conventional retrieval pipelines expand them into text before chunking, embedding, and indexing. This decompression-first boundary increases data movement, vector cardinality, ANN graph-construction work, and refresh scope. This paper demonstrates CEDAR, a compressed-object retrieval system that makes directly addressable compressed chunks the common unit of storage, parallel build, indexing, retrieval, refresh, and selective recovery. CEDAR combines ZPKG, a random-access Zstandard package format for independently refreshable shards, with CEM, a byte-level compressed embedding model that maps compressed chunks into a semantic retrieval space while preserving plain-text queries. The demonstration shows CEDAR across the retrieval-memory lifecycle, including shard construction, HNSW indexing, byte-offset evidence recovery, and localized refresh. We further integrate CEDAR with RAG over large corpora, agent memory with incremental updates, and OpenClaw as a memory tool, reporting build cost, retrieval latency, refresh scope, and recovered evidence.