Semantic Scholar

Helios: Exposing Context Lifecycles for Memory-Efficient Agentic LLM Serving

作者:Han-Fei Geng, Kun Li, Yunxin Liu

🔗 https://www.semanticscholar.org/paper/a5be5ee68851f51e7f026d756f2d8e97815564ca ↗

LLM serving is shifting from stateless queries to long-running agents that reshape working context through consolidation, updates, and forgetting. Existing serving engines still manage prefix-cache residency with execution-level signals such as recency, creating a semantic-to-execution divide: obsolete context can persist as zombie KV while live reusable prefixes are evicted. We present Helios, a semantic-lifecycle-aware serving engine that lets agent frameworks declare context lifecycle events through lightweight semantic handles. Helios translates these declarations into reference-safe KV reclamation and exposes recovered cache capacity to prefill scheduling. Across three long-horizon agent workloads, Helios improves prefix-cache hit rate by 1.3–3.5 × over SGLang, yielding 1.2–1.5 × higher throughput and 16–37% lower token costs.

內容為收集當日抓取的摘要原文。