Run fast, private agents on-device with LFM2.5-24B-A2B
https://huggingface.co/blog/LiquidAI/lfm25-24b-a2b📌 【LiquidAI 開源】24B 參數只點亮 2B，端側 Agent 也能跑推理
TL;DR：LFM2.5-24B-A2B 是只啟用 2B 參數的 24B MoE 推理模型，專為在自己硬體上長時間跑 agent 設計。
如果你曾經想在筆電或單張消費級 GPU 上跑一個會思考、會呼叫工具、還要連續運作好幾輪的 agent，多半會卡在一個問題：reasoning token 太貴，模型一邊想一邊燒錢也燒顯卡。LiquidAI 這次釋出的 LFM2.5-24B-A2B，用「總參數 24B、每個 token 只啟用 2B」的設計,試著把這個成本壓下來。
🤔 為什麼端側 agent 需要「便宜的 reasoning token」
Agent 任務的特徵是多步驟、要呼叫工具、還要在過程中反覆推理。如果每個 reasoning token 都要動用大量運算,在自己的裝置上連續跑 agent 幾乎不現實。LiquidAI 的切入點是:既然推理模型本身就要先想再答,那不如讓每個 token 的運算成本降到最低,這樣才能在自己的硬體上「持續」跑 agent,而不是跑一次就要等很久。
🧩 架構沿用 LFM2-24B-A2B,但把 context 和詞表都做大
LFM2.5-24B-A2B 延續前代 LFM2-24B-A2B 的架構:40 層、每個 MoE block 有 64 個專家、top-4 路由,並維持 gated short convolution 與 GQA 的混合設計。相較前代,這一版把 context window 從 32K 擴到 131K token,並透過 in-place tokenizer 擴充,把詞表從 65K 翻倍到 128K,改善非拉丁語系文字的 tokenization。與前代不同的是,這一版被設計成「純推理模型」,也就是回答前一定會先產出明確的 chain of thought。
訓練規模也同步放大:預訓練 token 量從約 17T 拉到約 33T。中段訓練再加上 2T token(32K context,偏重推理、數學、程式碼、指令遵循與多語言)以及 300B token(131K context,偏重長文本),且工具呼叫與多輪軌跡資料貫穿這兩個階段。後訓練則分五個階段:SFT、教師模型專精化、非 agentic 的多領域 on-policy distillation(MOPD)、agentic 的 MOPD,最後在 OpenClaw、Hermes Agent、Pi、OpenCode 等 agent harness 裡做 agentic 強化學習。
📊 用 50% 更少的參數,打平甚至打贏更大的 MoE
LiquidAI 用 16 個橫跨工具使用、知識、指令遵循、數學與程式碼的 benchmark,拿 LFM2.5-24B-A2B 和總參數、啟用參數都多出約 50% 的 Qwen3.6-35B-A3B,以及 Gemma-4-26B-A4B-it、Nemotron-3.5-Lightning(30B-A3B)比較:
| Benchmark | LFM2.5-24B-A2B | Qwen3.6-35B-A3B | Gemma-4-26B-A4B-it | Nemotron-3.5-Lightning |
|---|---|---|---|---|
| AutomationBench | 27.69 | 8.72 | 5.46 | 2.61 |
| BFCLv4 | 65.01 | 65.00 | 57.09 | 53.78 |
| ToolSandbox | 80.71 | 75.02 | 72.09 | 75.13 |
| LiveClaw | 48.09 | 38.19 | 41.25 | 37.72 |
| IFEval | 96.75 | 93.06 | 92.00 | 94.19 |
| SWE-Bench Verified (OpenCode) | 51.00 | 66.20 | 25.80 | 53.40 |
在 8 個工具使用/agentic benchmark 中,LFM2.5-24B-A2B 在 5 項領先或打平,AutomationBench 更是拿到next-best 模型三倍以上的分數,三項指令遵循 benchmark 也全部領先。
💡 強在工具呼叫,弱在純知識與程式碼
在 GPQA Diamond、HLE、AA-Omniscience 這類知識密集型 benchmark,以及 SWE-Bench Verified(OpenCode)、SciCode 這類程式碼 benchmark 上,較大的模型仍保有優勢。LiquidAI 自己的建議是:知識密集的工作負載,搭配 retrieval 或網路搜尋來補強,而不是硬要模型自己記住所有知識。
推理速度方面,官方數據顯示在 Apple M5 Max 上可達 184 tok/s,在 Ryzen AI Max+ 395 上為 114 tok/s,號稱是同尺寸 MoE 中最快且記憶體用量最低;單張 NVIDIA B300 在高併發下則可衝到超過 21K output tokens/s。
⚠️ 仍是既有稀疏化架構的延伸
LFM2.5-24B-A2B 本質上是在 LFM2-24B-A2B 的 MoE 架構上做規模放大與後訓練強化,並非全新的稀疏化設計。對於知識廣度與程式碼生成要求較高的場景,仍需要搭配外部工具或更大的模型。
🎯 怎麼接進你的 agent harness
模型已開源在 Hugging Face,支援 transformers、llama.cpp、MLX、vLLM、SGLang、ONNX 等推理生態系。只要用你偏好的後端把模型跑成 OpenAI 相容的服務,再把 Hermes Agent、OpenClaw、Pi 等 agent harness 指向對應的 port(llama.cpp/MLX 用 8080,vLLM 用 8000,SGLang 用 30000,LM Studio 用 1234),就能快速接上既有的 agent 流程。
🔗 來源
- 標題:Run fast, private agents on-device with LFM2.5-24B-A2B
- 作者／機構:LiquidAI(HuggingFace Blog)
- 連結:https://huggingface.co/blog/LiquidAI/lfm25-24b-a2b
#MixtureOfExperts #OnDeviceAI #LLM #AgenticAI #OpenSource #LiquidAI #EdgeInference #ToolUse #ReasoningModel #HuggingFace
原始資料 HuggingFace Blog · 收集於 2026-10-13
摘要原文
Back to Articles Run fast, private agents on-device with LFM2.5-24B-A2B Team Article Published October 13, 2026 Upvote 28 +22 Raahul Vignesh Raahul26 Follow LiquidAI Song Duong sduong Follow LiquidAI Akhilesh Deepak Gotmare akhileshdeepakgotmare Follow LiquidAI Fernando Fernandes Neto fernandofernandes Follow LiquidAI Jiahui Wang WWWjiahui Follow LiquidAI Mayank Agarwal mayankagarwal Follow LiquidAI nathan ranchin nathanrchn Follow LiquidAI Sergei Tilga tilgasergey Follow LiquidAI Sinoué GAD GAD-cell Follow LiquidAI Tim Seyde tseyde Follow LiquidAI Zetian Li zetianli Follow LiquidAI changyi yang changyiyang Follow LiquidAI Simon Lee yungisimon Follow LiquidAI Leonie Monigatti iamleonie Follow LiquidAI Maxime Labonne mlabonne Follow LiquidAI Karthik Macherla karthikrmacherla Follow LiquidAI Alfred Eng alfredeng Follow LiquidAI Khoi Nguyen KTLK Follow LiquidAI Paul Pak paulpak58 Follow LiquidAI xx tugot17 Follow LiquidAI Tarek Dakhran tdakhran Follow LiquidAI Anthony Maltsev amaltsev Follow LiquidAI Aldo Gael Carranza agcarranza Follow LiquidAI Tyler Murray undfined Follow LiquidAI Aditya Tadimeti adityatadimeti Follow LiquidAI Jimmy T.H. Smith jimmysmith1919 Follow LiquidAI Yuri Khrustalev ykhrustalev Follow LiquidAI LFM2.5-24B-A2B is a 24B-parameter reasoning model with just 2B active parameters, built for fast, private agents on-device. It thinks before it answers, calls tools, and works through multi-step tasks inside popular agent harnesses. Because only 2B parameters are active per token, each reasoning token is cheap, so you can run agents continuously on your own hardware. Best-in-class agent: Competitive with Qwen3.6-35B-A3B, a model with ~50% more total and active parameters, on agentic and tool-use tasks, scoring 48.09 on LiveClaw versus 38.19. Reasoning at MoE speed: Always thinks before it answers. With only 2B active parameters per token, each reasoning token is cheap. Efficient inference: 184 tok/s on an Apple M5 Max and up to 21K output tok/s on a single NVIDIA B300 GPU. How we built a reasoning agent with 2B active parameters LFM2.5-24B-A2B builds on LFM2-24B-A2B and keeps its architecture: 40 layers, 64 experts per MoE block with top-4 routing, and the same hybrid of gated short convolutions and GQA. The context window grows from 32K to 131K tokens, and an in-place tokenizer expansion doubles the vocabulary from 65K to 128K tokens for better non-Latin tokenization. Unlike its predecessor, it is a reasoning-only model that produces an explicit chain of thought before its final answer. Pre-training was scaled up from ~17T to ~33T tokens. Mid-training then adds 2T tokens at 32K context, weighted toward reasoning, math, code, instruction following, and multilingual text, and 300B tokens at 131K context, weighted toward long-context data. Tool calls and multi-turn trajectories appear throughout both phases. Post-training turns the base model into an agent in five stages: Supervised fine-tuning (SFT) Teacher specialization Multi-domain on-policy distillation (MOPD), non-agentic: distill the specialist teachers on non-agentic tasks. MOPD, agentic: a second distillation stage focused on agentic tasks. Agentic reinforcement learning (Agentic RL): train the model inside agent harnesses, such as OpenClaw , Hermes Agent , Pi , and OpenCode . Benchmark results We evaluated LFM2.5-24B-A2B against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, and Nemotron-3.5-Lightning on 16 benchmarks covering tool use and agentic workflows, knowledge, instruction following, math, and coding. It is the smallest model in the group, yet it is competitive with MoEs with ~50% more total and active parameters. Benchmark LFM2.5-24B-A2B Qwen3.6-35B-A3B Gemma-4-26B-A4B-it Nemotron-3.5-Lightning (30B-A3B) AutomationBench 27.69 8.72 5.46 2.61 BFCLv4 65.01 65.00 57.09 53.78 ToolSandbox 80.71 75.02 72.09 75.13 LiveClaw 48.09 38.19 41.25 37.72 PinchBench 83.18 89.93 77.12 84.20 τ³-Bench Banking 8.04 7.63 4.54 7.01 WebArena Verified 26.75 37.62 25.62 28.69 GPQA Diamond 74.42 83.69 74.90 72.42 AA-Omniscience -24.90 -24.35 -59.39 -3.58 HLE 7.40 7.69 8.57 5.65 IFEval 96.75 93.06 92.00 94.19 IFBench 74.69 64.21 47.44 71.67 Multi-IF 84.38 81.53 82.19 82.34 MATH-500 96.04 96.73 95.66 96.40 SWE-Bench Verified (OpenCode) 51.00 66.20 25.80 53.40 SciCode 32.71 37.35 35.42 27.78 *All the models were evaluated with vLLM, using each benchmark's recommended generation parameters. For your app, the strengths are tool use and instruction following. LFM2.5-24B-A2B leads or matches the competition on five of the eight tool-use and agentic benchmarks (AutomationBench, BFCLv4, ToolSandbox, LiveClaw, and τ³-Bench Banking), and on AutomationBench it scores more than three times the next-best model. It also leads on all three instruction-following benchmarks. Larger models keep an edge on knowledge-heavy benchmarks like GPQA Diamond, HLE, and AA-Omniscience, and on coding benchmarks like SWE-Bench Verified (OpenCode) and SciCode, so pair LFM2.5-24B-A2B with retrieval or web search for knowledge-heavy workloads. These results make LFM2.5-24B-A2B a strong foundation for agents that need to reason, call tools, and run reliably on your device. Inference speed on CPU and GPU LFM2.5-24B-A2B ships with day-one support across the inference ecosystem, including llama.cpp, MLX, vLLM, SGLang, and ONNX. Edge inference. LFM2.5-24B-A2B is the fastest of the compared MoEs, decoding 184 tokens/s on an M5 Max and 114 tokens/s on a Ryzen AI Max+ 395. That is up to 1.7x faster than the next-best model, while using the least memory. GPU inference. LFM2.5-24B-A2B is the fastest model in its size class, reaching over 21K output tokens per second at high concurrency on a single B300. How to use LFM2.5-24B-A2B Reach for LFM2.5-24B-A2B when you need agents that reason, call tools, and run reliably on your device. Install the latest version of transformers (compatible with transformers>=5.0.0 ): pip install -U transformers Then load and run the model: from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer model_id = "LiquidAI/LFM2.5-24B-A2B" model = AutoModelForCausalLM.from_pretrained( model_id, device_map= "auto" , dtype= "bfloat16" , # attn_implementation="flash_attention_2" <- uncomment on compatible GPU ) tokenizer = AutoTokenizer.from_pretrained(model_id) streamer = TextStreamer(tokenizer, skip_prompt= True , skip_special_tokens= True ) prompt = "What is C. elegans?" input_ids = tokenizer.apply_chat_template( [{ "role" : "user" , "content" : prompt}], add_generation_prompt= True , return_tensors= "pt" , tokenize= True , )[ "input_ids" ].to(model.device) output = model.generate( input_ids, do_sample= True , temperature= 0.1 , top_k= 50 , repetition_penalty= 1.05 , max_new_tokens= 512 , streamer=streamer, ) Use it in your agent harness Serve LFM2.5-24B-A2B locally behind any OpenAI-compatible backend, then point your agent harness at it. For example, with Hermes Agent: hermes config set model.provider custom hermes config set model.base_url http://localhost:8080/v1 hermes config set model.default LFM2.5-24B-A2B hermes config set model.context_length 131072 hermes config set model.api_mode chat_completions hermes config set agent.tool_use_enforcement true The port depends on your serving backend: llama.cpp and MLX use 8080 , vLLM uses 8000 , SGLang uses 30000 , and LM Studio uses 1234 . See the model card for OpenClaw and Pi configs, and our guide for the full setup. Check out our release blog for video examples. Get Started with LFM2.5-24B-A2B Both LFM2.5-24B-A2B and LFM2.5-24B-A2B-Base are open-weight and available on Hugging Face today: Download: LFM2.5-24B-A2B-Base and LFM2.5-24B-A2B on Hugging Face. Use in your harness: follow our guide on how to run a local agent, like OpenClaw, Hermes Agent, and Pi. Fine-tune: adapt LFM2.5-24B-A2B to your task with our fine-tuning tutorials . We can't wait to see what you build.
由 claude-code/sonnet 自動生成