Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU

來源原標題

Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU

作者

Sana Hassan

原始標籤

Agentic AI · AI Infrastructure · AI Shorts · Applications · Artificial Intelligence · Context Engineering · Deep Learning · Editors Pick · Knowledge Graphs · Language Model · Large Language Model · Machine Learning · New Releases · Open Source · Software Engineering · Staff · Tech News · Technology

原始連結

https://www.marktechpost.com/2026/09/24/fastino-releases-gliner2-5-decide-a-340m-open-weight-decision-model-that-runs-on-cpu/

摘要原文

Fastino Labs has released GLiNER2.5-Decide , a 340M-parameter open-weight decision model. It takes text and a schema of typed questions and returns structured answers. Each answer comes with a probability distribution, a confidence score, and constraint-feasibility metadata. It targets the frequent judgment calls inside agent pipelines: routing, triage, tool selection, and guardrails. Is it deployable? Yes, the weights ship under Apache 2.0 and install with pip install gliner2 . They run on CPU, GPU, or in air-gapped environments. Fastino team also offers hosted inference and fine-tuning through its GLiNER API . GLiNER2.5-Decide is a non-generative classifier. It uses a DeBERTa-v3-large encoder and is fine-tuned from gliner2-large-v1 . It produces no generated tokens and needs no prompt template. Label sets are passed at call time. Each question in the schema declares its permitted answers. It also declares whether it expects one answer, multiple answers, or an ordered value. Schemas can carry instructions, examples, label descriptions, and rules linking answers across questions. The pipeline has 2 stages. The encoder reads the text and schema together and scores every permitted answer. A constrained decoder then searches for the highest-scoring joint assignment the declared rules allow. Fastino is explicit about scope. The model does not reason, explain, or answer open questions. It is a specialist for operational decisions. Fastino team illustrates the value with a guardrail example. Decoded independently, the model flagged prompt injection at 0.82. It also labeled the same prompt safe at 0.52. The attack was detected, but the 2 outputs conflicted. Joint decoding applies a rule that any detected harm requires an unsafe verdict. The model then returns safety=unsafe and harm_type=prompt_injection together. Downstream code can use those scores to block, route, or escalate. Schemas can express implications, exclusions, cardinality limits, and ordinal bounds. The same encoder can also extract entities, relations, and structured records with character-level offsets in 1 forward pass. Classification answers do not return evidence spans. Fastino team evaluated the model on Fast Decisions , an internally generated, held-out suite. It contains 5,100 test examples across 17 datasets. The tasks cover customer operations, domain routing (banking, clinical, travel, benefits), and general content understanding. The metric is exact-match accuracy: a prediction counts only if its label set matches the reference exactly. GLiNER2.5-Decide led 9 of the 17 datasets. Intent routing was its strongest area. It scored 75.3% on support intent and 64.3% on banking intent. Those are 18.6 and 8.6 points ahead of the next-best models. Fastino team benchmarked the checkpoint end to end at batch 1 with a 2-head, 15-label schema. At 64 tokens, p50 latency was: Short requests are dominated by fixed preprocessing and kernel-launch overhead. That keeps the GPUs within 9 ms of each other. At 1,024 tokens, the A100 pulls ahead at 52.6 ms, versus 75.6 ms on V100 and 131.4 ms on L4. This multi-head call is adapted from the model card: Single-label heads return one string. Multi-label heads return every label above a cls_threshold . Labels can carry descriptions, and ordinal scales are passed as ordinary strings like “0” to “10”. Fastino targets model routing, tool calling, browser and computer use, guardrails, context pruning, LLM-as-a-judge, and simulations. Fine-tuning works locally, fully or with LoRA, through the GLiNER2 trainer . A SKILL.md file exposes the hosted workflow to coding agents. Fastino also published GLiNER2.5-Decide-1B , built on the Ettin 1B encoder . It scores 59.6% on the same suite, just under the 340M model. For multilingual input, Fastino points to GLiNER2.5-multi-Decide , a 287M model scoring 56.7%. The underlying architecture is described in the GLiNER2 paper . Check out the Technical Details and Model Card . All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter . Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU appeared first on MarkTechPost .