I built non-autoregressive decision models with RL a year ago

來源原標題

I built non-autoregressive decision models with RL a year ago

作者

nandakishor_ml

原始標籤

hackernews

來源訊號

HN points 777　HN 留言 186

原始連結

https://laya.convaiinnovations.com/

摘要原文

Everyone in AI right now is talking about a new kind of model: an architecture that is not autoregressive, does not generate text, and gives lightning-fast probability predictions over structured schemas. Seeing the hype online feels both validating and deeply frustrating. I worked on this literally one year back in March 2025. I spent months of hard work, sweat, and sleepless nights building it, published an arXiv paper ( arXiv:2503.23303 ), released the model weights on Hugging Face ( sales-conversion-model-reinf-learning ), published the open dataset ( saas-sales-conversations ), built a PyPI package, and posted the whole approach on Reddit ( r/LocalLLaMA discussion ). Then in September 2025, I published a second paper ( arXiv:2510.01237 ), formalizing the framework for schema-based decisions guided by reinforcement learning. The guiding brain in my system was always reinforcement learning, not just an embedding model or an autoregressive LLM. And then in September 2026, a well-funded frontier lab called TypeSafe AI (founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI) launched Jev. They proposed the exact same non-autoregressive decision concept as if it was a brand-new scientific breakthrough. Except they launched without technical papers, without open weights, and with zero open training datasets. My earlier model used PPO over sequence representations to output turn-by-turn conversion trajectories (probabilities from 0.0 to 1.0) in vertical sales conversations. Jev generalized parallel sampling using what they called RLCD (Reinforcement Learning for Calibrated Decisions) to output confidence distributions and schema choices horizontally, charging $0.042 per million input tokens with typical response times around 150 ms. Instead of staying bitter, I decided to take everything I learned, fix every architectural limitation of the old approach, and build a completely open, horizontal System 1 decision model family: Laya . And because we built it properly on bidirectional encoders, our models run in 32.8 milliseconds on a single GPU (7.2 ms/question batched) , making it 6 to 8 times faster than Jev , with full support for over 100 languages, zero API subscription costs, and 100% open-source Apache 2.0 weights. 1. The Core Realization: System 1 vs System 2 Every modern AI pipeline has a giant bottleneck: we use generative LLMs for simple reflex decisions. When a customer support ticket arrives, or an email hits your inbox, or a user submits a prompt to your API, you usually only need to answer simple, structured questions: Which department should this ticket route to? Is this incoming email a phishing attack or spam? Is this prompt trying to jailbreak or inject instructions? How urgent is this issue on an ordinal rubric (0 to 3)? Does this query require code execution or a simple factual reply? Calling an 8B, 70B, or frontier generative LLM for this is complete overkill. You wait 500 ms to 2,000 ms for tokens to stream out, spend real money on inference, and then have to write regex or JSON parsers to extract a clean label from free-form text. Worst of all, LLMs love to hallucinate and generate fake confidence. When an LLM outputs "confidence: 0.95" , it is just predicting tokens that sound confident. There is zero mathematical calibration behind it. We needed a model that works like the human brain's System 1: instant reflex decisions with honest, calibrated probabilities, taking only 30 to 35 milliseconds on standard commodity hardware. 2. The Three Decision Primitives Laya evaluates typed questions over any state (raw text, email, ticket, or JSON document) in a single forward pass . It relies on three primitives: choice : Pick one option from a dictionary of criteria. Returns the selected key, probability distribution across all options, and a calibrated confidence score. score : Place the state on an ordinal rubric (levels 0, 1, 2, ...). Returns the expected level, the distribution over rubric ranks, and confidence. noul : A direct boolean question returning calibrated probability P(true) from 0.0 to 1.0 (with P(false) = 1 - P(true) by construction). Because the output space consists purely of probabilities and numbers, the model never generates text, cannot hallucinate, and schema violations or malformed JSON are physically impossible. 3. The Three Checkpoints & Bundled Hub Architecture One model cannot be optimal for every task and language. We released three specialized checkpoints, now consolidated under a single repository hub on Hugging Face: Checkpoint Backbone Encoder Params Context Primary Strength convaiinnovations/laya ModernBERT-large 421M 512 English text classification, guardrails, email triage convaiinnovations/laya-multilingual mmBERT-base (256k vocab) 322M 1024 (up to 8k) 100+ languages, 2.2x faster, cross-lingual NLI convaiinnovations/laya-typed-decisions ModernBERT-large 421M 1024 Agent observability, customer service, invoice processing, security alerts (0.766 acc) Selective Subfolder Downloads Rather than forcing users to manage three separate repositories or download 2.5 GB of combined weights, the main repository convaiinnovations/laya bundles all three. Using Hugging Face's allow_patterns , Laya's SDK downloads only the specific subfolder requested: # Downloads English model (~808 MB) agent_en = laya.load("convaiinnovations/laya") # Downloads ONLY the multilingual subfolder (~647 MB), not the entire 2.5 GB bundle agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual") 4. Why Routing Is Essential: The Multi-Script Reality One of the most eye-opening findings from our 51-language sweep on the MASSIVE benchmark (20 options, random baseline = 0.050) was how English models fail outside Latin script. ModernBERT-large's 50,000-token English BPE vocabulary simply shreds non-Latin alphabets: Khmer: 0.000 accuracy at 0.952 mean confidence . Not one correct decision in 100 questions, while reporting ~95% confidence. Armenian: 0.050 accuracy (exact coin-flip random) at 0.885 confidence. Hebrew: 0.060 accuracy at 0.964 confidence. Bengali: 0.080 accuracy at 0.945 confidence. Hindi: 0.100 accuracy at 0.941 confidence. This is the crucial lesson: the model's own confidence gives no warning when it cannot read the input script . Across 51 languages, the English checkpoint's mean confidence never drops below 0.885, regardless of whether its accuracy is 82% or 0%. Therefore, confidence gating cannot protect you . The decision of which model to use must be made before the forward pass. Sub-Millisecond Pure Python Routing Laya includes a built-in Router that inspects the Unicode scripts of incoming text across 22 alphabets (Devanagari, CJK Han, Cyrillic, Arabic, Hebrew, Tamil, Thai, etc.) and analyzes Latin stopword distributions: Standard English text: 0.09 ms detection overhead. Devanagari / Indic text: 0.54 ms detection overhead. Large 200-row nested JSON documents: 0.73 ms detection overhead. Compared to a 33 ms forward pass, routing overhead is negligible (<2%). And with Router(preload=True) , all required models stay resident in VRAM/RAM, completely eliminating the 7 to 10-second cold-swap penalty when traffic alternates between languages. from laya import Router # Preload checkpoints into memory for instant sub-35ms routing router = Router(preload=True) # English -> automatically routed to ModernBERT-large res_en = router.predict({"body": "I was charged twice, please refund."}, questions) # Hindi -> automatically routed to mmBERT-base (100+ languages) res_hi = router.predict({"body": "मुझसे दो बार शुल्क लिया गया, कृपया पैसे वापस करें।"}, questions) # Explicit override when you already know the domain res_spec = router.predict(state, questions, model="typed-decisions") 5. Head-to-Head: Laya (with Routing) vs TypeSafe Jev We benchmarked Laya directly against TypeSafe Jev across public datasets and standard benchmarks. (777 points, 186 comments on Hacker News)