Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

來源原標題

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

原始連結

https://machinelearning.apple.com/research/selection-structured-reasoning

摘要原文

research area Methods and Algorithms content type paper published October 2026 Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents Authors Feiyu Gavin Zhu†**, Xiaoyu Zhu, Jiqi Yang, Rui Yang‡***, Arnab Kumar Mondal, Yancheng Wang, Xinke Deng, Jean Oh†, Reid Simmons†, Joerg Liebelt, Xiang Kong, Zhongyu Jiang View publication Copy Bibtex Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their likelihoods given the current context, without an auxiliary task head. Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluated SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28–54%. † Carnegie Mellon University ‡ University of Illinois Urbana-Champaign ** Work partially done while at Apple *** Work done while at Apple Related readings and updates. Learning Structured Reasoning via Tractable Trajectory Control July 2, 2026 research area Computer Vision , research area Speech and Natural Language Processing conference ICML Large language models can exhibit emergent reasoning behaviors, often manifested as recurring lexical patterns (e.g., “wait,” indicating verification). However, complex reasoning trajectories remain sparse in unconstrained sampling, and standard RL often fails to guarantee the acquisition of diverse reasoning behaviors. We propose a systematic discovery and reinforcement of diverse reasoning patterns through structured reasoning, a paradigm that… Read more Interleaved Reasoning for Large Language Models via Reinforcement Learning May 28, 2025 research area Knowledge Bases and Search , research area Speech and Natural Language Processing Long chain-of-thought (CoT) significantly enhances large language models’ (LLM) reasoning capabilities. However, the extensive reasoning traces lead to inefficiencies and an increased time-to-first-token (TTFT). We propose a novel training paradigm that uses reinforcement learning (RL) to guide reasoning LLMs to interleave thinking and answering for multi-hop questions. We observe that models inherently possess the ability to perform interleaved… Read more Discover opportunities in Machine Learning. Our research in machine learning breaks new ground every day. Work with us