Evaluating the Impact of Large Language Model Assistance on Human Attention Paradigm in Human–Robot Collaboration

Human–robot collaboration in industrial and semi-structured environments imposes significant attentional demands on the human operator, particularly during complex or time-sensitive tasks. While these collaborative robots are designed to assist the human operator, they can also introduce attentional fragmentation for humans due to coordination requirements, constant task switching, and decision-making overhead. Recently, large language models (LLMs) have emerged as promising tools that provide contextual guidance and assist with task execution. However, their impact on human attentional dynamics in real-world interaction scenarios remains insufficiently tested. In this paper, we propose a multimodal framework to assess the impact of LLMs on human attention during collaborative tasks. We demonstrate that the nature of collaboration can be identified by analyzing human attention dynamics using a neuro-symbolic attention model that incorporates physiological signals. We examine participants’ behavior under two experimental conditions: LLM-assisted and non-assisted interaction with cobots. By utilizing real-time attention modeling alongside human biometrics, we characterize the temporal dynamics of attention allocation and its relationship to different experimental conditions during task execution. In addition, we investigate how varying levels of assistance influence the dynamics of attention. Our results indicate that LLM assistance is associated with a more stable task-oriented attentional state, which could help reduce attentional disruptions and allow for more consistent engagement with cobots during a task. Additionally, we show that attention mapping with the neuro-symbolic model outperforms the traditional rule-based model by 39%, and provides better grounding to the neural-based model while outperforming by 1%. Our work proposes an adaptive cobot system that dynamically responds to human attentional states and thereby enhances efficiency, safety, and human-centered interaction in complex environments.