Impacts of Trigger-based Physical Props and Collaboration in In-Silico Care Workflows

Clinical workflows are inherently complex, and evaluating future technologies, e.g., the involvement of a humanoid robot, with patients is challenging. A bridge is needed when transitioning from the laboratory to the hospital. Immersive Virtual Environments (IVEs), a proven strategy for in-silico training scenarios, can ensure integration of technologies into existing care workflows, without the need for physical patients. We developed an authentic IVE in Unity that mimics a specific European hospital, including lifelike patient agents driven by a Large Language Model (LLM). Based on the findings of a pilot study, the focus was placed on including trigger-based physical props and integrating multi-user functionality. In a mixed 2x2 study design, 32 clinic staff performed an in-silico wound care scenario, providing care to the LLM-based patient. This main study is divided into two parts: Study I - the use of physical props vs. fully virtual objects; Study II - the collaborative work with another person (multi-user) vs. performing the task alone. We measured usability and presence to evaluate the physical props and the multi-user experience. These real-world results have been further analyzed, focusing on pre-existing knowledge in wound care management. Lastly, we compare the impact of the individual features overall on the simulation. Findings are then placed in context using existing studies from related publications. Our findings include: physical props contribute to presence and, through familiarity, also provide an entry point to the simulation. However, a personal selection of items as well as high fidelity is essential. Multi-user enhances presence but adds complexity to achieve the same level of task fidelity, suggesting the need for an initial training session. Other remarks include a promising future perspective on LLM-based patients, with an emphasis on situational awareness (following the action), for which more research will be necessary.