A Reproducible Multimodal LLM-as-a-Judge Workflow for Exploring Epistemic Dissonance

In interactive narrative experiences, interpreting higher-order cognitive-emotional states that involve epistemic conflict remains a significant challenge. Traditional methods relying on subjective self-reports are vulnerable to memory and observer biases, while single-agent Vision-Language Models (VLMs) often hallucinate or cherry-pick physiological data to fit predefined narratives. To address these vulnerabilities, we present a reproducible multimodal sense-making framework that integrates physiological event detection with multi-agent LLM verification. By using a 10 Hz synchronized master clock, we extract candidate regions of interest (ROIs) based on the Epistemic Dissonance Index (EDI)—an engineering heuristic derived from PCA-weighted physiological and facial signals. A two-tier LLM-as-a-Judge pipeline then cross-validates the multimodal context against raw biometric data to reduce logical inconsistencies. An N = 1 feasibility study using a survival horror game suggested instances where initial VLM outputs were revised toward interpretations more consistent with observed physiological evidence in this specific case. Although limited to a single participant, these preliminary findings suggest the feasibility of an evidence-aware multimodal sense-making architecture and provide initial insights for developing more grounded and reproducible approaches to context evaluation within the ICMI community.