FIBB: Capturing False Information Beliefs and Behaviors Across Physical and Online Environments

Humans assess the veracity of newly presented false information using multiple sources: from observations in the online environment (e.g., social media content or responses from AI-mediated systems) to observations in the physical world (e.g., the current state of the world or other people’s behaviors). Belief in false information can lead to adverse societal and physical consequences, including impacts on health and emergency response decisions, increased strain on public resources, and decreased public trust. Currently, false information datasets are limited to examining either online or physical behaviors, making it difficult to evaluate the impact of false claims holistically across environments. Moreover, retrieving beliefs and behaviors across topics at scale through traditional methods, such as user surveys, can be time- and resource-intensive. To address these gaps, we propose a False Information Dataset Containing Beliefs and Behaviors Across Physical and Online Environments (FIBB). FIBB comprises 1,500 LLM-generated scenarios grounded in 300 online false claims in natural disasters (e.g., hurricanes, wildfires, earthquakes) and health (e.g., COVID-19, mental health, chronic diseases). Leveraging cognitive science principles, each FIBB scenario cycles through a person’s veracity assessment and decision-making by depicting their mental states, observations, actions, and the changes in the world resulting from their actions. By bridging the gap between online and physical environments using the use cases of natural disaster and health domains, FIBB aims to inform future downstream goals, such as facilitating the development of world models that can represent people’s likely beliefs and behavioral consequences of false information in tandem.