Revising Context, Shifting Simulated Stance: Auditing LLM-Based Stance Simulation in Online Discussions
2026-06-04 • Computation and Language
Computation and LanguageMultimediaSocial and Information Networks
AI summaryⓘ
The authors studied how large language models (LLMs) simulate people’s opinions in online conversations and whether these simulations truly reflect individual beliefs or just change easily depending on context. They tested this by changing parts of a conversation and seeing if the simulated opinions shifted, using both just text and memes as context. Their findings showed that the simulated stances do change in predictable ways, showing that the models are influenced by the conversation context. The authors provide a way to evaluate how sensitive these LLM simulations are to different conversation details, pointing out both useful and risky aspects of using LLMs to mimic online opinions.
large language modelsstance simulationcontext sensitivitycounterfactual context revisiononline conversationsmultimodal contextmeme analysispolarizationstance transitionopinion dynamics
Authors
Xinnong Zhang, Wanting Shan, Hanjia Lyu, Zhongyu Wei, Jiebo Luo
Abstract
Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user's stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user's stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation. More broadly, it highlights both the promise and risk of using LLMs to simulate online opinion dynamics.