AI summaryⓘ
The authors studied how people handle uncertain information in long answers generated by large language models (LLMs). They found that users experience uncertainty in three stages: understanding the answer, judging its reliability, and deciding what to do next. Based on this, they created a system called U-Lens that helps organize and explain uncertain parts of the answers, making it easier for users to focus on what matters. Their tests showed U-Lens helped users check information faster, feel less overwhelmed, and feel more supported compared to just seeing confidence scores. This work shifts focus from just showing uncertainty signals to helping users actively manage and understand uncertainty in AI responses.
Large Language ModelsUncertainty CommunicationKnowledge-Intensive TasksInterpretationEvaluationDecision MakingConfidence ScoresUser-Centered DesignGenerative AIInformation Verification
Authors
Yu Mei, Qingyue Zhuang, Jie Cai, Chang Liu, Zhi Zheng, Zhoutong Ye, Chun Yu, Yuanchun Shi
Abstract
Large language models (LLMs) are increasingly used to generate long-form answers for knowledge-intensive tasks, but users often struggle to decide which parts of a response deserve scrutiny, why they may be unreliable, and what to do next. Prior work on uncertainty communication has largely focused on making uncertainty visible through cues such as confidence scores, leaving less support for the broader process of managing uncertainty distributed across a long response. Through a formative study, we examine how users manage such uncertainty across three stages: interpretation, evaluation, and decision. Based on these insights, we derive design guidelines that address both stage-specific and cross-stage needs: uncertainty target representation, evaluative explanation, response guidance, and interactive presentation. We instantiate these guidelines in U-Lens, an uncertainty-management support system that organizes uncertain information in long-form responses into contextual inspection targets, prioritizes them for attention, and connects each target with evaluative context and response options. We evaluated U-Lens in a controlled within-subjects study with 18 participants, comparing it against a confidence-cue baseline. Our results show that U-Lens improved verification efficiency and effort allocation, lowered perceived workload, and strengthened perceived support across interpretation, evaluation, and decision stages. This work reframes uncertainty support for generative AI from presenting isolated, text-centered cues toward supporting the user-centered process of interpreting, evaluating, and acting on uncertain information.