Large Language Models Are Overconfident in Their Own Responses
2026-06-02 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied why instruction-tuned large language models (LLMs), especially chat-based ones, tend to be overconfident in their answers compared to their original versions. They discovered that the chat format makes models more confident in their own responses than identical answers from users, a problem they call 'ownership bias.' By treating the model's own answers as if they came from the user during confidence checks, the authors found they could reduce this overconfidence significantly without retraining the models. This simple trick improved how well the models' confidence matched their actual accuracy.
large language modelsinstruction tuningmodel calibrationchat templateownership biasconfidence elicitationoverconfidenceinference-time strategyopen-weight LLMs
Authors
Mario Sanz-Guerrero, Manuel Mager, Katharina von der Wense
Abstract
Prior work has shown that instruction-tuned large language models (LLMs) are less well calibrated than their base pre-trained counterparts. However, little is known about the frequently used chat template's effect on the calibration of conversational LLMs. In this work, we investigate the mechanisms driving this miscalibration by decoupling the effects of the post-training algorithm and the chat format. We find that, while instruction tuning fundamentally harms calibration, the chat template aggravates the issue through an "ownership bias" -- models are significantly more confident in their own answers than in identical answers provided by a user. Extensive experiments across six recent open-weight LLMs, three benchmarks, and three confidence elicitation methods show that models assign up to 26% higher confidence to their own responses. Leveraging this insight, we propose a simple inference-time strategy: framing the model's answer as user input during confidence elicitation. This approach significantly reduces overconfidence and improves calibration by up to 26% without the need for retraining, narrowing the gap between base and instruction-tuned models.