SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
2026-05-28 • Machine Learning
Machine Learning
AI summaryⓘ
The authors created SoundnessBench, a test set of over 1,000 machine learning research proposals, to see if large language models (LLMs) can judge how methodologically sound new research ideas are before full review. They found that LLMs tend to be too optimistic, often rating bad ideas as good. Changing how prompts are given shifts mistakes but doesn't solve the problem. The authors conclude that current LLMs shouldn't be relied on alone to filter scientific proposals for quality.
Large Language ModelsResearch ProposalMethodological SoundnessPeer ReviewICLRBenchmarkScientific RigorPromptingFalse PositivesMachine Learning
Authors
Sy-Tuyen Ho, Minghui Liu, Huy Nghiem, Furong Huang
Abstract
Autonomous AI research agents aim to accelerate scientific discovery by automating the research pipeline, from hypothesis generation to peer review. However, existing benchmarks rarely test a fundamental bottleneck: whether Large Language Models can judge the methodological viability of a research idea before expending time and computational resources. We introduce SoundnessBench, a curated benchmark of 1,099 machine-learning research proposals reconstructed from ICLR submissions, labeled with reviewer soundness sub-scores, and audited against source papers. SoundnessBench should be interpreted as a benchmark for recoverable proposal-stage soundness rather than exact prediction of full-paper review outcomes. Across 12 frontier LLMs, we find a pervasive optimism bias: under standard prompting, models frequently rate low-soundness proposals as sound, while aggressive prompting largely shifts errors from false positives to false negatives. Additional controls for public-corpus contamination, paper-identifying phrases, surface features, and human audit quality suggest that this behavior is not explained by a single confounder. Our results indicate that current LLMs are not yet reliable as standalone first-gate evaluators for scientific rigor.