sciwrite-lint: Verification Infrastructure for the Age of Science Vibe-Writing
2026-04-09 • Digital Libraries
Digital LibrariesComputation and LanguageSoftware Engineering
AI summaryⓘ
The authors explain that current ways to check scientific paper quality either rely on slow, biased peer review or on open sharing without any checks, both of which have problems. To improve this, they created sciwrite-lint, a tool that runs on a researcher’s own computer to automatically check if references in a paper are real, not retracted, and actually support the claims made. They tested the tool on papers from arXiv and bioRxiv and also proposed an experimental SciLint Score to rate both honesty and scientific contribution based on philosophy of science ideas. The focus of their work is on verifying the integrity of scientific papers with their tool, while the contribution score is still in development.
peer reviewopen sciencescientific integrityreference verificationretracted paperslit reviewphilosophy of sciencePopperarXivbioRxiv
Authors
Sergey V Samsonau
Abstract
Science currently offers two options for quality assurance, both inadequate. Journal gatekeeping claims to verify both integrity and contribution, but actually measures prestige: peer review is slow, biased, and misses fabricated citations even at top venues. Open science provides no quality assurance at all: the only filter between AI-generated text and the public record is the author's integrity. AI-assisted writing makes both worse by producing more papers faster than either system can absorb. We propose a third option: measure the paper itself. sciwrite-lint (pip install sciwrite-lint) is an open-source linter for scientific manuscripts that runs entirely on the researcher's machine (free public databases, a single consumer GPU, and open-weights models) with no manuscripts sent to external services. The pipeline verifies that references exist, checks retraction status, compares metadata against canonical records, downloads and parses cited papers, verifies that they support the claims made about them, and follows one level further to check cited papers' own bibliographies. Each reference receives a per-reference reliability score aggregating all verification signals. We evaluate the pipeline on 30 unseen papers from arXiv and bioRxiv with error injection and LLM-adjudicated false positive analysis. As an experimental extension, we propose SciLint Score, combining integrity verification with a contribution component that operationalizes five frameworks from philosophy of science (Popper, Lakatos, Kitcher, Laudan, Mayo) into computable structural properties of scientific arguments. The integrity component is the core of the tool and is evaluated in this paper; the contribution component is released as experimental code for community development.