PsyPost, a psychology and neuroscience news website, has implemented a new automated Python program designed to monitor its archive for scientific studies that have been retracted, corrected, or flagged by publishers.
During its initial run against the databases of Retraction Watch and Crossref, the program reviewed PsyPost’s entire archive of more than 16,000 articles and identified six covered studies that had been retracted. Following this initial audit, the program now runs on a consistent schedule to answer one specific question: has any study covered by PsyPost been withdrawn or altered, and which articles are affected?
By automating this check, PsyPost aims to maintain the scientific integrity of its content, ensuring readers are informed when the status of previously covered research changes.
How the Program Works
The audit process runs weekly in four steps to translate publisher data into a direct checklist for the editorial team:
1. Import
The program scans PsyPost’s WordPress archive, looking for the scientific studies linked in each article, and pulls the Digital Object Identifier (DOI) for every study. Because publishers format links differently, the program uses specific rules to find DOIs whether they are clearly listed or embedded in the URLs of publishers like SAGE, Wiley, Springer, Frontiers, and PLOS. For sites that don’t display a DOI directly, like Elsevier, PubMed, or Nature, the program uses lookup tools from Crossref and the National Center for Biotechnology Information (NCBI) to find the correct identifier. The program currently monitors approximately 17,000 distinct scientific studies.
2. Check
The program cross-references these DOIs against two primary sources. First, it checks a local download of the Retraction Watch database—which has been made freely available through Crossref since 2023—to identify retractions and their corresponding reason codes. Second, it queries Crossref’s publisher update logs to find corrections and expressions of concern, which Retraction Watch doesn’t track comprehensively.
3. Triage
A raw check produces hundreds of records, many of which are administrative changes like minor typo corrections, updated links, or metadata glitches. The program sorts these events, separating retractions and expressions of concern from minor corrections. It also filters by timing, separating notices that existed before PsyPost covered the study from those that occurred after publication. The result is a concise table showing the affected study, the specific notice, and the exact PsyPost article that requires an update.
4. Close
Once an editor reviews the alert, a notice is placed at the top of the affected PsyPost article, explaining the retraction or correction to the reader. In the system, the event is marked as handled, and the next digest will only display new alerts.
Handling False Alarms
The program is maintained and refined by human oversight. When the system returns a false positive—such as a publisher temporarily pulling and reinstating a paper due to a formatting error—a developer reviews the issue and writes a new rule in the code to ignore that specific class of administrative error in the future. By fixing the class of error rather than just the single instance, the tool produces fewer false alarms over time.
“Science is a self-correcting process, and when the status of a study changes, our articles need to reflect that,” said Eric W. Dolan, the founder and editor of PsyPost. “By regularly checking our archive against Crossref and Retraction Watch, this program allows us to systematically apply those corrections and provide our readers with an accurate reflection of the current scientific record.”