Please use this identifier to cite or link to this item:
http://hdl.handle.net/1942/42637
Title: | Estimating the extent of selective reporting: An application to economics | Authors: | BRUNS, Stephan DERESSA, Teshome Stanley, T. D. Doucouliagos, Chris Ioannidis, John P. A. |
Issue Date: | 2024 | Publisher: | WILEY | Source: | Research Synthesis Methods, | Status: | Early view | Abstract: | Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies. | Notes: | Bruns, SB (corresponding author), Hasselt Univ, Ctr Environm Sci, Hasselt, Belgium. stephan.bruns@uhasselt.be |
Keywords: | meta-research;p-hacking;publication bias;selective reporting | Document URI: | http://hdl.handle.net/1942/42637 | ISSN: | 1759-2879 | e-ISSN: | 1759-2887 | DOI: | 10.1002/jrsm.1711 | ISI #: | 001173562200001 | Rights: | 2024 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. | Category: | A1 | Type: | Journal Contribution |
Appears in Collections: | Research publications |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
Research Synthesis Methods - 2024 - Bruns - Estimating the extent of selective reporting An application to economics.pdf | Early view | 1.2 MB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.