Manipulating the Alpha Level Cannot Cure Significance Testing

David Trafimow 1 Valentin Amrhein 2, 3 Corson Areshenkoff 4 Carlos Barrera-Causil 5 Eric Beh 6 Yusuf Bilgiç 7 Roser Bono 8 Michael Bradley 9 William Briggs Héctor Cepeda-Freyre 10 Sergio Chaigneau 11 Daniel Ciocca Juan Correa 12 Denis Cousineau 13 Michiel de Boer 14 Subhra Dhar 15 Igor Dolgov Juan Gómez-Benito Marian Grendar James Grice Martin Guerrero-Gimenez Andrés Gutiérrez Tania Huedo-Medina Klaus Jaffe 16 Armina Janyan Ali Karimnezhad Fränzi Korner-Nievergelt Koji Kosugi Martin Lachmair Rubén Ledesma Roberto Limongi Marco Liuzza Rosaria Lombardo Michael Marks Gunther Meinlschmidt Ladislas Nalborczyk 17 Hung Nguyen 18 Raydonal Ospina Jose Perezgonzalez Roland Pfister Juan Rahona David Rodríguez-Medina Xavier Romão Susana Ruiz-Fernández Isabel Suarez Marion Tegethoff Mauricio Tejo Rens van de Schoot 19, 20 Ivan Vankov 21 Santiago Velasco-Forero 22 Tonghui Wang 1 Yuki Yamada 23 Felipe Zoppino 23 Fernando Marmolejo-Ramos 24
Abstract : We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01957088
Contributor : Santiago Velasco-Forero <>
Submitted on : Monday, December 17, 2018 - 10:02:35 AM
Last modification on : Wednesday, September 18, 2019 - 9:42:09 AM

Identifiers

  • HAL Id : hal-01957088, version 1

Citation

David Trafimow, Valentin Amrhein, Corson Areshenkoff, Carlos Barrera-Causil, Eric Beh, et al.. Manipulating the Alpha Level Cannot Cure Significance Testing. Frontiers in Psychology, Frontiers, 2018, 9. ⟨hal-01957088⟩

Share

Metrics

Record views

265