Pot la psicologia rescatar-se a si mateixa?. Incentius, biaix i replicabilitat

  1. Blanco Bregón, Fernando
  2. Perales López, José César
  3. Vadillo, Miguel A.
Revista:
Anuari de psicologia de la Societat Valenciana de Psicologia

ISSN: 1135-1268

Año de publicación: 2017

Título del ejemplar: Homenatge a L. Kohlberg i D. Kahneman

Volumen: 18

Número: 2

Páginas: 231-252

Tipo: Artículo

Otras publicaciones en: Anuari de psicologia de la Societat Valenciana de Psicologia

Resumen

En los últimos años la Psicología está sufriendo, interna y externamente, una importante crisis de credibilidad, a la que tampoco han sido ajenas otras ciencias como la Medicina o la Biología. Varios proyectos de ciencia colaborativa sugieren que gran parte de los resultados de la investigación son difí­ ciles de reproducir. Esto viene acompañado de un conjunto de simulaciones y análisis cuantitativos que sitúan la cantidad de falsos positivos por encima del 50% del total de datos publicados, en la literatura psicológica actual. En este breve trabajo realizaremos un análisis actualizado de esta situación, e intentaremos identificar las causas psicológicas que han contribuido a la misma. Entre éstas destacan, primero, un sistema de incentivos alineado más con los intereses de promoción del investigador que con el descubrimiento y diseminación de una ciencia transparente, fiable y reproducible y, segundo, los sesgos individuales en la elaboración de juicios y toma de decisiones, que afectan de forma generalizada a los seres humanos –investigadores o no–. Terminamos discutiendo cómo estos sesgos, en concordancia con la investigación clásica sobre sesgos y heurísticos, se acumulan a través de individuos y acaban afectando de forma sustancial a la empresa colectiva de la ciencia psicológica. Enumeramos también las estrategias que podrían contribuir, de implementarse con éxito, a corregir en parte la situación, e incluso extenderse a otras áreas científicas.

Referencias bibliográficas

  • Alberts, B. (2013). Impact Factor Distortions. Science, 340(6134), pp. 787­787. American Society for Cell Biology (2013). The San Francisco Declaration on Research Assessment (DORA). Recuperat el 26 de desembre de 2016 de <http://am.ascb.org/dora/>.
  • Anderson, M. S.; Ronning, E. A.; De Vries, R. i Martinson, B. C. (2007). The perverse effects of competition on scientists’ work and relationships. Sci­ ence and Engineering Ethics, 13(4), pp. 437­461.
  • Arkes, H. R. (2013). The consequences of the hindsight bias in medical decision making. Current Directions in Psychological Science, 22, pp. 356­360.
  • Badenes-Ribera, L.; Frias-Navarro, D.; Iotti, B.; Bonilla-Campos, A. i Longo­ bardi, C. (2016). Misconceptions of the p­value among Chilean and Italian Academic Psychologists. Frontiers in Psychology, 7, a1247.
  • Badenes-Ribera, L.; Frías-Navarro, D. i Pascual-Soler, M. (2015). Errors d’in­ terpretació dels valors p en estudiants universitaris de Psicologia. Anuari de Psicologia, 16(2), pp. 15­31. doi: 10.7203/anuari.Psicologia.16.2.15.
  • Baker, M. (2015, abril). First results from psychology’s largest reproducibility test. Nature News. Recuperat el 26 de desembre de 2016 de <http://www.nature.com/news/first-results-from-psychology-s-largest-reproducibilitytest­1.17433>.
  • Bakker, M.; Hartgerink, C. H. J.; Wicherts, J. M. i van der Maas, H. L. J. (2016). Researchers’ intuitions about power in psychological research. Psychologi­cal Science, 27(8), pp. 1069­1077.
  • Bakker, M.; van Dijk, A. i Wicherts, J. M. (2012). The Rules of the Game Called Psychological Science. Perspectives on Psychological Science, 7(6), pp. 543­554.
  • Bargh, J. A.; Chen, M. i Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype­ activation on action. Journal of Personality and Social Psychology, 71(2), pp. 230­244.
  • Baumeister, R. F. (2016). Charting the future of social psychology on stormy seas: Winners, losers, and recommendations. Journal of Experimental So­cial Psychology, 66, pp. 153­158.
  • Brembs, B.; Button, K. i Munafò, M. (2013). Deep impact: unintended conse­ quences of journal rank. Frontiers in Human Neuroscience, 7, a291.
  • Button, K. S.; Ioannidis, J. P. A.; Mokrysz, C.; Nosek, B. A.; Flint, J.; Robinson, E. S. J. i Munafò, M. R. (2013). Power failure: Why small sample size un­dermines the reliability of neuroscience. Nature Reviews, 14, pp. 365­376.
  • Camerer, C. F.; Dreber, A.; Forsell, E.; Ho, T. H.; Huber, J.; Johannesson, M. i Wu, H. (2016). Evaluating replicability of laboratory experiments in eco­ nomics. Science, 351, pp. 1433­1436.
  • Chambers, C. D. (2013). Registered reports: A new publishing initiative at Cor­tex. Cortex, 49, pp. 609­610.
  • Chinn, C. A. i Brewer, W. F. (1998). An empirical test of a taxonomy of respon­ses to anomalous data in science. Journal of Research in Science Teaching, 35, pp. 623­654.
  • Cohen, J. (1990). Things I have learned (thus far). American Psychologist, 45, pp.1304­1312.
  • Coyne, J. C.; Thombs, B. D. i Hagedoorn, M. (2010). Ain’t necessarily so: Re­view and critique of recent meta­analyses of behavioral medicine interven­tions in health psychology. Health Psychology, 29(2), pp. 107­116.
  • Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), pp. 7­29.
  • DeKay, M. L. (2015). Predecisional information distortion and the self-fulfilling prophecy of early preferences in choice. Current Directions in Psychologi­cal Science, 24, pp. 405­411.
  • Dickersin, K. (2005). Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm. En Rothstein, H. R.; Sutton, A. J. i Borenstein, M. (eds.), Publication bias and meta­analysis: Prevention, assessment and adjustments (pp. 11­33). Nova York: John Wiley and Sons.
  • Dreber, A.; Pfeiffer, T.; Almenberg, J.; Isaksson, S.; Wilson, B.; Chen, Y. i Jo­ hannesson, M. (2015). Using prediction markets to estimate the reproduci­bility of scientific research. Proceedings of the National Academy of Scien­ces, 112(50), pp. 15343­15347.
  • Enserink, M. (2012, November). Final Report: Stapel Affair Points to Bigger Problems in Social Psychology. Science Insider, Recuperat el 26 de desem­bre de 2016 de <http://www.sciencemag.org/news/2012/11/final-reportstapel­affair­points­bigger­problems­social­psychology>.
  • Errington, T. M.; Iorns, E.; Gunn, W.; Tan, F. E.; Lomax, J. i Nosek, B. A. (2014). An open investigation of the reproducibility of cancer biology re­search. eLife, 3, e04333.
  • Etz, A.; Gronau, Q. F.; Dablander, F.; Edelsbrunner, P. A. i Baribault, B. (2016). Understanding Bayes: How to become a Bayesian in eight easy steps: An annotated reading list. Recuperat el 26 de desembre de 2016 de <https://alexanderetz.com/2016/02/07/understanding­bayes­how­to­become­a­bayesian­in­eight­easy­steps/>.
  • Fanelli, D. (2010). Do pressures to publish increase scientists’ bias? An empiri­cal support from US states data. PLoS One, 5(4), e10271.
  • Fischhoff, B. (1975). Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty. Journal of Experimental Psy­chology: Human Perception and Performance, 1, pp. 288­299.
  • Fiske, S. T. (2016). A call to change science’s culture of shaming. APS Obser­ver, 29(9), pp. 5­6.
  • Fons­Rosen, C. i Azoulay, P. (2015). Does Science Advance One Funeral at a Time? National Bureau of Economic Research, Working Paper, 21788. Recuperat el 26 de desembre de 2016 de <http://www.nber.org/papers/w21788>.
  • Franco, A.; Malhotra, N. i Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), pp. 1502­1505.
  • Galton, F. (1907a). Vox populi (the wisdom of crowds). Nature, 75, pp. 450­ 451.
  • Galton, F. (1907b). One vote, one value. Nature, 75, pp. 414­414.
  • Gargouri, Y.; Hajjem, C.; Lariviére, V.; Gingras, Y.; Carr, L.; Brody, T. i Harnad, S. (2010). Self­selected or mandated, open access increases citation impact for higher quality research. PLoS ONE, 5(10), e13636.
  • Gelman, A. i Loken, E. (2014). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no «fishing expedition» or «p­hacking» and the research hypothesis was posited ahead of time. Psychological Bulletin, 140(5), pp. 1272­1280.
  • Gigerenzer, G. (2004). Mindless statistics. Journal of Socio­Economics, 33(5), pp. 587­606.
  • Higgins, J. P. i Green, S. (eds.) (2008). Cochrane handbook for systematic revi­ews of interventions. Chichester: Wiley­Blackwell.
  • Higginson, A. D. i Munafò, M. R. (2016). Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions. PLOS Biology, 14(11), e2000995.
  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), pp. 16569­16572.
  • Hoekstra, R.; Finch, S.; Kiers, H. A. L. i Johnson, A. (2006). Probability as certainty: Dichotomous thinking and the misuse of p values. Psychonomic Bulletin & Review, 13, pp. 1033­1037.
  • Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.
  • Ioannidis, J. P. A. (2012). Why science is not necessarily self­correcting. Pers­pectives on Psychological Science, 7, pp. 645­654.
  • John, L. K.; Loewenstein, G. i Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psycholo­gical Science, 23, pp. 524­532.
  • Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Per­sonality and Social Psychology Review, 2, pp. 196­217.
  • Kidwell, M. C.; Lazarević, L. B.; Baranski, E.; Hardwicke, T. E.; Piechowski, S.; Falkenberg, L. S. i Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low­cost, effective method for increasing transparency. PLoS Biology, 14(5), e1002456. doi: 10.1371/journal.pbio.1002456.
  • Kirschner, M. (2013). A Perverted View of «Impact». Science, 340(6138), pp. 1265­1265.
  • Klein, R.; Ratliff, K.; Vianello, M.; Adams Jr, R.; Bahník, S.; Bernstein, M. i Cemalcilar, Z. (2014). Data from investigating variation in replicability: A «Many Labs» Replication Project. Journal of Open Psychology Data, 2(1), e4.
  • Lorenz, J.; Rauhut, H.; Schweitzer, F. i Helbing, D. (2011). How social influen­ ce can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences, 108(22), pp. 9020­9025.
  • Lurquin, J. H.; Michaelson, L. E.; Barker, J. E.; Gustavson, D. E.; Von Bas­ tian, C. C.; Carruth, N. P. i Miyake, A. (2016). No Evidence of the EgoDepletion Effect across Task Characteristics and Individual Differences: A Pre­Registered Study. PloS One, 11(2), e0147770.
  • Meehl, P. E. (1978). Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ro­nald, and the Slow Progress of Soft Psychology. Journal of Consulting and Clinical Psychology, 46, pp. 806­834.
  • Myers, D. G. i Lamm, H. (1976). The group polarization phenomenon. Psycho­logical Bulletin, 83(4), pp. 602­627.
  • Nelson, N.; Rosenthal, R. i Rosnow, R. L. (1986). Interpretation of significance levels and effect sizes by psychological researchers. American Psycholo­gist, 41, pp. 1299­1301.
  • Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2, pp. 175­220.
  • Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
  • PLoS Blogs. (2015, February 25). Positively Negative: A New PLOS ONE Collection focusing on Negative, Null and Inconclusive Results. Recuperat el 26 de desembre de 2016 de <http://blogs.plos.org/everyone/2015/02/25/positively­negative­new­plos­one­collection­focusing­negative­null­inconclusive­results/>.
  • Prinz, F.; Schlange, T. i Asadullah, K. (2011). Believe it or not: How much can we rely on publishd data on potential drug targets? Nature Reviews Drug Discovery, 10, p. 712.
  • Ranehill, E.; Dreber, A.; Johannesson, M.; Leiberg, S.; Sul, S. i Weber, R. A. (2015). Assessing the Robustness of Power Posing No Effect on Hormones and Risk Tolerance in a Large Sample of Men and Women. Psychological Science, 26(5), pp. 653­656.
  • Rohrer, D.; Pashler, H. i Harris, C. R. (2015). Do subtle reminders of money change people’s political views? Journal of Experimental Psychology: Ge­neral, 144, e73­e85.
  • Schervish, M. J. (1996). P­Values: What They Are and What They Are Not. The American Statistician, 50(3), pp. 203­206.
  • Sedlmeier, P. i Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, pp. 309­316.
  • Shanks, D. R.; Vadillo, M. A.; Riedel, B.; Clymo, A.; Govind, S.; Hickin, N.; ... i Puhlmann, L. (2015). Romance, risk, and replication: Can consumer choi ces and risk­taking be primed by mating motives? Journal of Experimental Psychology: General, 144(6), e142­e158.
  • Simmons, J. P.; Nelson, L. D. i Simonsohn, U. (2011). False-positive psycho­logy: Undisclosed flexibility in data collection and analysis allows pre­senting anything as significant. Psychological Science, 22, pp. 1359­1366.
  • Smolčić, V. Š. (2013). Salami publication: definitions and examples. Biochemia Medica, 23(3), pp. 137­141.
  • Sterling, T. D.; Rosenbaum, W. L. i Weinkam, J. J. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. American Statistician, 49, pp. 108­112.
  • Surowiecki, J. (2004). The Wisdom of Crowds. Nova York: Anchor.
  • Szucs, D. i Ioannidis, J. P. (2016). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology lite­rature. bioRxiv. doi: doi.org/10.1101/071530.
  • Van Bavel, J. J.; Mende-Siedlecki, P.; Brady, W. J. i Reinero, D. A. (2016). Con­textual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences, 113, pp. 6454­6459.
  • Van der Gucht, K.; Griffith, J. W.; Hellemans, R.; Bockstaele, M.; PascalClaes, F. i Raes, F. (2016). Acceptance and Commitment Therapy (ACT) for Adolescents: Outcomes of a Large­Sample, School­Based, Cluster­ Randomized Controlled Trial. Mindfulness. Advance online publication. doi: 10.1007/s12671­016­0612­y.
  • Van Dijk, D.; Manor, O. i Carey, L. B. (2014). Publication metrics and success on the academic job market. Current Biology, 24(11), R516­R517.
  • Wagenmakers, E.­J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin y Review, 14(5), pp. 779­804.
  • Wicherts, J. M.; Veldkamp, C. L. S.; Augusteijn, H. E. M.; Bakker, M.; van Aert, R. C. M. i van Assen, M. A. L. M. (2016). Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking. Frontiers in Psychology, 7, p. 1832. doi: 10.3389/fpsyg.2016.01832.
  • Young, E. (2012, October). Nobel laureate challenges psychologists to clean up their act. Nature News. Recuperat el 26 de desembre de 2016 de <http://www.nature.com/news/nobel­laureate­challenges­psychologists­to­clean­up­their­act­1.11535>.
  • Zatonski, T. i Witkowski, T. (2015). Psychology Gone Wrong: The Dark Sides of Science and Therapy. Boca Raton, FL, US: Brown Walker Publishers.
  • Zehetleitner, M. i Schönbrodt, F. (2016). When does a significant p-value indi­cate a true effect? Understanding the Positive Predictive Value (PPV) of a p­value. <http://shinyapps.org/apps/PPV/>.