Wednesday Seminar Series: Dr. Douglas Campbell, New Economic School in Moscow

Robust or just bust? The robustness reproducibility of the American Economic Review

Wednesday, January 31, 2024 12:00 PM to 1:15 PM EST
Carow Hall, 1

Wednesday Seminar Series: Dr. Douglas Campbell, New Economic School in Moscow

The Wednesday Seminar Series, now over 40 years old, invites scholars from around the world to share their work in a formal lecture, which is followed by a rigorous question and answer session. The Spring 2024 seminars will be hosted by Dr. Jonathan Schulz and coordinated by Lisa Hill-Corley. All papers made available will be posted on this site for downloading. For more information, call the Center: (703) 993-2316.

In person seminars will be held from 12 - 1:15 PM in Carow Hall, which is on George Mason University's Fairfax campus. 

A light lunch for seminar participants will be available at 11:30 AM.

Click here to see a list of our Spring 2024 schedule of speakers.

 

Abstract

We estimate the robustness reproducibility of 17 non-experimental AER papers published in 2013 (8 papers) and 2022/23 (9 papers). We find that most empirical papers published in the AER are not robust, with no improvement over time. The fraction of significant robustness tests (p<0.05) varies between 17% and 86% across the papers with a mean of 46%. The mean relative t/z-value of the robustness tests varies between 34% and 87% with a mean of 62%, suggesting that the papers selectively report analytical specifications that exaggerate effect sizes and statistical significance. A sample of economists (n=359) overestimates robustness reproducibility, but predictions of individual papers are correlated with observed reproducibility.

Add this event to your calendar