On doing relevant and rigorous experiments: Review and recommendations

Details

Ressource 1Download: JOpsM_manuscript SERVAL 2018.pdf (910.48 [Ko])
State: Public
Version: author
License: Not specified
Serval ID
serval:BIB_C55BB4D70D11
Type
Article: article from journal or magazin.
Collection
Publications
Institution
Title
On doing relevant and rigorous experiments: Review and recommendations
Journal
Journal of Operations Management
Author(s)
Lonati S., Quiroga B.F., Zehnder C., Antonakis J.
ISSN
0272-6963
1873-1317
Publication state
Published
Issued date
11/2018
Peer-reviewed
Oui
Volume
64
Number
1
Pages
19-40
Language
english
Abstract
Although experiments are the gold standard for establishing causality, several threats can undermine the internal validity of experimental findings. In this article, we first discuss these threats, which include the lack of consequential decisions and outcomes, deception, demand effects and unfair comparisons, as well as issues concerning statistical validity (e.g., minimum sample size per cell, estimating variance correctly). We expose each problem, show potential solutions, and bring to the fore issues of relevance of the findings (i.e., external and ecological validity). Thereafter, we take stock of the state-of-the-science regarding validity threats using a representative sample of 468 recent experiments from 258 articles published in top-tier journals. We compare research practices in three fields of study—management, social psychology, and economics, which regularly use experimental research—to operations management, which has recently begun to use the experimental paradigm. Our results underscore the importance for journals and authors to follow what we identify to be best-practice methodological suggestions (i.e., the “ten commandments” of experimental research). We show that—on average—markers of methodological rigor and generalizability positively and significantly predict the citations received by published articles. Finally, given that experiments are infeasible in some settings, we conclude with a brief review of often overlooked quasi-experimental designs, which are useful for generating strong counterfactuals and hence allow making causal claims in the field.
Keywords
Management Science and Operations Research, Strategy and Management, Industrial and Manufacturing Engineering
Web of science
Create date
19/10/2018 14:42
Last modification date
20/08/2019 16:40
Usage data