Dr. Marcel Böhme
Are we in a reproducibility crisis? Many scientific disciplines are. Earlier research findings cannot be reproduced or are not reproducible in the first place. However, the software engineering community has the unique opportunity to provide almost all artefacts that are required for successfully reproducing the entire set of a paper's findings. All our artefacts are digital. For instance, there are some excellent guidelines on how to effectively package all artefacts required for reproducing the findings.
This is a grassroots effort to inspire collective action from the local level. In terms of governance, reproducibility and open science have gained massive support at the community level, as well. For instance, the ROSE Festival celebrates replication in software engineering. The ACM provides artifact badges for repeatable, replicable, and reproducible artifacts that are published together with the article. Flagship software engineering conferences, such as ESEC/FSE'19 have Open Science guidelines, dedicated Open Science chairs, and Artefact Evaluation committees.
The reproducibility of research is at the very heart of the scientific method. In specific instances, it is fairly reasonable to avoid sharing data, scripts, and tools, e.g., if the authors wish to develop and commercialize the evaluated prototype or if the authors are bound by non-disclosure agreements. However, the general attitude towards science should be one of openness and sharing. We should be allowed to inspect and build upon each others findings. If a central claim in the paper is supported by empirical evidence, we should be able to provide that empirical evidence (within reason). If we are uncomfortable sharing a research prototype, what does it mean for the validity of empirical evidence? There is no need to protect. Science is not a competition.Other links: