-
Open Access: Making all published outputs freely accessible for maximum use and impact.
-
Open Scholarship: An extension to Open Research which relates to making other aspects of scientific research open to the public such as open educational resources, having inclusive practice and citizen science.
-
Reproducibility and replicability The Turing Way defines reproducible research as work that can be independently recreated from the same data and the same code that the original team/individual researcher used.
.The multiple dimensions in the image above are defined as follows:
-
Reproducible: A result is reproducible when the same analysis steps performed on the same dataset consistently produce the same answer.
-
Replicable: A result is replicable when the same analysis performed on different datasets produces qualitatively similar answers.
-
Robust: A result is robust when the same dataset is subjected to different analysis workflows to answer the same research question (for example one pipeline written in R and another written in Python) and a qualitatively similar or identical answer is produced. Robust results show that the work is not dependent on the specificities of the programming language chosen to perform the analysis.
-
Generalizable: Combining replicable and robust findings allow us to form generalizable results. Note that running an analysis on a different software implementation and with a different dataset does not provide generalized results. There will be many more steps to know how well the work applies to all the different aspects of the research question. Generalization is an important step towards understanding that the result is not dependent on a particular dataset or a particular version of the analysis pipeline.
Note More information on these definitions can be found in “Reproducibility vs. Replicability: A Brief History of a Confused Terminology” by Hans E. Plesse.
-
