October 25, 2022

Dear Colleagues:

A 2019 consensus study report published by the National Academies of Sciences, Engineering, and Medicine (NASEM) discussed the meaning of the terms replicability and reproducibility and identified approaches for researchers, academic institutions, journals, and funders to improve reproducibility and replicability in science ^[1]. In July 2021, at NSF's request, NASEM convened an expert meeting focused on National Science Foundation (NSF) policies and investments to make reproducible and replicable science easier for scientific communities to understand and execute and to embed reproducibility and replicability within the fundamental scientific method.

Through this Dear Colleague Letter (DCL), NSF reaffirms its commitment to advancing reproducibility and replicability in science. NSF is particularly interested in proposals addressing one or more of the following topics:

1. Advancing the science of reproducibility and replicability. Understanding current practices around reproducibility and replicability, including ways to measure reproducibility and replicability, what reproduction and replication means in practice, the right degree of replicability to target, quantitative measures of progress to understand the effectiveness of interventions to improve reproducibility and replicability, and exploration of reasons why studies may fail to replicate.
2. Research infrastructure for reproducibility and replicability. Developing and facilitating adoption of cyberinfrastructure tools and/or research methods that enable use of reproducible and replicable practices across one or more science and engineering communities.
3. Educational efforts to build a scientific culture that supports reproducibility and replicability. Enabling training in science and engineering communities to identify and encourage best practices for reproducibility and replicability, providing community-building and institutional support, and supporting broad public outreach about rigor, reproducibility, and replicability in science.

Investigators who wish to submit proposals on any of these topics, or others related to advancing reproducibility and replicability in research, are encouraged to reach out to programs and program officers to discuss the fit of their ideas to existing funding opportunities. Definitions of the terms replicability and reproducibility may be found in Reference ^[1].

Sincerely,

Simon Malcomber, Acting Assistant Director
Directorate for Biological Sciences (BIO)

Margaret Martonosi, Assistant Director
Directorate for Computer & Information Science & Engineering (CISE)

James L. Moore III, Assistant Director
Directorate for Education & Human Resources (EHR)

Susan S. Margulies, Assistant Director
Directorate for Engineering (ENG)

Alexandra R. Isern, Assistant Director
Directorate for Geosciences (GEO)

Alicia J. Knoedler, Office Head
Office of Integrative Activities (OIA)

Kendra V. Sharp, Office Head
Office of International Science and Engineering (OISE)

Sean L. Jones, Assistant Director
Directorate for Mathematical & Physical Sciences (MPS)

Kellina M. Craig-Henderson, Assistant Director
Directorate for Social, Behavioral, & Economic Sciences (SBE)

Erwin Gianchandani, Assistant Director
Directorate for Technology, Innovation and Partnerships (TIP)

References

[1] Reproducibility and Replicability in Science, National Academies of Sciences, Engineering, and Medicine, 2019. https://doi.org/10.17226/25303