Open Practice Badges: A Primer and How to Get Started
/The Center for Open Science (COS) provides tools, training, support, and advocacy that help researchers and scholars manage, share, and discover scientific research. The COS’ mission is to “increase the openness, integrity, and reproducibility of scholarly research. Acceleration of scientific progress can be a primary motivator for scholarship and a powerful driver of real solutions.”
The COS develops software tools, workflows, data storage solutions, and more based on its free Open Science Framework (OSF). The OSF is an ecosystem of solutions, partnering companies, technologies, and ideas that support researchers across the entire research life cycle. One initiative that is gaining momentum is the use of Open Practice Badges in the publishing workflow.
“Openness is a core value of scientific practice.”
The scholarly publishing community agrees on the relevance and importance of open communication for scientific research and progress. In 2009 there were approximately 4,800 OA journals publishing approximately 190,000 articles. In January 2017, the estimate is that there are around 9,500 active OA journals. At Cenveo Publisher Services, we work with a large number of society and commercial publishers who have launched or are preparing to add OA publication models to their workflows.
Awarding Open Practice Badges on published content is a way of designating and awarding authors badges that acknowledge their use of open practices during the research life cycle.
Incorporating Open Practice Badges Into Publishing Workflows
By acknowledging open practices in scientific research, journal publishers can use badges in their publications to certify that a particular research practice was followed. Badges can be awarded to the published content as part of the peer review process or they can be awarded post-publication. As long as processes and practices are transparent, any organization can issue badges. Most publishers are awarding the badges during peer review. Publishing platforms and review services are likely to use the badges post publication.
For publishers, the journal awards the badge and it is linked to the specific article. Each publisher tends to have specific methods for incorporating badges into the published article. However, it is critical that the badge is machine discoverable and readable.
Detailed information on incorporating OA badges into your publication workflow can be found at the OSF Wiki page here.
Badge Overview
There are three badges currently used:
Open Data
Open Materials
Preregistered
Following is an overview of the three badges and corresponding criteria. Detailed information is available on the OSF Wiki page, including corresponding links.
Open Data
The Open Data badge is earned for making publicly available the digitally-shareable data necessary to reproduce the reported results.
Criteria
Digitally-shareable data are publicly available on an open-access repository. The data must have a persistent identifier and be provided in a format that is time-stamped, immutable, and permanent (e.g., university repository, a registration on the Open Science Framework, or an independent repository at www.re3data.org).
A data dictionary (e.g., a codebook or metadata describing the data) is included with sufficient description for an independent researcher to reproduce the reported analyses and results. Data from the same project that are not needed to reproduce the reported results can be kept private without losing eligibility for the Open Data Badge.
An open license allowing others to copy, distribute, and make use of the data while allowing the licensor to retain credit and copyright as applicable. Creative Commons has defined several licenses for this purpose, which are described at www.creativecommons.org/licenses. CC0 or CC-BY is strongly recommended.
Open Materials
The Open Materials badge is earned by making publicly available the components of the research methodology needed to reproduce the reported procedure and analysis.
Criteria
Digitally-shareable materials are publicly available on an open-access repository. The materials must have a persistent identifier and be provided in a format that is time-stamped, immutable, and permanent (e.g., university repository, a registration on the Open Science Framework, or an independent repository at www.re3data.org).
Infrastructure, equipment, biological materials, or other components that cannot be shared digitally are described in sufficient detail for an independent researcher to understand how to reproduce the procedure.
Sufficient explanation for an independent researcher to understand how the materials relate to the reported methodology.
Preregistered/Preregistered+Analysis Plan badges
The Preregistered/Preregistered+Analysis Plan badges are earned for preregistering research.
Preregistered
The Preregistered badge is earned for having a preregistered design. A preregistered design includes: (1) Description of the research design and study materials including planned sample size, (2) Description of motivating research question or hypothesis, (3) Description of the outcome variable(s), and (4) Description of the predictor variables including controls, covariates, independent variables (conditions). When possible, the study materials themselves are included in the preregistration.
Criteria for earning the preregistered badge on a report of research are:
A public date-time stamped registration is in an institutional registration system (e.g., ClinicalTrials.gov, Open Science Framework, AEA Registry, EGAP).
Registration pre-dates the intervention.
Registered design and analysis plan corresponds directly to reported design and analysis.
Full disclosure of results in accordance with registered plan.
Badge eligibility does not restrict authors from reporting results of additional analyses. Results from preregistered analyses must be distinguished explicitly from additional results in the report. Notations may be added to badges. Notations qualify badge meaning: TC, or Transparent Changes, means that the design was altered but the changes and rationale for changes are provided. DE, or Data Exist, means that (2) is replaced with “registration postdates realization of the outcomes, but the authors have yet to inspect or analyze the outcomes.
Preregistered+Analysis Plan
The Preregistered+Analysis Plan badge is earned for having a preregistered research design (described above) and an analysis plan for the research and reporting results according to that plan. An analysis plan includes specification of the variables and the analyses that will be conducted. Guidance on construction of an analysis plan is below.
Criteria for earning the preregistered+analysis plan badge on a report of research are:
A public date-time stamped registration is in an institutional registration system (e.g., ClinicalTrials.gov, Open Science Framework, AEA registry, EGAP).
Registration pre-dates the intervention.
Registered design and analysis plan corresponds directly to reported design and analysis.
Full disclosure of results in accordance with the registered plan.
Notations may be added to badges. Notations qualify badge meaning: TC, or Transparent Changes, means that the design or analysis plan was altered but the changes are described and a rationale for the changes is provided. Where possible, analyses following the original specification should also be provided. DE, or Data Exist, means that (2) is replaced with “registration postdates realization of the outcomes, but the authors have yet to inspect or analyze the outcomes.”
What Journals Are Using Open Badges?
A list of journals currently using Open Practice Badges can be found here. The list continues to grow as more publishers understand the benefits of providing this acknowledgement to researchers and readers.
Cenveo Publisher Services is an advocate of Open Practice Badges. If your publishing organization would like to learn how we can support open badges in your workflow, feel free to reach out to us directly.
Are you currently using Open Practice Badges? Please share your findings or observations in the comments section below.