What is rOpenSci Software Peer Review?
rOpenSci’s suite of packages is partly contributed by staff members and partly contributed by community members, which means the suite stems from a great diversity of skills and experience of developers. How to ensure quality for the whole set? That’s where software peer review comes into play: packages contributed by the community undergo a transparent, constructive, non adversarial and open review process. For that process relying mostly on volunteer work, associate editors manage the incoming flow and ensure progress of submissions; authors create, submit and improve their package; reviewers, two per submission, examine the software code and user experience. This blog post written by rOpenSci editors is a good introduction to rOpenSci software peer review Other blog posts about review itself and reviewed packages can be find via the “software-peer-review” tag on rOpenSci blog.
You can recognize rOpenSci packages that have been peer-reviewed via a green “peer-reviewed” badge in their README, linking to their reviews (cf this example); and via a blue comment icon near their description on rOpenSci packages page, also linking to the reviews.
Technically, we make the most of GitHub infrastructure: each package review process is an issue in the ropensci/software-review GitHub repository. For instance, click here to read the review thread of the ropenaq package: the process is an ongoing conversation until acceptance of the package, with two external reviews as important milestones. Furthermore, we use GitHub features such as the use of issue templates (as submission templates), and labelling which we use to track progress of submissions (from editor checks to approval).
Why submit your package to rOpenSci?
- First, and foremost, we hope you submit your package for review because you value the feedback. We aim to provide useful feedback to package authors and for our review process to be open, non-adversarial, and focused on improving software quality.
- Once aboard, your package will continue to receive support from rOpenSci members. You’ll retain ownership and control of your package, but we can help with ongoing maintenance issues such as those associated with updates to R and dependencies and CRAN policies.
- rOpenSci will promote your package through our webpage, blog, and social media (like Mastodon and LinkedIn). Packages in our suite also get a documentation website that is automatically built and deployed after each push.
- rOpenSci packages can be cross-listed with other repositories such as CRAN and BioConductor.
- rOpenSci packages that are in scope for the Journal of Open-Source Software and add the necessary accompanying short paper, would, at the discretion of JOSS editors, benefit from a fast-tracked review process.
- If you write one, rOpenSci will promote gitbooks related to your package: the source of such books can be transferred to the
ropensci-books GitHub organisation for books to be listed at books.ropensci.org.
Why review packages for rOpenSci?
- As in any peer-review process, we hope you choose to review to give back to the rOpenSci and scientific communities. Our mission to expand access to scientific data and promote a culture of reproducible research is only possible through the volunteer efforts of community members like you.
- Review is a two-way conversation. By reviewing packages, you’ll have the chance to continue to learn development practices from authors and other reviewers.
- The open nature of our review process allows you to network and meet colleagues and collaborators through the review process. Our community is friendly and filled with supportive members expert in R development and many other areas of science and scientific computing.
- To volunteer to be one of our reviewers, fill out this short form providing your contact information and areas of expertise. We are always looking for more reviewers with both general package-writing experience and domain expertise in the fields where packages are used.
Why are reviews open?
Our reviewing threads are public. Authors, reviewers, and editors all know each other’s identities. The broader community can view or even participate in the conversation as it happens. This provides an incentive to be thorough and provide non-adversarial, constructive reviews. Both authors and reviewers report that they enjoy and learn more from this open and direct exchange. It also has the benefit of building a community. Participants have the opportunity to meaningfully network with new peers, and new collaborations have emerged via ideas spawned during the review process.
We are aware that open systems can have drawbacks. For instance, in traditional academic review, double-blind peer review can increase representation of female authors, suggesting bias in non-blind reviews. It is also possible reviewers are less critical in open review. However, we posit that the openness of the review conversation provides a check on review quality and bias; it’s harder to inject unsupported or subjective comments in public and without the cover of anonymity. Ultimately, we believe that having direct and public communication between authors and reviewers improves quality and fairness of reviews.
Furthermore, authors and reviewers have the ability to contact privately the editors if they have any doubt or question.
How will users know a package has been reviewed?
- Your package README will feature a peer-review badge linking to the software review thread.
- Your package will get a
docs.ropensci.org docs website that you can link from DESCRIPTION.
- Your package repo will be transferred to the rOpenSci organization.
- If reviewers agree to be listed in DESCRIPTION, their metadata will mention the review.
Editors and reviewers
Associate editors
rOpenSci’s Software Peer Review process is run by:
-
Noam Ross, EcoHealth Alliance
-
Karthik Ram, rOpenSci
-
Maëlle Salmon, rOpenSci
-
Mark Padgham, rOpenSci
-
Anna Krystalli, University of Sheffield RSE
-
Melina Vidoni, RMIT University (School of Science)
-
Mauro Lepore, 2 Degrees Investing Initiative
-
Laura DeCicco, USGS
-
Julia Gustavsen, Agroscope
-
Emily Riederer, Capital One
-
Adam Sparks, Department of Primary Industries and Regional Development
-
Jeff Hollister, US Environmental Protection Agency