Questionnaire for reviewers
Below we list a series of questions to help you conduct your review. You do not have to answer these questions. However, if you use them as a guide for your review report, they will help us to develop a review process that is comprehensive and at least partly consistent between reviewers.
We tested this questionnaire with a group of 10 reviewers. They all told us that it was a great tool to produce a complete and structured review report.
General questions
- Did you read the “guide for reviewers”? (see the Help menu of the thematic PCI or the dedicated blog post)
- Is the manuscript well written?
- Is the description of the rationale and methods clear and comprehensive?
- Are there flaws in the design of the research?
- Are there flaws in the analysis?
- Are there flaws in the interpretation of results?
- Do you have concerns about ethics or scientific misconduct?
- Did you detect a spin on the results, discussion or abstract? (a spin is a way of twisting the reporting of results such that the true nature and range of the findings are not faithfully represented, https://doi.org/10.1073/pnas.1710755115)
- Is something critical missing?
Evaluation of the various components of the article
Title/abstract/introduction
- Does the title clearly reflect the content of the article?
- Does the abstract present the supported findings of the study concerned and no other?
- Does the introduction clearly explain the motivation for the study?
- Is the research question/hypothesis/prediction clearly presented?
- Does the introduction build on relevant recent and past research performed in the field?
Materials and Methods
- Are the methods and analysis described in sufficient detail to allow replication by other researchers?
- Is the experimental plan consistent with the questions?
- Are the statistical analyses appropriate?
- Have you evaluated the statistical scripts and program codes?
Results
- Have you checked the raw data and their associated description?
- Have you run the data transformations and statistical analyses and checked that you get the same results?
- To the best of your ability, can you detect any obvious manipulation of data (e.g. removal)?
- Do the statistical results strongly support the conclusion (p< 10-3 or BF>20)?
- In the case of negative results, was a statistical power analysis (or an appropriate Bayesian analysis) performed?
- Did the authors conduct many experiments but retain only some of the results?
Discussion
- Do the interpretations of the analysis go too far?
- Are the conclusions adequately supported by the results?
- Does the discussion take into account relevant recent and past research performed in the field?
- Did the authors test many hypotheses but consider only a few in the discussion?
References
- Are all the references appropriate?
- Are the necessary references present?
- Do the references seem accurate?
Tables and figures
- Are the tables and figures clear and comprehensive?
- Are all the tables/figures useful?
- Are there too many/too few tables and figures?
- Do the tables and figures have suitable captions such that they can be understood without having to read the main text?
Peer review of the revised version of a preprint
- Did the authors correctly address all your comments?
- Do you disagree with the other reviewer(s) and agree with the authors’ explanation and defense of their original article?