​

Step by step Guide to Reviewing A Manuscript

​

​

It is important to note that as you receive an invitation to peer review, you should be sent a copy of the paper's abstract to help you decide whether you wish to do the review. It is ethical to respond to invitations promptly to prevent delays. At this stage it is really essential to declare any potential Conflict of Interest.

 

Overview of the Review Report Format

 

The review report structure varies between journals. Generally, an informal structure is followed, while, others have a more formal approach.

 

Informal Structure

Basically, journals don't provide any specific criteria for reviews beyond asking for your 'analysis of merits'. If this is the case, then you may wish to familiarize yourself with examples of other reviews done for the journal, which the editor should be able to provide or, gradually as soon as you gain experience, you can rely on your own evolving style.

 

Formal Structure

 

Other journals require a more formal approach. Sometimes even you may be asked to address specific questions in your review via a questionnaire. You might even be wished to rate the manuscript on various attributes using a scorecard. Basically, you are not able to observe these until you log in to submit your review. So soon after accepting to review the work, it's really worth checking for any journal-specific guidelines and requirements. If you receive formal guidelines, it is suggested get them to direct the structure of your review.

 

In Both Cases

 

Even if specifically, the reporting format is provided by the journal it seems important to note that you are expected to compile comments to authors and possibly confidential ones to editors only.

 

The First Read-Through

It is important to move along the invitation to review. As you'll have received the article abstract, you should already understand the aims, key data and conclusions of the manuscript. If you don't, make a note now that you are required to feedback on how to specifically improve those sections.

​

You are suggested the first read-through that is skim-read. It is important to note that you will be assisted if you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper.

 

First Read Considerations

Always Keep a pen and paper handy when skim-reading.

​

Keep the following question in your mind - they'll help you form your overall impression:

 

• Identify the main question addressed by the research? Is it relevant and interesting?

• How original is the topic? Does it add something new to the subject area compared with other published material?

• Is the paper written with appropriate format? Do you find the text clear and easy to be read?

• Do you find the conclusions consistent with the evidence and arguments presented? Do they actually address the main question posed?

• Do you think author has a substantial case, if he is disagreeing significantly with the current academic consensus? If not, what would be required to make their case credible?

• What quality do the tables or figures add to the paper? Do they really aid understanding or are they superfluous?

 

Spotting Potential Major Flaws

While reading the whole paper, it is essential to make the right choice of what to read first can save time by flagging major problems early on.

​

Editors say, "Significant recommendations for remedying flaws are VERY welcome."

​

Major flaws possibly include these Examples:

​

• Drawn a conclusion that is contradicted by the author's own statistical or qualitative evidence

• Discredited method is used.

• Ignored a process that is known to have a strong influence on the area under study

​

In any case experimental design features prominently exist in the paper, then you should first of all check that the methodology is sound - if not, this is likely to be a major flaw.

You might examine:

​

• The sampling in analytical papers

• The sufficient use of control experiments

• The precision of process data

• The regularity of sampling in time-dependent studies

• The validity of questions, and the use of a detailed methodology and the data analysis being done systematically (in qualitative research)

• It is observed that qualitative research always extends beyond the author's opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

 

Major Flaws in Information

If you find methodology has fewer issues, it's often a good idea to look at the data tables, figures or images first. Science research specifically is about the information gathered. It is important to note that if there are some critical flaws then, it's very likely that the manuscript will need to be rejected. Such issues include:
 

• Insufficient data

• Unclear data tables

• Some contradictory data that either are not self-consistent or disagree with the conclusions

• The confirmatory data that really adds little, if anything, to current understanding - unless strong arguments for such repetition are made

If a major problem is observed, note your reasoning and clear supporting evidence (including citations).
 

Concluding the First Reading

Just after the initial reading and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first should be summarizing the research question addressed and the second should be the contribution of the work. If you have received any prescribed reporting format, this draft will still help you compose your thoughts.

​

The First Paragraph

​

In the beginning the main question should be stated and then addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

​

• Help the editor in appropriate manner and contextualize the research and you are advised to add weight to your judgement

• Guide the author about the key messages conveyed to the reader, so that they can be sure they are achieving what they set out to do

• Thoroughly focus on the successful aspects of the paper to convey a sense of whether the author has done well or not.

The Second Paragraph

A conceptual overview should be provided by this regarding the contribution of the research. So, consider:

​

• Is the paper's premise interesting and important?

• Are the methods used appropriate?

• Do the data support the conclusions?

​

Once you have drafted these two paragraphs, now you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). if you find it publishable in principle and merits a detailed, careful read through.

Rejection After the First Reading

If you come to the conclusion that an article has serious flaws, make sure you thoroughly read the whole paper. This really seems significant because you may find some really positive aspects that can be communicated to the author. This could even help them with future submissions.

​

Sometimes a full read-through will make you sure that any initial concerns are indeed correct and fair. At the end you will surely need the context of the whole paper before deciding to reject. Still if you intend to recommend rejection, see the section "When recommending rejection."

​

Before Starting the Second Read-Through

If the paper has passed your first read and you've ultimately decided the article is publishable in principle, an important purpose of the second, detailed read-through is to help prepare the manuscript for publication. You may still decide to reject the paper following a second reading.

​

Here it seems important to note that benchmark for acceptance is whether the manuscript makes a useful contribution to the knowledge base or understanding of the subject matter. It may not be fully complete research - it may even be an interim paper. After all, here we can say that research is an incomplete, on-going project by its nature. It is generally observed that detailed read-through should take no more than an hour for the moderately experienced reviewer.

​

Preparation

Simplify the review just to save your time:

​

• You should not rely solely upon inserting comments on the manuscript document - make separate notes

• You should try to group similar concerns or praise together

• If you have selected a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes - it helps later

• It is suggested to note line numbers of text upon which your notes are based - this helps you find items again and also aids those reading your review

• You should keep images, graphs and data tables in clear view - either print them off or have them in view on a second computer monitor or window.

​

As now you have completed your preparations, you're ready to spend an hour or so reading carefully through the manuscript.

​

Doing the Second Read-Through

It is suggested that if you're reading through the manuscript for a second time, you'll need to keep in mind the argument's construction, the clarity of the language and content.
 

It is significant to note that with regard to the argument’s construction, you should identify:

​

• If you find any places where the meaning is unclear or ambiguous

• If you find any factual errors

• If you find any invalid arguments

 

It is also suggested to consider:

​

• Does the title of the paper properly reflect the subject of the paper?

• Does the abstract seem to provide an accessible summary of the paper?

• Do the keywords of abstract accurately reflect the content?

• Does the length of the paper seem appropriate?

• Do the key messages seem short, accurate and clear?

 

Check the Language
 

It is important to note that every submission may not be well written. Your role here is to make sure that the text’s meaning is clear.

​

Editors say, "If a manuscript has many English language and editing issues, please do not try and fix it. If you find it in worse condition, it is essential to note that in your review and it should be up to the authors to have the manuscript edited."

​

If the article seems difficult to understand, you should have rejected it already. If you observe that the language is poor but it is really easy for you to understand the core message, you should see if you can suggest improvements to fix the problem:

​

• Are there certain specific aspects that could be communicated better, such as parts of the discussion?

• Do you think the authors should consider resubmitting to the same journal after language improvements?

• Would you like to consider looking at the paper again once these issues are dealt with?

​

On Grammar and Punctuation

​

Primary role on your part is judging the research content. Don't spend too much time polishing grammar or spelling. Editors will ensure that the text is at a high standard before publication. If you are fortunate to spot grammatical errors that affect clarity of meaning, then it's important to highlight these. You are expected to suggest such amendments - it's rare for a manuscript to pass review with no corrections.

​

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

​

The Second Read-Through: Section by Section Guidance

​

1. The Introduction

​

A suggested well-written introduction

​

• Sets out the argument perfectly.

• Summarizes recent research relevant to the topic

• Highlights major gaps observed in the current understanding or conflicts in current knowledge.

• Establishes basically the originality of the research aims by demonstrating the need for investigations in the topic area

• Provides a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript

​

Originality and Topicality:

​

It is observed that originality and topicality can be established only in the light of recent authoritative research. For instance, it really seems impossible to argue that there is a conflict in current understanding by referencing articles that are 10 years old.

​

Sometimes it’s possible that authors may make the case by putting an excuse that a topic hasn't been investigated in several years and that new research is required. The point can only seem valid if researchers can point to recent developments in data gathering techniques or to research in indirectly related fields that suggest the topic needs revisiting. Authors can clearly do this only by referencing recent literature. Sometimes where older research seems seminal or where aspects of the methodology rely upon it, obviously, then it seems really appropriate for authors to cite some older papers.

​

Editors say, "Is the report providing new information; is it novel or just confirmatory of well-known outcomes?"

Aims:

​

Generally, the introduction ends by stating the research aims. So, by this point you should surely have a good impression of them - if the explicit aims come at the end as a surprise, then the introduction requires improvement.

​

2. Materials and Methods

​

The academic research is suggested to be replicable, repeatable and robust - and follow best practice.

​

Replicable Research:

​

Observe that it makes sufficient use of-

​

• Control experiments

• Repeated analyses

• Repeated experiments

• Sampling

​

Basically, these are used to ensure that observed trends are not due to chance and that the same experiment could be repeated by other researchers - and result in the same outcome. It is important to note that statistical analyses will not be sound if methods are not replicable. The paper should be recommended for rejection, where research is not replicable.

​

Repeatable Methods:

​

Here enough details are provided so that other researchers are able to carry out the same research. For instance, details of equipment used or sampling methods should all be described so that others could follow the same steps. Where it is observed that methods are not detailed enough, it's genuine to ask for the methods section to be revised.

​

Robust Research:

​

Enough data points are provided here to make sure the data are reliable. If you find insufficient data, it might be appropriate to recommend revision. It is advised to consider whether there is any in-built bias not nullified by the control experiments.

Best Practice:

You should always keep in mind best practice during these checks:

​

• Standard guidelines were strictly followed (e.g., the CONSORT Statement for reporting randomized trials)

• The health and safety of all the participants in the study was not compromised at any cost.

• Ethical standards were thoroughly maintained

​

If the research completely fails to reach relevant best practice standards, it's expected to recommend rejection. What's more, you don't then need to read any further.

​

3. Results and Discussion

​

This section should always tell a coherent story - What happened? What was discovered or confirmed?

Certain patterns of good reporting are expected to be followed by the author:

​

• It is expected to start by describing in simple terms what the data show

• They should make proper reference to statistical analyses, such as significance or goodness of fit

• If once described, they should always evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research.

• It is expected that the outcome is a critical analysis of the data collected.

​

It is advised to observe that discussion should, at some point, gather all the information together into a single whole. Authors are suggested to describe and discuss the overall story formed. If gaps or inconsistencies in the story are found, they should address these and suggest ways future research might confirm the findings or take the research forward.

​

4. Conclusions

​

This section may be presented as part of the results and discussion, or in a separate section, generally this section is no more than a few paragraphs. The conclusions should always reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If it is observed that the conclusions are not evidence-based, then it's appropriate to ask for them to be re-written.

​

5. Information Gathered: Images, Graphs and Data Tables

​

If you are able to find yourself looking at a piece of information from which you cannot discern a story, then you should ask for improvements in presentation. It is important to note that it can be an issue with titles, labels, statistical notation or image quality.

​

If the information is clear, you should check that:

​

• The results are plausible, in case there is an error in data gathering

• The trends you can check support the paper's discussion and conclusions

• The data seem sufficient. For instance, in studies carried out over time are there sufficient data points to support the trends that are described by the author?

You are advised to check whether images have been edited or manipulated to emphasize the story they tell. Although, it may be appropriate to some extent but only if authors report on how the image has been edited (e.g. by highlighting certain parts of an image). If you feel that an image has been edited or manipulated without explanation, you are advised to highlight this in a confidential comment to the editor in your report.

​

6. List of References

​

It is required to check referencing for accuracy, adequacy and balance.
 

Accuracy

​

If a cited article is central to the author's argument, in that case you should check the accuracy and format of the reference - and bear in mind various subject areas may use citations differently. Otherwise, the editor's role is to exhaustively check the reference section for accuracy and format.

​

Adequacy

​

It is important to note that it should come into your consideration if the referencing is adequate:

​

• Are the significant parts of the argument poorly supported?

• Are there published studies that reflect similar or dissimilar trends that should be discussed?

• If a manuscript specifically uses half the citations typical in its field, then this may be an indicator that referencing should be improved - but don't be guided solely by quantity.

• Check the references they are relevant, recent and readily retrievable

​

Balance

​

Always check a well-balanced list of references that is:

​

• Helpful for the reader

• Fair to the competing authors

• Not only over-reliant on self-citation

• Provides due recognition to the initial discoveries and related work that led to the work under assessment

Another important aspect is that you should evaluate whether the article meets the criteria for balanced referencing without looking up every reference.

7. Plagiarism

You will have a deep and thorough understanding of the paper's content by now - and you may have some concerns about plagiarism.

Identified Concern

​

If by any chance you find - or already knew of - a very similar paper, this may be because the author overlooked it in their own literature search. Or even it may be possible because it is very recent or published in a journal slightly outside their usual field.

​

Sometimes you may feel that you can advise the author how to emphasize the novel aspects of their own study, to better differentiate it from similar research. If so, you may even ask the author to discuss their aims and results, or modify their conclusions, in light of the similar article. The research similarities of course may be so great that they render the work unoriginal and you have no choice but to recommend rejection.

​

"It's really seems helpful particularly when a reviewer can point out certain recent similar publications on the same topic by other groups, or that the authors have already published some data elsewhere." (Editor feedback)

​

Suspected Concern

In case you suspect plagiarism, including self-plagiarism, but unable to recall or locate exactly what is being plagiarized, notify the editor of your suspicion and ask for guidance.

​

Editors mostly have access to software that can check for plagiarism.

Editors are not out to police every paper, but when the plagiarism is discovered specially during peer review it can be properly addressed ahead of publication. If by any chance plagiarism is discovered only after publication, the consequences actually are worse for both authors and readers, because a retraction may be necessary.

​

8. Search Engine Optimization (SEO)

 You will be in a position to advice after the detailed read-through whether the title, abstract and key words are optimized for search purposes. As far as effectiveness is concerned, good SEO terms will reflect the aims of the research.

​

Usually, clear title along with the abstract will definitely improve the paper's search engine rankings and will influence whether the user finds and then decides to navigate to the main article. It is important to note that title should always contain the relevant SEO terms early on. This really has a major effect on the impact of a paper, since it really helps the paper to appear in search results. Reader's interest can be lost due to the poor abstract and undo the benefit of an effective title - whilst the paper's abstract may appear in search results, the potential reader may not go further.

​

You are advised to ask these questions to yourself, while the abstract may have seemed adequate during earlier checks, does it:

​

• Do complete justice to the manuscript in this context?

• Highlight the important findings sufficiently?

• Present the most interesting data?

Editors say, "Does the abstract sincerely highlight the important findings of the study?"

​

How to Structure Your Report

If a formal report format is there, always remember to follow it. Further a range of questions are followed by the comment sections. You should try to answer all the questions. It is significant to note that the questions are there because the editor felt that they are important. In case you are following informal report format then you could structure your report in three sections: summary, major issues, minor issues.

​

Summary

​

• Give positive feedback first. Generally, authors are more likely to read your review if you do so. It is not suggested to overdo it if you will be recommending rejection.

• Summarize briefly about the paper and its findings.

• The findings of the paper are advised to keep into the context of the existing literature and current knowledge.

• It is necessary to indicate the importance of the work and if it is novel or mainly confirmatory.

• Properly indicate the work's strengths, its quality and completeness.

• State the major flaws or weaknesses and also note any special considerations. For instance, if you find that previously held theories are being overlooked.

​

Major Issues

Are there any major flaws? Explain them and the severity of their impact on the paper.

• Has similar work been published already without the authors acknowledging this?

• Are the authors properly presenting the findings that challenge current thinking? Is the evidence provided strong enough to prove their case? Has all the relevant work been cited by them that would contradict their thinking and addressed it appropriately?

• If revisions are required, try to indicate clearly what they are

• Are there any major presentational problems? Do you find figures & tables, language and manuscript structure all are clear enough for you to accurately assess the work?

• Are there any ethical issues? If you are not sure then it may be better to disclose these in the confidential comments section.

​

Minor Issues

Are there places where meaning is ambiguous? How can this be corrected?

• Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?

• If you observe any factual, numerical or unit errors? If so, what are they?

• Check are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not.

On Presentation and Style

It seems important to note that your review should ultimately help the author improve their article. So be polite, honest and clear. You are advised to try to be objective and constructive, not subjective and destructive.

You should also:

​

• You should clearly write so that you can be understood by people whose first language is not English.

• You should avoid complex or unusual words, especially ones that would even confuse native speakers.

• You should number your points and refer to page and line numbers in the manuscript when making specific comments

• You should indicate clearly the parts, if you have been asked to comment on specific parts or aspects of the manuscript.

• You should treat the author's work the way you would like your own to be treated

​

Criticisms & Confidential Comments to Editors

Usually, journals provide reviewers the option to provide some confidential comments to editors. This is where generally editors will want reviewers to state their recommendation - see the next section - but otherwise this area is best reserved for communicating malpractice like; suspected plagiarism, fraud, unattributed work, unethical procedures, duplicate publication, bias or other conflicts of interest.

This doesn't provide reviewers permission to 'backstab' the author. Even authors are unable to see the feedback and are unable to give their side of the story unless the editor asks them to. You should write in the spirit of fairness, write comments to editors as though authors might read them too.

​

The Recommendation

Reviewers are advised to check the preferences of individual journals as to where they want review decisions to be stated. You have to particularly bear in mind that some journals will not want the recommendation included in any comments to authors, as this can cause editors difficulty later - see Section 11 for more advice about working with editors.

​

Generally, you will be asked to indicate your recommendation (e.g., accept, reject, revise and resubmit, etc.) specifically from a fixed-choice list and then you may be asked to enter your comments into a separate text box.

​

Recommending Acceptance

​

If acceptance is recommended on your part, give details outlining why and any areas that could be improved. Don't just provide a short, cursory remark such as 'great, accept'. See Improving the Manuscript.

​

Recommending Revision

You should find the places where improvements are required, a recommendation for major or minor revision is typical. It is also possible that you choose to state whether you opt in or out of the post-revision review too. If you are recommending revision, state specific changes you feel need to be made. Then it becomes easy for the author to reply to each point in turn.

​

Sometimes the option to recommend rejection is offered by some journals with the possibility of resubmission – however, this is most relevant where substantial, major revision is necessary.

​

What can reviewers do to help? "Be clear in their comments to the author (or editor) which points are absolutely critical if the paper is given an opportunity for revision." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

​

Recommending Rejection

It is advised to state this clearly in your review, if recommending rejection or major revision. (and see the next section, 'When recommending rejection').

​

When Recommending Rejection

If you find serious flaws in the manuscripts you should not spend any time polishing the review you've drafted or give detailed advice on presentation.

​

Editors say, "If rejection is suggested by the reviewer, but her/his comments are not detailed or helpful, in that case it does not help the editor in making a decision."

In your recommendations for the author, you should:

​

• Always give constructive feedback describing in the ways that they could improve the research.

• Be focused on the research and not the author. This is really an extremely significant part of your job as a reviewer.

• Avoid giving critical confidential comments to the editor while being polite and encouraging to the author - the latter may not be able to understand why their manuscript has been rejected. Also, they won't get proper feedback on how to improve their research and it could trigger an appeal.

​

Remember to provide constructive criticism even if recommending rejection. This assists researchers to improve their work and even explains to the editor why you felt the manuscript should not be published.
 

"When the comments seem really positive, but the recommendation is rejection…it puts the editor in a tough position of having to reject a paper when the comments make it sound like a great paper." 

​