I’ve written before about writing a review and there’s even guidelines on the DTRAP website. I wanted to revisit the topic because I think it’s useful.
Writing a good review is a skill, you’re not going to get it right the first time but there’s a few things I want to talk about that can help you. This time with examples.
A review should be useful to the person who wrote the paper. It should tell them what they did right, what they missed, and how they can improve. No paper is completely perfect but the author won’t know how to make it better if we don’t tell them.
First, I know I’ve said this many times before, but be respectful in your review. Saying:
This paper sucks.
Is neither helpful nor polite. It’s a very bad review, so please don’t do that. No one who reads that review is going to take anything away from it about how to improve their work, just that the author of the review couldn’t be bothered to do it right. And is disrespectful to the author. A disrespectful review can put people off fixing their work and even doing research. Don’t be that person.
Suppose someone submits a paper on researching vulnerabilities and the people that find them. You’re a vulnerability researcher (I’m making all of this up) and we ask you to review it. You read it and find the paper very interesting and well written but it’s completely missing the impact of Vulnerability DIsclosure Programs on researchers and vulnerabilities. (Did I mention I’m making all of this up? Because I am.)
If your review only focuses on the positives of the paper, ‘well written!’, ‘very interesting!’, ‘useful to researchers in vulnerabilities!’ then you’re missing out on part of your job as a reviewer. You should let the authors know that they missed this information. It’s important to the field and omitting that information makes the paper that much weaker. You should write something like:
The authors have written a very good paper, however, I am concerned by the omission of Vulnerability Disclosure Programs (VDPs). They have been shown (maybe add some citations!) to affect vulnerability researchers and the vulnerabilities found. The authors should revise and include this information and how it affects their research or they should explicitly detail why it doesn’t affect their results.
See? Useful. Tells the authors what to fix and how to fix it. And written respectfully. Writing something like:
I can’t believe the authors omitted Vulnerability DIsclosure Programs. Any researcher worth their salt knows they’re important. This paper is completely useless without it.
Is a very bad review. Don’t do that.
I’m going to revisit this topic often because it’s important to a functioning journal. The reviewers keep us going once we get the submissions and good reviews are a skill.