A few weeks ago, I asked people to share the meanest referee comment they’ve received. The response was overwhelming; the question clearly hit a nerve. I will now re-share some of these comments and discuss how to avoid being that referee. I’m publishing the comments (with permission) partly to shed light on the problem, partly to make it clear that if you’ve received a similar one, you are not alone, and partly because reading others’ mean comments might help you discern whether you yourself have ever been mean. If after reading this post you realize you have been mean in the past, forgive yourself but make a serious effort to not do it again (and maybe find the authors at a conference, buy them a drink, and say something nice about their work).

So, what is a “mean” referee comment? It’s not just a critical comment—pointing out flaws is indeed part of a referee’s job description—it’s a comment that is meant to insult the paper or the author(s) who wrote it. At best, a mean comment is unprofessional. At worst, well, you can decide for yourself based on actual referee comments below.

I spent some time thinking about what made the submitted comments mean, especially ones that didn’t explicitly involve rude or unprofessional language but still felt inappropriate. Several types of mean comments to avoid emerged (with some falling into multiple categories), yielding a guide for how to write up your critical report without being mean. I’ve ordered the suggestions below roughly from easiest-to-implement to hardest-to-implement.

Suggestion #1: Review the paper, not the person who wrote it

What not to write:

• “I am friends with both of the authors and have great admiration for them and their work. I am surprised they would send such inchoate work to [journal].”
• “Go flip a hamburger.”
• “This is not a top researcher.”
• “This author has no soul”
• “lemma 2 was written by a child”
• “As a ‘researcher’ you should know better.”
• “And I thought these guys were good economists!”
• “The lead researcher on this project should be embarrassed for having submitted this paper”
• “I can only assume that the authors are illiterate”
• “The authors need to get a book and learn how to interpret interactions.”
• “It is 2021!! I don’t see why another paper by two male economists on the labor supply of men is of general interest to the profession.”
• “I believe the author is Jewish, and I do not find it appropriate that she speaks on this topic. The work of Jewish scholars on issues of banking does not pass the smell test for audiences. I would recommend seeking out a co-author to reduce any issues of manipulation in the framing of the paper.”

These are personal attacks and have no place in a referee report. The identity of the author should not affect whether or not you raise a particular issue in a report. Similarly, it’s not appropriate to comment on the authors’ overall abilities based on the quality of the paper; stick to reviewing the paper. The only exception to “do not review the author(s)” is grants, where the recommendation of whether to fund a grant can legitimately hinge on how experienced the researcher is and how much relevant work they have done. But even then, follow the other suggestions to avoid being mean.

Suggestion #2: Don’t try to turn your criticisms into works of literature

What not to write:

• “… this paper read like a settling of scores with the literature by a disgruntled author that thinks his past contributions have not been fully recognized. … There is no scientific value to any of this beyond convincing the reader that the author was talking about these issues a long time ago.”
• “you are kicking at an open door while trying to knock over a strawman”
• “This would be shoddy journalism even by a cub reporter in a small-town newspaper.”
• “Now my impression of the paper is a sequence of unfocused paragraphs, mostly vague intuitions, and grandiose statements that could raise eyebrows.”
• The paper is “old wine in new wine skirts”
• “This paper is like spaghetti thrown in the wall: let’s see what sticks”.
• “this is a strange paper to my eyes, a good candidate for an academic parlour game: give the reader just the footnotes and make her guess what the body of the text might say“

These may be fun to read when they’re directed at someone else’s paper, and make the reviewer who came up with the quip feel clever, but statements like these are clearly meant to be insulting rather than objective evaluations of the work. They also make it appear as though the reviewer put more effort into coming up with a way to insult the paper than into evaluating the content.

Suggestion #3: keep your complaints about editorial decisions out of referee reports

What not to write:

• “The editors should have never sent this out to reviewers.”
• “This paper has evidently been sent to Metroeconomica on the basis that it uses the word ‘uncertainty’, and ‘uncertainty’ appears as an important concept in contemporary (post-, not ‘new-) Keynesian analysis. This is a category error, and a waste of scarce referee time for those who populate the referee universe for serious analytical models in the heterodox economic traditions.”
• “Overall I can’t see how this paper reached the stage of getting peer reviews.”
• “I will be upfront and admit that I see my role as a referee these days more as a gatekeeper and less as a taskmaster as papers now tend to be high jacked by referees and publication times consequently lag. This paper is not suitable in terms of content, scope, and style for [Journal Name]”

It’s the job of the editor to decide which papers get desk-rejected and which papers get reviewed. If you’re unhappy that a paper was not desk-rejected, take it up with the editor (in a more polite fashion).

Suggestion #4: Don’t accuse the authors of acting in bad faith without proof

What not to write:

• “Unsurprisingly, because it is so self-serving, the authors’ account of the history of economic thought is also incorrect in many places.”
• “a very big part of the paper is devoted to re-writing history to give the author a prominent role in it.”
• “In the future, the authors should only cite papers that they have read and clearly understood.”
• “This is a dreadful paper – it is poorly written and I am left, having read it, wondering if anyone discussed it and proof read it before it was submitted.”
• “The results of the authors are clearly at odds with the existing evidence, which speaks against their integrity and quality of this research.”
• “The authors report that they analyzed 1500 datasets, which is obviously not possible, thus the whole paper is a fabrication and I will not comment on its substance.”

There are authors who act in bad faith. But the correct way to handle situations where you suspect dishonesty is to bring up the patterns that make you suspect dishonesty without accusing the authors of anything. Unless the issue is plagiarism, which is straightforward for a reviewer to verify, it is essentially impossible to know whether something fishy in a paper is due to deliberate dishonesty or carelessness. So give authors the benefit of the doubt and just focus on the issue at hand, not on why it arose. Additionally, in two of these cases, the authors noted that the reviewer missed something major about the paper, leading them to make an incorrect claim. So if you think something is very wrong, double-check that it’s not you who’s missing something.

Suggestion #5: be specific

What not to write:

• “I don’t believe in economies of scale in agriculture.” [That was the whole review.]
• “I believe your research is completely irrelevant and if it were relevant, you are doing it completely wrong.”
• “We hope to publish original papers. Yours is child’s play.”
• “The first sentence of the first paragraph was painful to read.”
• “This is not economics”
• “Your paper is written by economists for economists” – comment from a top management journal.
• “I would fight to prevent this paper from being published”
• “[that] coefficient doesn’t pass the smell test.”
• “This paper is beyond rescue.”
• “Nobody wants to read this boring stuff”
• “The conclusion (called discussion) is awful”
• “It’s ridiculous that you call this an experiment and wrote a whole paper based on this.”
• “the exposition of the rest […] is filled with vague and disparaging claims that are extremely difficult to distinguish from total nonsense.”

I think reviewers use generic statements like these to try to convey to the reader just how many problems the paper has. The problem is, statements like these are unlikely to help anyone. The authors will either write the reviewer off as a jerk or they will be dismayed but have no guidance as to what the actual problems are. If the paper is deeply flawed, you do not need to list every specific issue that caused you to make that conclusion. But do replace any generic statement condemning the whole paper (or sections of a paper or an entire analysis) with concrete statements of the key problem(s). And don’t cheat by making a generic statement about how terrible the paper and then follow up with specifics. Something like “Here are the main issues with the paper as I see them:” is a good enough prelude.

Suggestion #6: don’t use compliments to give yourself permission to be mean

• “I was excited by the topic and so left the manuscript to my Sunday reading. You ruined my Sunday.”
• “Based on the title, your article seems promising. Unfortunately, I’ve read it.“
• “The paper is very well written. Unfortunately, this is the only plus of the paper I could find, as its scientific content is next to null.”

“Find something nice to say about the paper no matter how bad you think it is” is a frequent suggestion given to reviewers. But just because a reviewer said something nice about a paper does not give him or her the right to let loose with mean comments.

Suggestion #7: apply the “does it need to be said?” standard

What not to write:

• “Here I stopped reading. Too much is going wrong.”
• “It sounds like an undergraduate project – not a paper that is intended for submission to a journal.”
• “Although the pre-colon part of the title appeared flippant, it nevertheless perked my interest”
• “Such an analysis would receive a C+ at best in my undergraduate econometrics course.”
• “At its present form, this work cannot even be used as a term paper in an Agricultural Economics course”
• “In its current phrasing, the review resembles a graduate seminar”
• “This analysis wouldn’t pass a candidacy exam at my institution“
• “The whole paper reads like a B.Sc. thesis, but not a good one.”
• “I do not like the title. I thought I had to review a baseball paper or something like that. I understand that the title is paraphrasing a published paper “XXX”. That title I like but the title of the current paper is awful.”
• “If this paper is published in a good journal, then I will not consider that journal a good journal anymore”

Even if objectively true, none of these statements serve a purpose except to signal to the author(s) just how much you disliked the paper.

I should note that I have no idea which papers these comments correspond to, but I’m comfortable claiming that no paper, no matter how poorly conceived and executed, deserves comments like these.

An overarching suggestion is to read your penultimate report draft to see what negative feedback can be deleted without affecting the substance of your report. Personal insults are not substance. Wholesale condemnation is not substance. Disparaging comparisons are not substance.

You might say that not being mean sounds hard. And it is true that it’s easier to write a generic statement like “this paper is not interesting” or “this is stupid” than to dig deeper, ask yourself why you don’t think the paper is interesting, and then explain the reasons using professional language. But doing the latter will not only make your report more constructive but is also good exercise for the brain!

As a parting thought, remember that the paper you are reviewing may be the first (or second or third) ever paper submission of the author. The kind of reports they receive may well shape their beliefs about the publishing process and academia. So make your report a kind one that makes them want to try harder rather than the kind that makes them want to quit academia.