Peer review is so frequently discussed and so often maligned that one might wonder if there is anything new to be added on the topic. The new part for me is that over the last 12 months, I received 36 peer reviews, and 34 of these were high quality reviews: Not merely positive, but substantively useful.

Maybe I had lucky 12 months, but 34/36 seems a statistic unlikely to reflect just luck. I think it reflects experience from a domain, in which peer review works well. This domain is the focus of this post. If you cannot tolerate hearing about functional peer review, you may directly go to the section on dysfunctional peer review, from which – sadly – I have experience as well.

Functional peer review 

Peer review is almost never perfect, but it does have a large domain within which it adds values. We should not hold peer review to perfectionist standards to appreciate it. For example, our recent article Leduc et al. was reviewed at Nature Biotechnology by 4 reviewers who made constructive suggestions that helped us improve the description and the demonstration of nPOP. One of them insisted on a factually incorrect assertion (that many groups have published lots of papers reporting mass spec proteomics data from thousands of single cells), and this precluded publication in Nature Biotech. We can see this as the glass being half empty and focus on an unsubstantiated claim by a reviewer. However, I prefer to focus on the glass being half full: The review process at Nature Biotech helped us improve our initial article and publish the improved version in a strong journal, where it will be influential because of its intrinsic qualities and the improvements made in response to constructive reviewers, including the one who made the incorrect assertion.

The above example illustrates a useful peer review process with a clear problem. Yet, most of our recent experiences have been less affected by problems. For example, the peer review process for prioritized mass spectrometry (pSCoPE), a record of which is available from the website of Nature Methods, illustrates a very thoughtful and constructive process. I cannot read it and agree with claims that peer review “is broken” and does not work. Such claims are at best oversimplifying the problems and selectively excluding evidence.    

Dysfunctional peer review 

Peer review tends to perform the least well with exceptional papers whose ideas are a few steps ahead of the thinking of the reviewers. Most papers receiving very critical reviews are likely deserving of the criticism. When criticism of innovative papers is unfair, it may be mitigated by 2 factors:

• Exceptional papers often can succeed in advancing the thinking of the community even without being published in the most selective journals. Some of these papers are very influential even as preprints.
• The authors of strong innovative papers can ease their acceptance by (1) facilitating reproduction and critical evaluation of their data and analysis and (2) communicating their findings with clarity and logic that lower the energy barrier to understanding. Both 1 and 2 require effort, skill, dedication and intelligence, and these factors are generally required for shaping the intellectual frameworks of a scientific field. 

Let’s be specific about what works and what does not and seek to improve what is within our circle of influence!