Reviewing bioinformatics papers in 10 days or less
Lee Katz, Andrew Page and Nabil-Fareed Alikhan discuss how to make peer review faster without overlooking fundamental problems. Lee says reviewing a manuscript can take him up to 10 days; Andrew and Nabil explain how they screen invitations, assess methods and results, and decide whether a detailed review is worthwhile. They also discuss code availability, accessible data and ways authors can make manuscripts easier to review.
Listen to Episode 75: How to review a bioinformatics paper in 10 days or less on SoundCloud
In this episode
Screen requests before committing
Andrew starts with the abstract and a quick scan. If a manuscript is fundamentally unsound, he asks the editor to consider a desk rejection rather than spending reviewers’ time on a full report. Where a review is needed, being upfront about a few fatal problems can be enough, with no need to go through every nitty-gritty detail.
He also screens invitations by available time, publisher and who the authors are. He says he declines unpaid Elsevier reviews, offering instead to review at the full economic cost of his time—around $1,000 in his example. Difficult login systems are another deterrent. His practical rule is not to accept a review and then leave it unfinished.
Review against the journal’s criteria
The hosts distinguish technical soundness from novelty. Andrew uses PLOS ONE as an example where solid science, including negative results, need not offer novelty; other journals explicitly require an assessment of originality and relevance to the field.
Nabil advises declaring limits to your expertise rather than struggling through an assessment you cannot support. Laboratory experiments or an unfamiliar organism such as E. coli might need another reviewer’s judgement. He also warns against rejecting work simply because you would have used a different tool or experiment. Andrew describes going back to editors to challenge other reviewers who lacked expert knowledge, asked for silly things or would impose unnecessary extra work.
Separate the first read from detailed checking
Lee normally reads a paper straight through, then returns for a slower second pass that includes checking software and accession numbers. Nabil instead starts with the introduction’s final paragraph to establish the aim, reads the figures, skims the results and consults the methods where necessary. If the terminology is unfamiliar, he reads more of the introduction.
Nabil describes spending roughly half an hour to an hour on this first pass, taking a break, then returning for the detailed review. Andrew also makes a first pass before deciding whether to proceed to an in-depth review. The shared priority is to establish whether methods and results hold together before assessing interpretation, structure and presentation.
Look for methodological failures
Andrew treats the methods as a place where vague or impressive-sounding prose cannot conceal unsuitable analysis. His examples include applying a long-read assembler to short-read data, using Nanopore software for PGM data, or ignoring sequencing error rates. He describes a 16S paper lacking controls and replicates, and challenges its species-level identification claims.
Nabil gives a subtler example: assembly, alignment and figures appeared reasonable, but claims linking hosts, genes and operons lacked an explained analytical basis. Gene presence and association needed definitions, and the word significant needed statistical support. Conversely, he would not dwell on a routine assembly step using Shovill if it did not affect the result under review.
Inspect code, accessions and earlier versions
Andrew says he reviews supplied code, recalling a roughly 300-line Bash pipeline dominated by print statements. Nabil describes a purported new algorithm that instead ran another tool with customised parameters and altered its output. For software papers, he regards missing code as grounds for rejection.
Data checks include whether accession numbers are real and accessible, and whether supplementary material supports the manuscript. Andrew also sometimes checks bioRxiv or medRxiv for earlier versions. Comparing a preprint with the submission can raise questions about whether authors have revised the work or incorporated earlier feedback, although these checks can become time-consuming rabbit holes.
Make feedback and manuscripts easier to use
Andrew separates major, potentially blocking comments from optional minor comments and typos. Feedback can also identify strengths: an important result buried deep in the paper may deserve prominence in the abstract. Lee recalls struggling with a three-figure limit for a Haiti cholera paper.
Nabil reserves presentation checks for a later pass: figure legends, table references, accession codes and supplementary files. He asks for strain tables as Excel sheets rather than PDFs converted from Word, so accession numbers are easier to extract. He also wants figures sized consistently with A4 manuscript pages, not enormous pages requiring constant zoom changes. Page numbers help too. Clear scientific writing should state the finding rather than build suspense.
Highlights
- 00:00:00 — Lee introduces peer review and his struggle with reviews taking up to 10 days.
- 00:01:28 — Andrew explains initial screening and when to ask an editor for a desk rejection.
- 00:07:18 — Nabil screens out unsuitable invitations, then prioritises results and methods.
- 00:08:42 — Andrew separates major comments, minor comments and typos before deeper checks.
- 00:12:06 — Match the review to the journal’s scope and declare gaps in your expertise.
- 00:14:16 — Nabil distinguishes objective flaws from a reviewer’s preferred way of doing the work.
- 00:17:38 — Andrew discusses inspecting code, including a 300-line Bash pipeline.
- 00:18:50 — Nabil explains his figures-first reading strategy, starting from the introduction’s last paragraph.
- 00:23:25 — Andrew explains why methods reveal unsuitable software and sequencing assumptions.
- 00:24:44 — A rejection report can identify major technical failures, illustrated with a 16S paper.
- 00:25:31 — Custom methods add review work, while association claims need an analytical basis.
- 00:27:43 — Nabil orders review stages: methods and results, interpretation, then presentation checks.
In their own words
I might make comments about the manuscript, but I will always reject a paper without code for software.
— Nabil-Fareed Alikhan, 00:17:48
It’s like a film you have to watch twice because you don’t catch everything the first time around.
— Nabil-Fareed Alikhan, 00:18:50
Who is talking
- Lee Katz (host)
- Andrew Page (host)
- Nabil-Fareed Alikhan (host)
Tools and resources mentioned
Shovill, Bash, bioRxiv, medRxiv, PGM, Inkscape, Illustrator, PowerPoint, Excel.
Questions this episode answers
What should I read first when reviewing a bioinformatics paper?
Nabil starts with the introduction’s final paragraph, then figures, results and relevant methods. Lee describes reading straight through first, and Andrew makes a quick first pass before deciding how deep to go. All three discuss establishing technical soundness before spending time on detailed presentation feedback.
Do I need to review every detail of a fundamentally flawed manuscript?
The hosts say a review can focus on clearly explained major problems when those invalidate the work. Andrew also suggests asking the editor to consider a desk rejection when an initial scan reveals glaring, irredeemable problems.
What code and data checks do the hosts recommend?
Inspect supplied code, test software where appropriate, and check that accession numbers are real and accessible. Nabil says he rejects software papers that do not provide code.
How can authors make a manuscript easier to review?
State the main finding clearly, explain the analytical basis of claims and make supporting data easy to access. The hosts also recommend usable spreadsheet tables, consistently sized figures, informative legends and page numbers.
This page was written from a machine transcript of the episode and checked against it. Listen to the episode for the whole conversation.
Show notes
We discuss how to effectively review bioinformatics papers, what to look out for and tips for researchers to when writing bioinformatics or microbial genomics papers to ease their way through review.