Deacon: why removing host reads is harder than it looks
Lee Katz, Andrew Page and Nabil-Fareed Alikhan talk with Bede Constantinides, a research fellow at the University of Birmingham, about removing host DNA from metagenomic sequencing data. They discuss the tension between retaining microbial reads and removing human sequences, alongside the memory and cost implications of using human pangenomes. This first part establishes the motivation for Deacon and its predecessor, Hostile; the next episode will examine Deacon’s method in more detail.
Listen to Episode 151: Deacon part 1 on SoundCloud
In this episode
The practical problem Deacon addresses
Bede introduces Deacon as a tool for removing human DNA sequences from FASTA or FASTQ input, including compressed files and streams. More generally, it searches for sequences matching a gene, genome or pangenome and lets users retain or discard matches. Its potential uses extend beyond humans to hosts such as mice and plants. One motivation is microbial sequencing where users lack permission to store or process human DNA.
Andrew describes a practical example: rectal mucus samples containing roughly half human material and half gut microbiome. His team needs to analyse both components. Their workflow uses nf-core taxprofiler for initial mapping-based removal, followed by Kraken2, yet human sequences remain detectable even among unclassified reads. Simply filtering harder risks losing microbial signal.
Sensitivity depends on what must be preserved
Bede’s Kraken2 benchmarks show how strongly host-removal performance depends on the database and its parameters. Large databases with dense minimiser sampling can classify human reads very sensitively, but can also remove bacterial and viral reads. Shared sequences introduce another complication: reads matching multiple genomes may receive a taxonomic assignment that does not identify them as human. A human-only database addresses that ambiguity, but not excessive classification of microbial reads as human.
The acceptable trade-off depends on the biological question. Bede uses Mycobacterium tuberculosis as an example with little sequence overlap with humans, allowing aggressive filtering at relatively little cost. Broad metagenomics is harder: viruses may share sequence with endogenous retroviral elements in the human genome. A filter suitable for one bacterium may therefore be destructive when looking for an unknown range of pathogens.
Larger references bring costs and unexpected behaviour
Mapping against a single human reference can leave genuine human reads unmapped because genomes differ. Conversely, error-tolerant mapping can align bacterial or viral reads to the host reference. Bede argues that broader reference diversity is desirable so that human reads from different ethnicities are removed equally well, but conventional linear-reference mappers do not scale easily to large, redundant pangenomes.
Recalling an earlier Human Pangenome Reference Consortium release, he describes roughly 50 diploid genomes: about 100 haploid assemblies at three gigabases each. Andrew puts this scale into operational terms. His team sequences around 60 gigabases per sample, and larger-memory AWS machines make expanded references expensive.
More reference sequence does not automatically improve removal. When Bede tested an index containing 20 human genomes, he initially mapped fewer reads. He traced this to a minimap2 default that ignored what he recalls as the most abundant 1% of minimisers. Collapsing redundant reference sequence might help, but would require additional design choices and would not remove the need to contend with mapper heuristics.
Why Hostile was built for local processing
Hostile grew out of Bede’s work on a cloud platform for microbial genomics. The aim was to decontaminate private data locally and offline, avoiding sending it over the internet. He describes legal as well as privacy concerns about sending human sequence data over the internet. Existing approaches could be difficult to install, require more memory than a laptop offered, or remove microbial reads that users wanted to retain.
Bede also cites work led by Stephen Bush and supervised by Sarah Walker, which found human contamination carrying sensitive information, including disease predisposition, in bacterial datasets. That work identified benefits from using two complementary aligners, but running both was not feasible for Hostile’s intended use case. Hostile instead uses minimap2 for long reads and Bowtie2 for short reads, chosen for their sensitivity–specificity balance on the expected samples.
A prototype is not a reliable wrapper
Andrew asks how quickly Hostile could be rebuilt from scratch today, given the rise of vibe coding. Bede thinks a basic prototype could be produced very quickly, perhaps in a single attempt, but distinguishes that from software incorporating the practical lessons learned along the way.
The difficult work includes subtle edge cases when general-purpose tools are used outside their original purpose: handling standard input and output, single and paired reads, multiple aligners, and retrieving indexes in different formats from cloud storage. GitHub issues and accumulated experience reveal requirements that a working demonstration may miss. Bede is nevertheless cautious about predicting limits as coding agents improve.
From alignments to a simpler search problem
Bede describes Hostile’s SAM-stream workflow as convenient but wasteful: it spends effort aligning human reads that will immediately be discarded. Using minimap2’s C API, as Martin Hunt at the EBI did for a host-depletion tool, might reduce some overhead, but still leaves the constraints of an alignment-based approach.
Deacon’s motivation is to ask how well a faster, simpler approach based on storing minimisers can perform. Bede explains that k-mers and deterministic subsets such as minimisers can represent pangenome diversity compactly by retaining differences rather than repeatedly storing shared sequence. The episode ends at this design shift, leaving the detailed method for part two.
Highlights
- 00:02:12 — What Deacon searches, which input formats it accepts, and who might use it
- 00:03:16 — Andrew’s rectal mucus samples and the difficulty of separating human and microbial reads
- 00:05:01 — How Kraken2 database size and minimiser sampling affect host removal
- 00:06:48 — Human-only Kraken2 databases and why specificity depends on the pathogens being sought
- 00:09:55 — Why mapping misses human reads, removes microbial reads and struggles with pangenomes
- 00:11:42 — The scale of early human pangenome reference assemblies
- 00:12:47 — Processing 60 gigabases per sample and the cost of larger AWS machines
- 00:13:58 — Compact references and the unexpected effects of minimap2 heuristics
- 00:15:42 — Hostile’s origins in local, offline decontamination to avoid sending private data over the internet
- 00:17:47 — Fast AI-assisted prototypes versus the edge cases needed for reliable software
- 00:20:38 — Why alignment wastes work and why Deacon instead stores minimisers
In their own words
you’re spending all this effort aligning human reads that you’re about to throw away
— Bede Constantinides, 00:20:38
Who is talking
- Lee Katz (host)
- Andrew Page (host)
- Nabil-Fareed Alikhan (host)
- Bede Constantinides (guest, University of Birmingham)
Also mentioned: Stephen Bush, Sarah Walker, Martin Hunt.
Tools and resources mentioned
Deacon, Hostile, nf-core taxprofiler, Nextflow, Kraken2, Bowtie2, minimap2.
Questions this episode answers
What is Deacon used for?
Deacon removes human DNA sequences from FASTA or FASTQ data, including compressed files and streams. It can also search for other genes, genomes or pangenomes and either retain or discard matching sequences, including sequences from non-human hosts.
Why can host read removal discard microbial sequences?
Error-tolerant alignment can map bacterial or viral reads to a human reference, while sensitive Kraken2 configurations can also classify microbial reads as human. Bede highlights viruses sharing sequence with endogenous retroviral elements as a reason broad metagenomic filtering requires particular care.
Why not just add more human genomes to the mapping reference?
Larger references increase memory requirements and computing costs, and mapper heuristics can produce unexpected results. Bede describes testing 20 human genomes and initially mapping fewer reads because of minimap2’s treatment of highly abundant minimisers.
How does Deacon differ from Hostile?
Hostile uses Bowtie2 for short reads and minimap2 for long reads, filtering their alignment output. Deacon takes a simpler minimiser-based search approach intended to avoid unnecessary alignment work and represent pangenome diversity more compactly; its detailed method is reserved for the next episode.
This page was written from a machine transcript of the episode and checked against it. Listen to the episode for the whole conversation.
Show notes
In this Software Deep Dive episode, we talk with Dr. Bede Constantinides (University of Birmingham) about Deacon, a tool for removing host DNA reads from metagenomic datasets. We discuss why host read removal is a deceptively difficult problem, the limitations of alignment- based approaches, and how Deacon evolved from Bede’s earlier tool Hostile.
The conversation covers practical issues such as human contamination in metagenomes, balancing sensitivity vs specificity when filtering reads, and the computational challenges of working with large human reference datasets and pangenomes.
This episode focuses on the background and motivation for Deacon. The next episode will dive deeper into how the method works.
Links
Deacon https://github.com/bede/deacon
Hostile https://github.com/bede/hostile/
Bede Constantinides http://bede.im/
nf-core taxprofiler https://nf-co.re/taxprofiler