Sourmash, reference quality and the limits of microbial taxonomy
Titus Brown joins hosts Lee Katz and Andrew Page to discuss reference databases, sourmash and the difficulty of assigning microbial species accurately. They contrast flexible research tools with the confidence required for public health decisions. Brown outlines a conditional argument for near-perfect species classification, while the hosts explain why biological definitions and regulatory requirements complicate that goal.
In this episode
Searching references is not the same as curating them
Andrew asks how sourmash handles poor-quality reference genomes. Brown distinguishes database searching from database curation: sourmash should search whatever references users want, rather than limit them to a selection dictated by software capacity. He puts GenBank’s bacterial genome collection at around 1.3 million genomes at the time of the conversation.
Brown’s group takes GTDB releases as supplied, valuing their transparent quality metrics. Andrew asks about filtering using CheckM results, but Brown emphasises GTDB’s balance between quality and representation. Much of the knowledge of about a dozen phyla comes from metagenome-assembled genomes (MAGs), including imperfect genomes worth retaining. Brown also describes Charcoal, a side effort using sourmash to identify suspiciously similar contigs across widely separated taxonomic groups in GenBank and GTDB.
Filtering for research and public health
Andrew would remove contamination early: an E. coli assembly with 3,000 contigs is his example of a questionable reference. Lee similarly prioritises removing potentially noisy SNPs to obtain a high-fidelity phylogeny, even at the cost of losing some genuine signal. For public health, the hosts want confidence that an organism is Salmonella and belongs to an outbreak.
Brown favours retaining information in the initial search and filtering afterwards. Drawing on staged filtering in particle physics, he argues that discarded information cannot be recovered. Sourmash therefore supports custom databases, subsets of its distributed databases and access to broader matches. A messy reference might still contain something biologically interesting.
Accuracy depends on software foundations
Brown argues that most bacterial assignment tools for metagenomics perform poorly on either sensitivity or specificity at species level. He names sourmash, mOTUs and MEGAN among the few tools he considers comparatively strong on both measures, while noting that taxonomy was not sourmash’s original purpose.
Andrew points to three-year grants, departing postdocs and repeated reinvention as reasons the field struggles. Lee uses sourmash and SAMtools as examples of foundational software that others need to build upon. The tension is practical: public health needs working solutions now, even while developers continue improving the tools underneath them.
Preserving what software development teaches
Brown wants sourmash’s eventual replacement to retain its useful lessons. Publishing and explaining those lessons matters more to him than ensuring everyone continues using sourmash. He describes the risk of a successor improving most aspects of a tool while losing capabilities that were never properly documented.
His examples include a hashing approach that can work well for some tasks despite discarding 99.9% of the data, and MINSET-COV, which distils highly redundant microbial genome matches into a short list covering the known matches. Brown describes discovering that the latter approach used an established algorithm already taught in computer science courses.
A conditional case for perfect species classification
Brown argues that a virtually perfect species classifier is possible given a fixed taxonomy and reference database. That qualification matters: the taxonomy could be wrong and the database could be poor. He also acknowledges about a dozen genomes in his analysis that cannot be distinguished from one another using k-mers alone.
He credits Leighton Pritchard with suggesting the cryptographic concept of unicity distance. Applied here, it describes the smallest set of k-mers needed to distinguish a genome from the others. Brown reports that most GTDB genomes have a unicity distance of one: a single distinguishing k-mer. He says sourmash implicitly exploits this property, but frames high accuracy around the availability of distinguishing k-mers or combinations of them.
Species names must also work biologically and legally
Andrew raises Mycobacterium tuberculosis as an example where historical species definitions complicate genomic classification. Lee describes species that his reference laboratory distinguishes by one gene, and four lineages of Listeria monocytogenes that behave like different species but must retain the same species designation within the regulatory framework.
These examples explain why the hosts still care deeply about species-level calls, rather than only strain-level resolution. Brown closes by distinguishing what computers can resolve from what biology is saying. His remaining challenge is communicating the limits clearly: mathematical distinguishability within a reference collection does not, by itself, settle the biological and regulatory questions.
Highlights
- 00:00:50 — Andrew asks how sourmash handles poor-quality reference genomes.
- 00:01:20 — Brown explains why database selection should not be dictated by software capacity.
- 00:02:58 — Using GTDB releases as supplied and investigating contamination with Charcoal.
- 00:05:43 — Lee explains why applied phylogenetic pipelines remove potentially noisy SNPs.
- 00:06:19 — Sourmash’s flexibility and the case for staged filtering rather than early exclusion.
- 00:08:43 — Public health decisions require confidence in organism and outbreak assignments.
- 00:09:46 — Brown challenges the sensitivity and specificity of metagenomic assignment tools.
- 00:11:23 — Three-year grants and staff turnover contribute to repeated software reinvention.
- 00:14:23 — Preserving sourmash’s lessons so replacement software does not lose useful capabilities.
- 00:16:51 — Brown qualifies his classification claim by fixing the taxonomy and reference database.
- 00:18:23 — Unicity distance and distinguishing genomes through individual k-mers or combinations.
- 00:20:33 — Lee discusses single-gene species differences and regulatory treatment of Listeria lineages.
In their own words
And if you give it the wrong database, it’s going to search the wrong database just fine.
— Titus Brown, 00:02:58
Who is talking
- Lee Katz (host)
- Andrew Page (host)
- Titus Brown (guest)
Also mentioned: Nabil-Fareed Alikhan, Leighton Pritchard.
Tools and resources mentioned
sourmash, GTDB, GenBank, CheckM, Charcoal, SAMtools, mOTUs, MEGAN, MINSET-COV.
Questions this episode answers
Does sourmash remove poor-quality reference genomes?
Brown presents sourmash as a search tool, not a database curator. His group uses GTDB releases as supplied, while users can choose their own curated references or filter matches afterwards.
Can microbial species classification be 100% accurate?
Brown argues that near-perfect classification is possible given a fixed taxonomy, a reference database and distinguishing k-mers. He acknowledges indistinguishable genomes, while the hosts raise biological definitions and regulatory requirements that complicate a broader claim.
What does unicity distance mean for genome classification?
Brown uses it to describe the smallest set of k-mers that uniquely distinguishes a genome within a reference collection. He reports that most GTDB genomes can be distinguished by one k-mer.
Why does species identification matter in public health?
The hosts need confident organism assignments for public health decisions, and species names can determine regulatory treatment. Lee’s example is four Listeria monocytogenes lineages that behave differently but must be treated equally under the regulatory framework.
This page was written from a machine transcript of the episode and checked against it. Listen to the episode for the whole conversation.
Show notes
In this episode, Andrew Page and Lee Katz continue their chat with Titus Brown, focusing on taxonomy assignment in metagenomics:
Topics discussed:
Dealing with contamination and low quality genomes in reference databases Sourmash as a versatile search tool, not a curated database The need for high confidence in taxonomic assignment in public health Most microbial assignment tools have low specificity or sensitivity Possible ways to achieve perfect species classification (in theory) The challenges around defining species based on small genomic differences Interesting cryptography concept of ‘unicity’ distance for classification Conveying the nuances and uncertainties in taxonomic assignment The conversation highlights the difficulties around taxonomic classification, especially at the species level, but explores ideas for improving accuracy. Overall it emphasizes the complexities of biology and need for transparent conveyance of uncertainties.
Papers: Spacegraphcats - https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-02066-4 Sourmash - https://www.biorxiv.org/content/10.1101/2022.01.11.475838v2 IBD exploration - <https://dib- lab.github.io/2021-paper-ibd/>