ChatGPT authorship and ethics in microbial genomics publishing
Lee Katz and Andrew Page discuss Andrew’s editorial in Microbial Genomics, which combines a human-written introduction with AI-generated text. They examine authorship, plagiarism, prompt disclosure and reproducibility, alongside potential uses of AI in peer review and scientific image manipulation. Bioinformatics examples show why plausible output still needs checking against current methods and primary sources.
Listen to Episode 116: AI Authorship and Ethics in Academic Publishing for Genomics on SoundCloud
In this episode
A human introduction and an AI-written editorial
Andrew describes how a conversation with Sam Sheppard at a Cambridge hackathon led to an editorial about AI in scholarly publishing. The Microbial Genomics article, co-authored with Niamh Tumelty and Sam, has a human-written introduction followed by AI-generated content. Andrew had already asked ChatGPT to produce a podcast script, read by AI voices from ElevenLabs, before asking it to write a review of the area. He also used it to draft promotional tweets with emojis.
Andrew initially entered ChatGPT as the last author in Editorial Manager. It remained there until copy-editing, when it was removed. This raises the episode’s central authorship question: is generative AI simply another writing tool, or does its contribution require a different form of acknowledgement?
Plagiarism, sources and narrow research questions
The hosts do not settle on a single answer to whether AI-generated writing constitutes plagiarism. Andrew asks whether taking generated work and presenting it as one’s own resembles copying from Wikipedia without attribution. Lee distinguishes broad subjects, discussed by millions of people, from narrow scientific questions. His example is using Roary with Salmonella enterica: relatively few sources may underpin a response, making direct dependence on particular researchers’ ideas harder to dismiss.
Lee compares ChatGPT with the warnings students once received about Wikipedia: check the underlying evidence and consult primary sources. He contrasts the lack of citations in the main ChatGPT interface with attempts by Bing and Google to display sources. The hosts warn that a generated reference can look convincing while being incorrect. Andrew also describes PubMed material published under a CC BY licence before 2021 as part of the model’s scientific source material.
What should researchers disclose?
Andrew suggests that prompts, including intermediate prompts, might belong in supplementary material. Earlier exchanges can shape later responses: in his own workflow, the podcast script preceded the editorial request. He also asked ChatGPT to suggest a better prompt. The hosts disclose that they used ChatGPT to identify discussion points for this episode.
They compare prompt reporting with systematic reviews, where researchers record search terms, databases and dates. Lee asks whether reporting versions and random seeds could make AI use reproducible. Andrew points to changing versions, inaccessible underlying models and differing functionality within the interface as obstacles. Publishing prompts would therefore not guarantee identical output. Their discussion of alternatives includes Stanford’s Alpaca and Meta’s LLaMA, alongside Andrew’s concern that openness about training datasets can expose model developers to legal challenges.
AI reviewers and convincing scientific images
Andrew asked ChatGPT for a highly critical review of his Roary paper and received a damning response that was not entirely correct. Lee recalls an unfair question about whether the software would be open source, despite his understanding that the paper already addressed this. They consider a possible publishing loop in which AI-written papers receive AI-written reviews, with humans increasingly peripheral. Andrew worries that journals struggling to recruit reviewers could substitute plausible-looking automated assessments without adequate transparency.
The conversation also covers Adobe’s image-expansion features, including examples extending the Mona Lisa. The hosts connect convincing generated images with scientific figure integrity. They worry that manipulation could become harder to detect through visual inspection, especially when existing checks rely on spotting repeated patterns in images.
Bias and outdated bioinformatics advice
Andrew identifies unrecognised plagiarism and inherited biases as major risks. He worries that online source material could reproduce misogyny, extremist views or stereotyped representations of scientists. Lee recalls Microsoft’s earlier Twitter AI adopting Nazi views and being shut down. Their concern extends beyond obvious offensive output to subtler biases that a researcher might not notice.
Andrew’s concrete coding example involves pandas data frames. AI-generated suggestions repeatedly used deprecated methods, producing warnings or crashes because libraries and APIs had moved on. Familiar or widely represented approaches are not necessarily current or correct. Lee extends this to literature coverage, noting that a model limited to material before 2021 would miss newer research. Andrew uses rapidly improving Nanopore basecalling as another example: assumptions appropriate to 90% accuracy may lead to different analytical decisions from those appropriate to 98% accuracy. These are illustrative figures, not a benchmark reported in the episode.
Different perspectives on responsible assistance
Andrew describes Niamh Tumelty’s contribution as particularly focused on ethics, provenance and the need for policies. Her work with the London School of Economics library and its press brings an information and publishing perspective. Sam Sheppard, as editor-in-chief of Microbial Genomics, faces the practical problem that journals will receive AI-assisted submissions whether or not they explicitly welcome them.
The hosts distinguish these concerns from potentially useful assistance. Improving English, tidying language or restructuring prose may help authors communicate their research. Andrew notes that researchers already employ people to check their writing without necessarily granting authorship. Rather than treating all assistance as equivalent, the discussion calls for clarity about how it is used. Andrew expects the technology to move faster than the policies and procedures intended to govern it.
Highlights
- 00:01:01 — A Cambridge hackathon conversation leads to a partly AI-generated editorial
- 00:02:30 — Andrew’s attempt to list ChatGPT as an author ends during copy-editing
- 00:07:20 — Could AI-written papers end up being assessed by AI reviewers?
- 00:09:53 — Whether AI-generated academic writing should be considered plagiarism
- 00:10:20 — Roary and Salmonella enterica illustrate the risks of narrow source coverage
- 00:12:12 — The Wikipedia comparison and the need to verify primary sources
- 00:14:41 — Best practices for acknowledging AI assistance and reporting prompts
- 00:17:47 — Open models, training-data disputes, Alpaca and LLaMA
- 00:20:01 — Convincing image expansion raises concerns about scientific figure integrity
- 00:23:50 — Unrecognised plagiarism and biases inherited from online material
- 00:26:37 — Deprecated pandas methods show why generated code needs checking
- 00:28:44 — Niamh Tumelty’s and Sam Sheppard’s perspectives on ethics and publishing
In their own words
just because it’s a consensus doesn’t mean it’s correct.
— Andrew Page, 00:26:37
And it’s going to be, I think the technology is going to move much faster than we can actually put policies and procedures in place to deal with it.
— Andrew Page, 00:28:44
Who is talking
- Lee Katz (host)
- Andrew Page (host)
Also mentioned: Nabil-Fareed Alikhan, Niamh Tumelty, Sam Sheppard.
Tools and resources mentioned
ChatGPT, GPT-4, Editorial Manager, ElevenLabs, Roary, pandas, PubMed, Bing, Google, Alpaca, LLaMA.
Questions this episode answers
Was ChatGPT credited as an author on Andrew Page’s editorial?
Andrew says he entered ChatGPT as the last author in Editorial Manager, but it was removed during copy-editing. The episode discusses this experience without establishing a general authorship rule.
Should researchers publish the prompts used for AI-assisted writing?
Andrew suggests reporting prompts, including intermediate exchanges, in supplementary material. The hosts note that disclosure would improve transparency but would not guarantee reproducibility because model versions, functionality and generated responses can change.
What bioinformatics problems did Andrew encounter with AI-generated code?
While working with pandas data frames, Andrew repeatedly received suggestions using deprecated methods that produced warnings or crashes. He also discusses Nanopore basecalling to illustrate why older assumptions may be unsuitable for rapidly improving technologies.
Do the hosts distinguish language assistance from writing an entire paper?
Yes. Andrew identifies improving English, tidying prose and restructuring text as potentially useful forms of assistance, comparing them with human language-checking services that do not necessarily receive authorship. The discussion emphasises transparency and clearer policies rather than a blanket rejection of AI.
This page was written from a machine transcript of the episode and checked against it. Listen to the episode for the whole conversation.
Show notes
The podcast discusses an article co-authored by Andrew Page, examining the use of GPT-4 for research publication. The conversation focuses on the authorship of articles generated by GPT-4 and the implications for academic publishing.
Authorship and Ethics:
Andrew discusses the question of authorship when AI-generated content is involved in research articles. He explores the ethical implications and potential biases associated with AI-assisted writing, such as the omission of minority figures and novel discoveries. He emphasizes the importance of transparency when using AI and its potential to democratize research, as long as ethical guidelines are maintained.
AI & Scientific Journals:
The podcast delves into the current landscape of AI in academic publishing. It addresses the commercial use of AI in crafting manuscripts for research articles and the necessity of distinguishing between manual and AI-generated contributions. The possible misalignment of GPT-4’s commercial objectives with academic goals is highlighted.
Risks and Benefits:
Andrew outlines the risks of using AI in publishing, such as unintentional plagiarism, biases, and outdated methods. He provides an example of bioinformatics software recommending deprecated methods, illustrating the need for caution. The conversation also touches upon the AI’s potential to introduce bias unintentionally, citing past incidents where AI models quickly adopted extremist views.
Andrew’s co-authors, Niamh Tumelty and Sam Sheppard, bring different perspectives on ethics and the impact of AI on publishing. Niamh, associated with the London School of Economics, emphasizes ethical considerations, while Sam, editor-in-chief of Microbial Genomics, underscores the need to adapt to the reality of AI contributions in journal submissions. In conclusion, the podcast underscores the importance of recognizing and navigating the ethical challenges posed by AI in academic publishing. It suggests that the technology may evolve faster than policies can adapt, necessitating an ongoing conversation among researchers, publishers, and AI developers.
Links: <https://microbiologysociety.org/blog/microbe-talk-ai-a-useful-tool-or-dangerous- unstoppable-force.html> https://www.microbiologyresearch.org/content/journal/mgen/10.1099/mgen.0.001049