Research opportunity · Individual gain, collective narrowing? Does AI make science, creative work and shared knowledge more alike?
In science, is it the wording that converges or the ideas?
Question
In scientific papers and proposals written with language models, how much of the measured rise in similarity is shared wording and style, and how much is shared research questions, methods and claims, when the two are measured separately and validated against expert judgement?
Wrong or missing validation targetOne PhD chapter or paperLow to medium costPartly addressed
Why it matters
Most homogenization measures in writing are text-similarity scores, which cannot tell shared wording from shared ideas. One study of code finds convergence in syntax but not in approach, and the surface-not-substance position (P5) depends on this distinction. If ideas converge as well as wording, the narrowing concern is about knowledge, not style; if only wording converges, much of the measured homogenization is cosmetic.
The debate lists this among what we still do not know: When writing converges, is it the wording or the ideas?
Evidence that motivates it
- Kaggle competition code from 2019 to mid-2026 has become more alike in its literal syntax, but the author finds no evidence of homogenization in the approach or intent of the code.
Kaggle code converged in syntax, with no evidence of homogenization in approach or intent. - By February 2024 an estimated 17.5% of sentences in computer-science abstracts on arXiv were substantially modified by language models; the share was higher in papers closer to their nearest peer, a proxy for more crowded research areas (22.2% against 14.7% for less similar papers).
Model-modified sentences were more common in papers closer to their nearest peer. - Re-analysing a published set of five brainstorming experiments, the authors found ChatGPT-aided idea sets less diverse than comparison sets in all five (significantly in four), and lower on all of their alternative diversity metrics, with 37 of 45 comparisons statistically significant; in one task 6% of ChatGPT-only ideas were unique against 100% of human-only ideas.
Idea sets were less diverse on most alternative measures in a brainstorming re-analysis.
A possible design
Sample pairs of papers within subfields before and after 2023, matched on topic. For each paper, extract a structured record of research question, method, data and main claim, checked by domain experts, and separately a style representation from the text with content words masked. Estimate change in idea-level and style-level similarity as a function of estimated model use, and validate both measures against blind expert ratings of how alike pairs of papers are in substance.
Data needed
Full texts in two or three fields; domain experts for extraction checks and ratings; a distribution-level estimator of model-modified text.
Feasibility
One PhD chapter or paper. Low to medium; mostly expert time. Model-based extraction of ideas can itself homogenize; experts should check a random sample and the analysis should be repeated with a second extraction model.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).
Studies measure stylistic markers or overall semantic similarity of abstracts, and one measures proposal distinctiveness, but none was found that validates separate idea-level and style-level measures of convergence against expert judgement.
How strong this evidence is
- Who searched and checked: .
- What was read: 5 of the 5 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by two web searches and arXiv record lookups on the date shown; scientometrics journals were not searched directly. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Qian, Wen, Furnas, Bai, Shao & Wang, "The Rise of Large Language Models and the Direction and Impact of US Federal Research Funding," arXiv:2601.15485 (2026) Judged from an abstract, summary or search listing only
Abstract read. Lower semantic distinctiveness of proposals with more model involvement; one embedding measure that mixes wording and substance. - Kousha & Thelwall, multi-database study of model-associated vocabulary, arXiv:2509.09596 (2025) Judged from an abstract, summary or search listing only
Abstract read. Vocabulary markers only. - Raccuglia, master's thesis on homogenization of academic abstracts, University of Padua (2025) Judged from an abstract, summary or search listing only
Seen in search results only; not read. Reported to find style changes without significant homogenization of lexicon and overall meaning in about 60,000 abstracts. - Burtch, Kaggle code study (LD09-SL-C33) Judged from an abstract, summary or search listing only
Abstract read for this debate. Separates syntax from approach, in code only. - Safari & Babaei, "Research Novelty in Information Systems Journals After ChatGPT: Differences Across Institutional Language Contexts," arXiv:2603.22510 (2026) Judged from an abstract, summary or search listing only
Abstract read. One embedding-based novelty measure of titles and abstracts, which mixes wording and substance.
Search queries used
LLM scientific abstracts style versus idea convergence separate stylistic homogenization from conceptual novelty measurementPNAS 2026 LLM involvement research proposals NSF NIH less semantically distinct funded arXivarXiv id_list: 2601.15485, 2509.09596, 2607.13077, 2603.22510
Provenance
RO-LD09-06 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-09 debate on 2026-10-15. Not yet reviewed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.