Research opportunity · Individual gain, collective narrowing? Does AI make science, creative work and shared knowledge more alike?
In science, is it the wording that converges or the ideas?
In short: has this been done?
Related work: Partly addressed. Related studies cover parts of this question; what they leave open is set out below. Only a first-pass search has been done, so it is not yet known whether a fuller search would find a study that does the whole design.
How far checked: First-pass screen only. A first-pass AI web search on 15 October 2026. No deeper check yet. Every search and check so far was done by AI; no person or domain expert has checked them yet.
The searches, the nearest work found and what the search could not reach
This page shows RO-LD09-06 version v1, whose record never changes. It is the current version; if the opportunity is revised, the newer version appears at its main page.
Question
In scientific papers and proposals written with language models, how much of the measured rise in similarity is shared wording and style, and how much is shared research questions, methods and claims, when the two are measured separately and validated against expert judgement?
Gap type: Wrong or missing validation targetOne PhD chapter or paperLow to medium costRelated work: Partly addressed
Why it matters
Most homogenization measures in writing are text-similarity scores, which cannot tell shared wording from shared ideas. One study of code finds convergence in syntax but not in approach, and the surface-not-substance position (P5) depends on this distinction. If ideas converge as well as wording, the narrowing concern is about knowledge, not style; if only wording converges, much of the measured homogenization is cosmetic.
The debate lists this among what we still do not know: When writing converges, is it the wording or the ideas?
Evidence that motivates it
- Kaggle competition code from 2019 to mid-2026 has become more alike in its literal syntax, but the author finds no evidence of homogenization in the approach or intent of the code. (LD09-SL-C33 v1.0)
How this study uses it: Kaggle code converged in syntax, with no evidence of homogenization in approach or intent. - By February 2024 an estimated 17.5% of sentences in computer-science abstracts on arXiv were substantially modified by language models; the share was higher in papers closer to their nearest peer, a proxy for more crowded research areas (22.2% against 14.7% for less similar papers). (LD09-SL-C09 v1.0)
How this study uses it: Model-modified sentences were more common in papers closer to their nearest peer. - Re-analysing a published set of five brainstorming experiments, the authors found ChatGPT-aided idea sets less diverse than comparison sets in all five (significantly in four), and lower on all of their alternative diversity metrics, with 37 of 45 comparisons statistically significant; in one task 6% of ChatGPT-only ideas were unique against 100% of human-only ideas. (LD09-SL-C12 v1.0)
How this study uses it: Idea sets were less diverse on most alternative measures in a brainstorming re-analysis.
Each claim is shown as it read when this version was published.
A possible design
Sample pairs of papers within subfields before and after 2023, matched on topic. For each paper, extract a structured record of research question, method, data and main claim, checked by domain experts, and separately a style representation from the text with content words masked. Estimate change in idea-level and style-level similarity as a function of estimated model use, and validate both measures against blind expert ratings of how alike pairs of papers are in substance.
Data needed
Full texts in two or three fields; domain experts for extraction checks and ratings; a distribution-level estimator of model-modified text.
Feasibility
One PhD chapter or paper. Low to medium; mostly expert time. Model-based extraction of ideas can itself homogenize; experts should check a random sample and the analysis should be repeated with a second extraction model.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).
Studies measure stylistic markers or overall semantic similarity of abstracts, and one measures proposal distinctiveness, but none was found that validates separate idea-level and style-level measures of convergence against expert judgement.
How strong this evidence is
- Who searched and checked: no checker is recorded beyond the first-pass screen, which was part of the AI drafting (Claude (SocioLens Project Lead), AI-drafted from the LD-09 debate). Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 5 of the 5 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by two web searches and arXiv record lookups on the date shown; scientometrics journals were not searched directly. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Qian, Wen, Furnas, Bai, Shao & Wang, "The Rise of Large Language Models and the Direction and Impact of US Federal Research Funding," arXiv:2601.15485 (2026) Judged from an abstract, summary or search listing only
Abstract read. Lower semantic distinctiveness of proposals with more model involvement; one embedding measure that mixes wording and substance. - Kousha & Thelwall, multi-database study of model-associated vocabulary, arXiv:2509.09596 (2025) Judged from an abstract, summary or search listing only
Abstract read. Vocabulary markers only. - Raccuglia, master's thesis on homogenization of academic abstracts, University of Padua (2025) Judged from an abstract, summary or search listing only
Seen in search results only; not read. Reported to find style changes without significant homogenization of lexicon and overall meaning in about 60,000 abstracts. - Burtch, Kaggle code study (LD09-SL-C33) Judged from an abstract, summary or search listing only
Abstract read for this debate. Separates syntax from approach, in code only. - Safari & Babaei, "Research Novelty in Information Systems Journals After ChatGPT: Differences Across Institutional Language Contexts," arXiv:2603.22510 (2026) Judged from an abstract, summary or search listing only
Abstract read. One embedding-based novelty measure of titles and abstracts, which mixes wording and substance.
Search queries used
LLM scientific abstracts style versus idea convergence separate stylistic homogenization from conceptual novelty measurementPNAS 2026 LLM involvement research proposals NSF NIH less semantically distinct funded arXivarXiv id_list: 2601.15485, 2509.09596, 2607.13077, 2603.22510
Cite this page
SocioLens project. (2026, October 15). In science, is it the wording that converges or the ideas? [Research opportunity RO-LD09-06, version v1]. SocioLens. https://sociolens-web.pages.dev/opportunities/ro-ld09-06/v1/
@misc{sociolens_ro_ld09_06_v1,
author = {{SocioLens project}},
title = {In science, is it the wording that converges or the ideas?},
year = {2026},
date = {2026-10-15},
howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld09-06/v1/}},
note = {Research opportunity RO-LD09-06, version v1}
}The link points to this version, whose record never changes. For a finding, cite the original study as well: SocioLens summarizes research and does not replace it.
Provenance
RO-LD09-06 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-09 debate on 2026-10-15. Not yet reviewed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.