Research opportunity · Individual gain, collective narrowing? Does AI make science, creative work and shared knowledge more alike?
Does pooling different models keep restoring diversity as models converge?
Question
Across successive generations of language models from several providers, how much diversity of ideas does pooling models add compared with one model, and is that gain shrinking as models' outputs and errors become more alike?
Untested beyond the original settingOne PhD chapter or paperLow to medium costPartly addressed
Why it matters
The design position (P3) rests on remedies such as pooling ideas across vendors, varied personas and chain-of-thought, which raised diversity almost to human levels in one study. But larger, more accurate models share their errors across providers, and different models give similar creative answers. If the pooling gain shrinks with each model generation, the design remedy has an expiry date.
The debate lists this among what we still do not know: Does pooling models and prompts keep working as models' errors and outputs converge?
Evidence that motivates it
- Ideas generated by language models for a new product had higher average purchase intent than human ideas and were 7 times more likely to rank in the top 10%, but were less diverse; pooling ideas across vendors, chain-of-thought prompting and varied personas or constraints raised their diversity “almost to the level of human idea generation”.
Pooling across vendors, chain-of-thought and personas raised the diversity of product ideas almost to human levels. - Across more than 350 language models, when two models both answer a leaderboard question wrongly they give the same wrong answer 60% of the time on one dataset, and larger, more accurate models have highly correlated errors even across different architectures and providers.
Larger, more accurate models have highly correlated errors across architectures and providers. - In a large dynamic experiment where each participant saw example ideas from ChatGPT or from earlier participants before brainstorming, high exposure to AI ideas did not make individual ideas more creative but increased the amount and rate of change of collective idea diversity: in the authors’ words, “AI made ideas different, not better”.
AI ideas used as passive examples increased collective diversity. - Across 27 language models, 155 topics and 12 countries, the epistemic diversity of models’ answers rose substantially over three years, but every model was less diverse than a web-search baseline, larger models were less diverse than smaller ones, and for country-specific topics models’ built-in knowledge reflected English more than local-language knowledge.
Larger models gave less diverse answers than smaller ones.
A possible design
Fix a set of open-ended ideation and writing tasks with human baselines. For each provider, collect outputs from model versions released from 2023 to 2026 (open-weight and archived API versions), under single-model, pooled and diversity-prompted conditions, with equal numbers of ideas. Measure idea-level diversity with blind human coding of a sample and validated embeddings for the rest, and estimate how the pooled-minus-single gain changes with release date and capability, with preregistered hypotheses.
Data needed
Access to archived model versions; a task set with human baselines; rater time for idea coding.
Feasibility
One PhD chapter or paper. Low to medium; mostly inference costs and rater time. Retired closed models may be unavailable, which biases the sample toward open-weight models; record exact versions and dates.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).
Studies show homogeneity across current models, a preliminary decline in output diversity across three years of releases, and gains from ensembles on benchmarks, but none was found that tracks the diversity gain from pooling across model generations.
How strong this evidence is
- Who searched and checked: .
- What was read: 5 of the 5 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by two web searches and arXiv record lookups on the date shown; HCI and creativity-research venues were not searched separately. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Wenger & Kenett, "We're Different, We're the Same: Creative Homogeneity Across LLMs," arXiv:2501.19361 (2025) Judged from an abstract, summary or search listing only
Abstract read. Responses of many models to standard creativity tests are more similar to each other than humans' responses are; one point in time. - Mathur, Boscoboinik, Tsai & Yager, "LLM Jaggedness Unlocks Scientific Creativity," arXiv:2605.10574 (2026) Judged from an abstract, summary or search listing only
Abstract read. 19 base models from 8 providers on a scientific-idea benchmark; pooled meta-model ensembles outperform single models; counts valid ideas, not diversity over generations. - Vallecillos-Ruiz, Hort & Moonen, "Wisdom and Delusion of LLM Ensembles for Code Generation and Repair," arXiv:2510.21513 (2025) Judged from an abstract, summary or search listing only
Abstract read. Ensembles of ten models from five families; consensus selection falls into a "popularity trap" while diversity-based selection works; correctness, not creative diversity. - Patel, Crossman, Aggarwal & Wenger, "Are LLMs becoming similarly creative? Evidence from three years of models," arXiv:2608.19437 (2026) Judged from an abstract, summary or search listing only
Abstract read. A preliminary analysis finding a statistically significant decrease in model output diversity over three years of releases, on open-ended user queries and a creativity test; it tracks convergence but not whether pooling still adds diversity. It points the other way from the rise in epistemic diversity reported in LD09-SL-C32, on a different task. - Haase, Hanel & Pokutta, "Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability," arXiv:2504.12320 (2025) Judged from an abstract, summary or search listing only
Abstract read. 14 models on two creativity tests; no rise in creative performance over 18–24 months and large variability within one model; scores, not pooled diversity.
Search queries used
multi-model ensemble pooling LLMs restore idea diversity across model generations convergence outputs studycreative output diversity across successive LLM versions over time GPT-3.5 GPT-4 GPT-4o homogeneity increasing studyarXiv id_list: 2501.19361, 2605.10574, 2510.21513, 2608.19437, 2504.12320
Provenance
RO-LD09-04 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-09 debate on 2026-10-15. Not yet reviewed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.