Research opportunity · What kind of thing is generative AI, and does the answer predict anything?
Do products built on the same base model fail on the same people and cases?
Question
Among deployed generative AI products that are built on the same base model (for example, customer-service assistants, writing tools or screening tools from different firms), are errors and harmful outputs more strongly correlated across products, by case and by the group affected, than among products built on different base models?
Not yet measuredA PhD projectNeeds a budget (medium or higher)Partly addressed
Why it matters
The infrastructure account predicts that homogenization creates single points of failure and that flaws are inherited by every adapted model. Those are its distinctive observables, the ones the normal-technology and cultural-technology accounts do not imply, but the sources read for this debate state them without measuring them in deployment. Earlier work tested outcome homogenization on benchmarks with shared components, not across live products.
The debate lists this among what we still do not know: Are failures and biases correlated across products built on the same base models, as the infrastructure account predicts?
Evidence that motivates it
- Bommasani and colleagues named “foundation models” for their “critically central yet incomplete character,” argued that their significance lies in “emergence and homogenization,” warned that homogenization “creates single points of failure,” and predicted that the barrier to entry for developing them “will continue to rise.”
Foundation models are said to create single points of failure whose flaws are inherited downstream. - Van der Vlist, Helmond and Ferrari describe “Big AI” as the structural convergence of AI and Big Tech, arguing that AI is “an actual technology stack encompassing infrastructure, models, applications” and that Amazon, Microsoft and Google have integrated all three into their cloud offerings.
AI described as a stack integrated into three firms' cloud offerings. - Luitse and Denkena examine large language models as “infrastructural components for AI research and development,” arguing that they are intertwined with big technology companies’ business models and, using GPT-3, describing “corporate efforts to commodify” them “through paid API access and exclusive licensing.”
Language models as infrastructural components sold through metered access.
A possible design
Identify sets of deployed products in one domain whose base model is disclosed or can be identified, half sharing a base model and half not. Submit a common, pre-registered set of test cases, including cases from different demographic groups, through each product's public interface, repeat over time, and estimate the correlation of errors across products by shared versus different base model, controlling for task and vendor. Where possible, compare with the base model's own errors on the same cases.
Data needed
Product inventories with base-model disclosures; a test-case set with group labels; repeated access to public interfaces; terms-of-service review for testing.
Feasibility
A PhD project. Medium; subscription and API fees, and coder time for judging outputs. Base models are often undisclosed, and vendors change them; record versions and dates. Testing through public interfaces must respect terms of service, which may limit the sample.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 16 October 2026; not yet checked in depth).
No empirical study was found that measures correlated failures across independently deployed generative AI products built on one base model. Nearest work measures outcome homogenization on benchmarks or models it formally.
How strong this evidence is
- Who searched and checked: .
- What was read: 2 of the 2 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by one web search, arXiv record lookups and the LD-01 source scan on the date shown; industry audits and model-card disclosures were not searched. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Bommasani, Creel, Kumar, Jurafsky & Liang, "Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?", arXiv:2211.13972 (2022) Judged from an abstract, summary or search listing only
Abstract read. Formalizes outcome homogenization and tests it on benchmarks with shared training data and adapted foundation models, not on deployed products. - Raghavan, "Competition and Diversity in Generative AI," arXiv:2412.08610 (2024) Judged from an abstract, summary or search listing only
Abstract read. A game-theoretic model in which competition can reduce monoculture; no measurement across products.
Search queries used
outcome homogenization correlated failures deployed generative AI products sharing same foundation model empirical studyarXiv id_list: 2211.13972, 2412.08610LD-01 source scan for measurements of homogenization in the infrastructure strand (16 October 2026)
Provenance
RO-LD01-03 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-01 debate on 2026-10-16. Not yet reviewed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.