LD-07 — AI IN THE RESEARCH APPARATUS
Living Debate v0.1 — LOCKED
Last updated: 25 September 2026
QUESTION
Under what conditions can foundation-model-mediated social research produce valid knowledge?
And when do foundation models instead introduce measurement error, contamination, epistemic dependence, interpretive delegation or non-reproducibility?
SHORT ANSWER
Foundation models are not simply “good” or “bad” research instruments.
They are most defensible when treated as bounded, calibrated and versioned instruments for a clearly specified inferential target — with an appropriate human or external comparator, explicit validation and uncertainty carried into the downstream scientific claim.
The evidence is much weaker when models are asked to substitute for populations, stand in for causal mechanisms, autonomously interpret qualitative material or judge model-generated evidence without task-specific validation.
The central methodological rule is:
VALIDATE THE SCIENTIFIC CLAIM YOU CARE ABOUT — NOT JUST THE MODEL OUTPUT THAT FEEDS INTO IT.
EVIDENCE LANGUAGE
SocioLens distinguishes:
FACT — directly supported empirical or bibliographic result.
ESTIMATE — a measured quantity whose scope and estimator matter.
INTERPRETATION — SocioLens synthesis across evidence.
UNKNOWN — an unresolved empirical question.
WHY THIS DEBATE EXISTS
Foundation models now sit inside the research process in several different roles.
They can be:
• coders and annotators;
• evaluators or “LLM judges”;
• synthetic survey respondents or generative agents;
• assistants used by real research participants;
• retrieval and synthesis systems;
• qualitative co-analysts;
• components of reproducible or non-reproducible computational pipelines.
Those roles create different validity problems.
A model that classifies political texts is a measurement instrument.
A model that answers a survey as a synthetic persona is being treated as a population proxy.
A participant who uses a foundation model during a survey introduces AI mediation into the data-generating process; whether that constitutes contamination depends on the study’s estimand and protocol.
A model that interprets interview transcripts delegates part of the researcher’s interpretive work.
A proprietary API that changes over time is a non-stationary scientific instrument.
These cases should not be evaluated with a single benchmark or a single notion of “accuracy.”
1. PERFORMANCE IS NOT VALIDITY
FACT
A model can classify text accurately while still producing biased downstream coefficients or invalid confidence intervals.
The relevant question is not only:
“Did the model assign the same label as a human?”
It is also:
“Does using those labels preserve the scientific inference?”
Egami, Hinck, Stewart and Wei showed in a peer-reviewed methods study that treating imperfect predicted labels directly as observed variables can yield substantial bias and invalid confidence intervals even when prediction accuracy is high. Their design-based supervised-learning approach uses a probability-sampled human-labelled subset to recover valid downstream inference under stated assumptions.
A large 2025 preprint by Baumann and colleagues reaches a related conclusion empirically. Across 37 annotation tasks drawn from 21 published social-science studies, plausible model and configuration choices generated incorrect inferential conclusions in roughly one third of tested hypotheses for state-of-the-art models, with higher rates for smaller models.
This does not mean that one third of published social-science findings are wrong.
It means that, within the studies and configurations tested, apparently strong annotation performance did not reliably preserve the downstream statistical conclusion.
2. HUMAN AGREEMENT IS NOT A UNIVERSAL GOLD STANDARD
FACT + INTERPRETATION
Human coders are not interchangeable.
Crowdworkers, trained research assistants, domain experts, original study coders and newly recruited expert panels are different comparators.
This matters when claims say that an LLM “matches humans.”
A September 2026 preprint by Nakamura, Tan and Yean finds that recent LLMs can be observationally equivalent to expert coders on some standardized binary political-text classification tasks. On easier tasks, LLM–expert agreement is approximately as high as expert–expert agreement. Disagreement becomes concentrated in ambiguous texts, and clearer codebooks improve agreement for both humans and models.
That finding does not contradict studies showing downstream instability.
A conditionally accepted AJPS manuscript by Yang, Wang, Zhou and Xu asks a different question: if LLM labels are substituted into published political-science pipelines, do the original results survive? In their 14-study evaluation, around 35–42% of resulting estimates changed the substantive statistical conclusion under the tested configurations.
Both findings can be true.
LLM ≈ EXPERT DOES NOT IMPLY LLM LABELS PRESERVE A PUBLISHED INFERENCE.
3. REPLACEMENT IS A TEST RESULT, NOT A PROPERTY OF “LLMS”
FACT
The replacement question can be tested directly.
Calderon, Reichart and Dror propose an Alternative Annotator Test that compares an LLM with human annotators on a task-specific statistical basis. Across ten datasets, some LLMs passed replacement tests and others did not; model and prompting choice mattered.
The implication is not that LLM replacement is generally safe.
It is that replacement should be treated as an empirical claim about:
• this model;
• this task;
• this codebook;
• this dataset;
• this comparator.
A new model version, domain or construct can require revalidation.
SYNTHETIC RESPONDENTS: WHAT EXACTLY IS BEING REPRODUCED?
Synthetic respondents create one of the most confusing parts of the current debate because positive and negative studies often estimate different things.
WHAT CAN WORK
FACT
Ashokkumar, Hewitt, Ghezae and Willer tested whether GPT-4 could forecast the results of 70 preregistered, nationally representative US survey experiments covering 469 effects and 119,330 human participants.
The model’s predicted effects were strongly correlated with realized effects and performed similarly to pooled human forecasts. Performance remained strong for studies that were not public by the model’s training cutoff.
But the models systematically overestimated effect sizes.
This supports a bounded use:
LLMs may help with piloting, prioritization or forecasting which interventions are likely to produce larger effects.
It does not establish that synthetic respondents are valid replacements for actual populations.
WHAT DOES NOT FOLLOW
FACT + INTERPRETATION
Bisbee and colleagues found that persona-conditioned ChatGPT responses could reproduce some survey averages while showing less variation than human respondents. Nearly half of their synthetic regression coefficients differed significantly from corresponding ANES estimates; among those differences, sign reversals were common. Results also shifted under small prompt changes and across model updates.
Xie and colleagues’ SSDataBench extends the problem across 15 LLMs and multiple cross-sectional and longitudinal datasets from the United States, China and the United Kingdom. Under sparse demographic conditioning, models tend to compress distributions toward typical profiles, exaggerate some associations and fail to reproduce important life-course structure.
Chinese-language studies show a related pattern.
Gong Weigang and Huang Siyuan benchmarked LLM-generated data against CGSS2021. Their study reports partial recovery of macro-level relationships alongside representational bias, including a tendency to reinforce mainstream discourse and neglect marginal voices.
Wanying Ling and Sizhan Cui compared DeepSeek-v3, GPT-4o and Qwen-plus with CGSS 2011 and 2021 health modules. They report instability at individual, aggregate and relational levels, as well as differences across models, and conclude that current silicon samples cannot replace human respondents.
The strongest lesson is not that synthetic respondents are useless.
It is that different validation targets must not be substituted for one another.
MATCHING MEANS DOES NOT VALIDATE REGRESSIONS.
MATCHING RESPONSE DISTRIBUTIONS DOES NOT VALIDATE TREATMENT EFFECTS.
FORECASTING EFFECT RANKINGS DOES NOT VALIDATE CAUSAL MECHANISMS.
A 2026 cross-national preprint by Li and Ji makes this problem explicit. Across 62 countries and 59,508 human participants, statistical realism of synthetic responses was only weakly related to treatment-effect accuracy. Models could look realistic at the response level while producing substantially worse treatment-effect estimates than supervised baselines.
SURVEY CONTAMINATION: CAPABILITY IS NOT PREVALENCE
FACT
There is strong evidence that autonomous LLM agents can complete online surveys while evading conventional quality checks.
Westwood’s 2025 PNAS study built an autonomous synthetic respondent that passed 99.8% of 6,000 standard attention-check trials. It could maintain a demographic persona, answer logic questions, generate plausible open text and simulate human-like interaction.
That is a capability result.
It is not an estimate of how many real survey responses are currently generated by autonomous agents.
FIELD PREVALENCE IS MORE COMPLICATED
ESTIMATE + INTERPRETATION
Several 2025–2026 studies now measure AI-mediated responding in real online research.
But they measure different things.
Westwood and Frederick flagged 4.4% of one Prolific sample using at least one of two passive detection methods.
Asher, Gold, Chen and Carvalho found that approximately 9% of participants across three Prolific studies submitted responses consistent with AI assistance or other outsourced responding under a conservative keystroke-based definition.
Zhang, Xu and Alvero found that 34% of participants in an online research sample self-reported using LLMs to help answer open-ended questions.
Other studies find much larger rates on some text-production tasks and on MTurk. A preregistered 2026 cross-platform preprint led by Andrew Gordon found detections concentrated on MTurk and at or below about 1% on the other tested providers under its detection protocol.
These estimates should not be averaged.
They target different phenomena:
• autonomous agents;
• human participants using AI assistance;
• overt outsourcing;
• copy-paste or keystroke anomalies;
• failure on AI-detection batteries.
There is no defensible single “AI contamination rate.”
The better current statement is:
AI-MEDIATED RESPONDING IS MEASURABLE, BUT ITS PREVALENCE IS PATHWAY-, TASK-, DETECTOR- AND PLATFORM-DEPENDENT.
Full-agent prevalence in high-assurance and probability-based panels remains much less well established.
LLM-AS-JUDGE: AGREEMENT DOES NOT MEAN NEUTRALITY
FACT
Using one LLM to evaluate another is now common in AI research.
But an LLM judge is itself a measurement instrument.
Peer-reviewed studies document systematic evaluator effects.
Shi and colleagues evaluated 15 LLM judges across more than 150,000 judgments and found that presentation order can systematically alter judgments. Position bias varies across judge, candidate and task.
Panickssery, Bowman and Feng showed that evaluators such as GPT-4 and Llama 2 can recognize and favor their own generations. Their controlled experiments link self-recognition to self-preference.
This does not mean LLM judging is unusable.
It means human agreement alone does not make a judge unbiased, and evaluation protocol is part of the instrument.
A judge intended to measure a social-science construct — argument quality, deliberativeness, incivility, trustworthiness or moral reasoning, for example — requires validation against that construct, not merely against another model’s preferences.
PROVIDER, ALIGNMENT AND MODEL HISTORY
MODEL VERSION MATTERS
FACT
The model itself is not a timeless object.
Chen, Zaharia and Zou compared March and June 2023 snapshots of GPT-3.5 and GPT-4 and found material changes across several tasks, including opinion questions.
The point is not the size of any one historical change.
The methodological lesson is that exact version and date belong in the research protocol.
“GPT-4” or “Claude” is not enough information to identify a scientific instrument.
PROVIDER AND JURISDICTION CAN MATTER
FACT + INTERPRETATION
Pan and Xu compared China-originating and non-China-originating models on 145 politically sensitive questions.
China-originating models showed substantially more refusal, shorter answers and more inaccurate answers on sensitive China-related prompts. The gaps became smaller on less-sensitive questions.
The authors explicitly caution that their design is observational and does not establish that regulation caused these differences.
The defensible methodological conclusion is narrower:
PROVIDER AND ALIGNMENT ENVIRONMENTS CAN BE ASSOCIATED WITH MATERIALLY DIFFERENT RESPONSE SURFACES; EXISTING EVIDENCE DOES NOT ISOLATE THE CAUSAL MECHANISM.
A model used to study politically sensitive attitudes is not simply a neutral questionnaire with better language generation.
QUALITATIVE RESEARCH: AUGMENTATION IS NOT INTERPRETIVE DELEGATION
FACT + INTERPRETATION
Qualitative research makes the distinction between coding and interpretation especially important.
Than, Fan, Law, Nelson and McCall use a long-running qualitative project on US newsweeklies to show how generative LLMs can participate in an iterative coding workflow. Their proposed process remains explicitly researcher–LLM–researcher: detailed coding guides, human concept development and human validation remain central.
A blinded 2026 PLOS Digital Health comparison similarly finds strong performance for deductive coding against a predefined codebook but more variable performance for inductive theme generation, especially where context, tone and nuance matter.
Even “agreement with humans” can mislead here.
A 2026 preprint by Liu and colleagues compared five LLM systems and three trained human coders on 2,560 educator messages. LLMs agreed with humans less than humans agreed with one another, but a blind domain expert rated human and LLM code sets at statistically indistinguishable rates.
This result cuts in both directions.
It warns against assuming that human consensus is automatically ground truth.
It also warns against using agreement metrics as a substitute for independent quality judgment.
The evidence currently supports:
• codebook-guided assistance;
• bounded deductive coding;
• selective scaling;
• researcher–model iteration.
It does not establish:
• autonomous reflexive interpretation;
• reliable unattended inductive theme discovery;
• replacement of researcher positionality and judgement.
THE FOUR POSITIONS
P1 — REPLACEMENTIST
Claim:
For sufficiently capable models, human coding or participation can often be replaced.
Current status:
Supported only narrowly.
Some annotation tasks can pass explicit replacement tests.
The evidence does not support generic replacement of expert coders, populations or qualitative interpreters.
P2 — BOUNDED-INSTRUMENT
Claim:
Treat the model as a fallible measurement instrument whose validity must be established for a specified estimand.
Current status:
Best supported.
This position fits the strongest methods evidence: design-based correction, task-specific replacement testing, versioning and downstream validation.
P3 — REFLEXIVE-CRITICAL
Claim:
Model outputs carry institutional, political and epistemic priors that must themselves become objects of methodological scrutiny.
Current status:
Supported as a validity condition.
Provider context, alignment, non-stationarity and researcher/model recursion matter. This evidence does not imply that model-mediated research should be rejected wholesale.
P4 — HYBRID / SEQUENCED
Claim:
Models and humans should do different parts of the research process, with humans retaining conceptualization, validation and interpretive responsibility.
Current status:
Best supported alongside P2.
The evidence is strongest for workflows in which models supply scale or forecasting while humans retain external validation and responsibility for the inferential target.
WHAT IS OLD — AND WHAT IS NEW?
Many problems in this debate are familiar.
OLD VALIDITY PROBLEMS
• measurement error;
• weak human baselines;
• construct validity;
• sampling and coverage;
• order effects;
• researcher degrees of freedom;
• uncertainty propagation;
• qualitative reflexivity.
FOUNDATION MODELS AMPLIFY THEM
A natural-language prompt creates a large new configuration space.
Crowdworkers and research participants can themselves use foundation models.
Model versions and providers can change the instrument.
Prompt iteration can become a new garden of forking paths.
SOME PROBLEMS ARE MORE DISTINCTIVE
Participants and researchers can use foundation models within the same data-generating and measurement pipeline.
Provider and alignment environments can be associated with materially different response surfaces; existing evidence does not isolate the causal mechanism.
Interpretation can be delegated at a scale that was previously impossible.
WHAT WE STILL DO NOT KNOW
UNKNOWN
1. How stable are 2025–2026 model snapshots over time when used for the same social-science measurement task?
2. Can synthetic populations ever recover subgroup treatment effects and causal mechanisms out of distribution?
3. How common are fully autonomous agents in probability-based or high-assurance survey panels?
4. How much do jurisdictional alignment regimes causally change social-science measurement, rather than merely refusal behavior?
5. How often does retrieval-augmented synthesis change which social-science evidence enters a review and therefore changes its conclusion?
6. How should research ethics boards evaluate delegated interpretation, synthetic participants and confidential-data use?
7. For open-weight models, do deployment choices such as quantization materially change social-science reliability metrics?
WHAT EVIDENCE WOULD CHANGE THE PICTURE?
The current assessment would shift materially with:
• independent journal replications showing that modern models preserve downstream inference across many annotation tasks without human validation;
• preregistered synthetic-population studies that recover subgroup distributions and heterogeneous treatment effects against probability-sample ground truth;
• independent prevalence audits of autonomous agents in high-assurance and probability panels;
• longitudinal audits of pinned contemporary model versions;
• causal alignment interventions that isolate the effect of institutional or policy constraints on measurement;
• multi-site qualitative studies showing robust unattended inductive interpretation.
METHODS NOTE
SocioLens treats these claims as properties of research designs, not model rankings.
For each major claim we record:
• the inferential target;
• model role and version where relevant;
• comparator;
• design;
• sample and geography;
• publication status;
• what the evidence does not establish;
• what evidence would change the claim.
GEOGRAPHIC AND LANGUAGE SCOPE
The evidence base remains disproportionately US- and English-centered.
This page incorporates:
• Chinese-language CGSS validation studies;
• China-origin vs non-China-origin model comparisons;
• cross-national treatment-effect evidence;
• some multilingual annotation and evaluation work.
Important gaps remain in:
• African languages and contexts;
• Arabic;
• Southeast Asia;
• Latin America;
• low-resource multilingual qualitative research.
WHAT MOVED THIS DEBATE?
2023–2024
Early work established both striking annotation performance and major concerns about synthetic-population inference and model drift.
2025
The debate moved from “Can LLMs code text?” toward configuration dependence, survey contamination and more formal methods for validating replacement.
2026
The central question became more precise:
Which estimand is valid under which model-mediated design?
New evidence now distinguishes:
• forecasting from population substitution;
• capability from prevalence;
• agreement from inferential validity;
• deductive coding from interpretive delegation.
CURRENT SOCIOLENS POSITION
INTERPRETATION
The strongest evidence supports foundation models as bounded, calibrated research instruments — not as generic substitutes for populations or autonomous interpreters.
The relevant standard is not whether a model is “human-level.”
It is whether the specific model-mediated measurement process preserves the scientific claim being made.
KEY EVIDENCE
Annotation and downstream inference
• Egami, Hinck, Stewart & Wei. “Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models.” NeurIPS 2023. arXiv:2306.04746. Peer reviewed.
• Baumann, Röttger, Urman, Wendsjö, Plaza-del-Arco, Gruber & Hovy. “Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation.” arXiv:2509.08825v2. Preprint.
• Yang, Wang, Zhou & Xu. “Data Annotation with Large Language Models: Lessons from a Large Empirical Evaluation.” Conditionally accepted at AJPS; pre-publication manuscript.
• Nakamura, Tan & Yean. “Observational Equivalence of LLM and Human Annotation.” arXiv:2609.22133. Preprint.
• Calderon, Reichart & Dror. “The Alternative Annotator Test for LLM-as-a-Judge.” ACL 2025. doi:10.18653/v1/2025.acl-long.782. Peer reviewed.
Synthetic respondents
• Ashokkumar, Hewitt, Ghezae & Willer. “Large language models can predict the results of social science experiments.” Nature 656:115–122 (2026). doi:10.1038/s41586-026-10742-x. Peer reviewed.
• Bisbee, Clinton, Dorff, Kenkel & Larson. “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models.” Political Analysis 32(4):401–416 (2024). doi:10.1017/pan.2024.5. Peer reviewed.
• Xie et al. “Evaluating the statistical realism of LLM-generated social science data.” PNAS 123(19):e2538145123 (2026). doi:10.1073/pnas.2538145123. Peer reviewed.
• Gong Weigang & Huang Siyuan. “整体事实还是有偏样本——基于大语言模型生成数据的测量.” Academic Monthly 57(6):123–136 (2025). Peer reviewed.
• Ling Wanying & Cui Sizhan. “复现的限度——‘硅基样本’的可能与可为.” Journal of Intelligent Society 4(2):179–201 (2025). Peer reviewed.
• Li, Zonghan & Ji, Feng. “Statistical realism is not evidence that LLMs can estimate treatment effects in social science experiments.” arXiv:2604.02458. Preprint.
Survey contamination
• Westwood. “The potential existential threat of large language models to online survey research.” PNAS 122(47):e2518075122 (2025). doi:10.1073/pnas.2518075122. Peer reviewed.
• Westwood & Frederick. “Reply to Van der Stigchel et al.: Empirical evidence that AI survey contamination is real and substantial.” PNAS 123(8):e2537420123 (2026). doi:10.1073/pnas.2537420123. Peer reviewed letter.
• Asher, Gold, Chen & Carvalho. “Chatbots Are Undermining Crowdsourced Research in the Behavioral Sciences.” AMPPS 9(1) (2026). doi:10.1177/25152459261424723. Peer reviewed.
• Zhang, Xu & Alvero. “Generative AI Meets Open-Ended Survey Responses.” Sociological Methods & Research 54(3):1197–1242 (2025). doi:10.1177/00491241251327130. Peer reviewed.
• Gordon et al. “AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers.” PsyArXiv (2026). Preprint.
• Lin. “Artificial-Intelligence-Mediated Contamination in Online Research: Taxonomy, Risk Gradient, and Recommendations.” AMPPS 9(3) (2026). doi:10.1177/25152459261454231. Peer reviewed.
Judges, priors and qualitative interpretation
• Pan & Xu. “Political censorship in large language models originating from China.” PNAS Nexus 5(2):pgag013 (2026). doi:10.1093/pnasnexus/pgag013. Peer reviewed.
• Chen, Zaharia & Zou. “How is ChatGPT’s behavior changing over time?” Harvard Data Science Review 6(2) (2024). doi:10.1162/99608f92.5317da47. Peer reviewed.
• Shi et al. “Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge.” IJCNLP-AACL 2025. doi:10.18653/v1/2025.ijcnlp-long.18. Peer reviewed.
• Panickssery, Bowman & Feng. “LLM Evaluators Recognize and Favor Their Own Generations.” NeurIPS 2024. arXiv:2404.13076. Peer reviewed.
• Than, Fan, Law, Nelson & McCall. “Updating ‘The Future of Coding’: Qualitative Coding with Generative Large Language Models.” Sociological Methods & Research 54(3):849–888 (2025). doi:10.1177/00491241251339188. Peer reviewed.
• “Large language models for thematic analysis in healthcare research: A blinded mixed-methods comparison with human analysts.” PLOS Digital Health (2026). doi:10.1371/journal.pdig.0001189. Peer reviewed.
• Liu et al. “Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth.” arXiv:2607.28890. Preprint.
RELATED SOCIOLENS DEBATES
LD-01 — What kind of thing is generative AI?
LD-06 — The evaluation apparatus
LD-08 — Who can know about AI? Epistemic dependence and jurisdiction
LD-09 — Individual gain, collective narrowing
LD-13 — Safety and justice
PROVENANCE
Research build:
R03 — LD-07 — AI in the Research Apparatus — Evidence Build
Claims were not selected by model vote.
The final v0.1 claim set was adjudicated against primary sources in four HQ batches:
1. Annotation & downstream inference
2. Synthetic respondents
3. Survey contamination
4. Model priors, judges & qualitative interpretation
Internal status:
LD-07 Living Debate v0.1 — LOCKED
25 September 2026
Future changes require a versioned changelog and evidence-based amendment.