Research opportunity · AI in the research apparatus
How much AI-assisted or agent responding reaches probability panels?
Earlier version (v1)
This is RO-LD07-03 as it was published at v1, kept unchanged. The current version is v2; its changes are listed there.
Question
In probability-based panels, what share of completed interviews show AI assistance or autonomous-agent completion when measured with embedded multi-method detection validated against consented chatbot-history linkage, and how does that compare with opt-in platforms measured the same way?
Not yet measuredOne PhD chapter or paperLow to medium costNo study found
Why it matters
Probability panels are the reference standard that much of social science relies on, and providers say they are less exposed. That is currently an assumption rather than a measurement. A measured rate, even a low one, would give the dispute about AI contamination of survey research its first evidence from this tier of panels.
The debate lists this among what we still do not know: How common are fully autonomous agents in probability-based or high-assurance survey panels?
Evidence that motivates it
- Autonomous LLM agents can complete online surveys while passing conventional attention and logic checks at very high rates under tested conditions.
Agents can pass conventional quality checks, so standard panel QA cannot be assumed to catch them. - AI-mediated responding is measurable in current online research, but self-reported assistance, overt outsourcing, AI-test failure and autonomous-agent detection are different estimands and cannot be pooled into one prevalence rate.
Existing prevalence estimates measure different pathways (assistance, outsourcing, AI-test failure, agents) and cannot be pooled into one rate; none of them comes from a probability panel.
A possible design
Partner with a probability panel that accepts researcher-proposed modules. Embed a short detection module: paste and keystroke paradata, honeypot items, and AI-capability test items, with a consented sub-sample linking chatbot histories (following Sehgal et al.) to validate the detectors. Field the identical module on two opt-in platforms. Report pathway-specific rates (assistance, outsourcing, agent completion) rather than one contamination rate.
Data needed
A module in a probability panel (several panels accept academic proposals at low or no cost); parallel opt-in samples; consent and ethics approval for history linkage.
Feasibility
One PhD chapter or paper. Low to medium: panel access is often free for academic proposals; opt-in samples cost a few hundred to a few thousand US dollars. Depends on a panel partner; the ethics approval for history linkage is the main hurdle.
Has this been done?
The search found no study that does the core design (first-pass screen of 12 October 2026; not yet checked in depth).
No measurement in a probability panel was found. Since the debate's evidence cut, Sehgal et al. (7 October 2026) linked donated chatbot histories to over 700,000 submissions on a curated opt-in platform and found about 1% AI-assisted, highly concentrated among a few workers, with half of assisted submissions falling outside what the platform detector covers. That method could validate detectors in a probability panel. Opt-in platform studies (Gordon et al.) and NORC's AmeriSpeak detector work exist, but neither reports a prevalence rate in a probability panel measured with validated detection.
How strong this evidence is
- Who searched and checked: an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: none of the 5 nearest works below is recorded as judged from an abstract only. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search on the date shown; Google Scholar, SSRN and paywalled journal pages were only partly reachable. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Sehgal, Tonneau, Folk, Ungar & Guntuku, "AI-Assisted Submissions in Online Research Are Rare and Highly Concentrated but Routinely Approved," arXiv:2610.09279 (7 Oct 2026)
About 1% AI-assisted submissions on a curated opt-in platform via chatbot-history linkage, concentrated among few workers; half of the assisted submissions were bounded responses that the platform detector does not cover. ChatGPT only, no agents, no probability panel. Published after LD-07's evidence cut. - Gordon, Rothschild, Affonso, Sulik, Hauser, Pepin & Jones, "AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers," PsyArXiv (Apr 2026)
Pre-registered study of 5,200 respondents across 10 opt-in platforms: agent detections concentrated on MTurk (11–16%), at or below 1% elsewhere; already cited in LD-07; no probability panel and no validation against known AI use. - Chen, Urminsky, Zhang et al., "Estimating the threat of AI-agent responding across online survey platforms," PsyArXiv (Mar 2026)
AI-test failure rates of 6–41% across seven platforms; the abstract does not indicate any probability panel. - NORC, "Fraudulent respondents and bots in nonprobability surveys: A literature review" (Apr 2026)
States that probability panels are less susceptible but reports no measured AI-responding rate. - Sepulvado, "Detecting AI Responses in Survey Data: NORC's Next Leap for Data Quality," NORC (Dec 2025)
Describes a classifier for AI-written open-ended answers trained on AmeriSpeak panelists versus LLM output; reports detector accuracy, not a prevalence rate in the panel.
Search queries used
LLM-generated responses probability-based panel LISS OR AmeriSpeak OR Understanding America Study OR KnowledgePanel detection AI use open-ended answersSocArXiv OR OSF prevalence LLM-assisted survey responding high-quality panel paste detection keystroke 2026AI agent bot responses probability-based panel prevalence LISS Understanding America Study GESIS detectionZhang Urminsky AI agents survey responses prevalence online panels Chicago Booth
Cite this page
SocioLens project. (2026, October 12). How much AI-assisted or agent responding reaches probability panels? [Research opportunity RO-LD07-03, version v1]. SocioLens. https://sociolens-web.pages.dev/opportunities/ro-ld07-03/v1/
@misc{sociolens_ro_ld07_03_v1,
author = {{SocioLens project}},
title = {How much AI-assisted or agent responding reaches probability panels?},
year = {2026},
date = {2026-10-12},
howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld07-03/v1/}},
note = {Research opportunity RO-LD07-03, version v1}
}The link points to this version, which never changes. For a finding, cite the original study as well: SocioLens summarizes research and does not replace it.
Provenance
RO-LD07-03 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-07 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.