Claim LD07-HQ-C06 (v1.0)

Synthetic respondents can match selected survey averages while failing to reproduce real-world variance, regression relationships, individual structure and subgroup heterogeneity.

Research opportunities this claim motivates

What this claim leaves open for research. Each is a hypothesis about a gap, not a finding.

Can synthetic respondents recover strong treatment-effect heterogeneity?Synthetic samples can match selected averages while failing to reproduce variance and subgroup heterogeneity.
How do research ethics boards judge synthetic participants and AI-interpreted data?Synthetic respondents do not substitute for populations, which bears on whether they reduce or relocate research risk.
Do silicon samples work outside the North Atlantic, and in which language?Subgroup and variance failures are documented in the contexts tested so far.

Provenance and status

Claim type: FACT + INTERPRETATION (as printed: “FACT / INTERPRETATION”)
Publication status: ELIGIBLE (as printed: “ELIGIBLE”)
Verification state: CONTENT_VERIFIED_PRIMARY (as printed: “CLEARED”; verified by: UNKNOWN)
Does not license: “Synthetic data are never useful”; impossibility of future validated simulators
Source (field 6, as printed): Bisbee et al. Political Analysis 2024; Xie et al. PNAS 2026; Wang et al. NMI 2025; Gong & Huang 2025; Ling & Cui 2025
Source type (field 7): PEER_REVIEWED_JOURNAL_ARTICLE (as printed: “Multiple peer-reviewed studies”)
Institutional position: Independent academic
Maturity (claim evidence / publication maturity): CONTESTED (NEGATIVE_EVIDENCE_MULTI_STUDY) (as printed: “CONTESTED, NEGATIVE EVIDENCE MULTI-STUDY”)
Recorded date: 2026-09-25
Currentness / correction: This is the current version. No correction or concern relation is recorded on its Sources.

Version

Current version v1.0, resolved through the explicit current pointer CLAIM--LD07-HQ-C06.

Versions:v1.0 · current

Pinned Sources (field 6, version-specific)

“Bisbee et al. Political Analysis 2024” → SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024--v1.0 — Bisbee, Clinton, Dorff, Kenkel & Larson, “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models,” Political Analysis 32(4):401–416 (2024); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
“Xie et al. PNAS 2026” → SRC-LD07-XIE-SSDATABENCH-2026--v1.0 — Xie et al., “Evaluating the statistical realism of LLM-generated social science data,” PNAS 123(19):e2538145123 (2026); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
“Wang et al. NMI 2025” → SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025--v1.0 — Wang et al. NMI 2025; class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
“Gong & Huang 2025” → SRC-LD07-GONG-HUANG-CGSS-2025--v1.0 — Gong Weigang & Huang Siyuan, “整体事实还是有偏样本——基于大语言模型生成数据的测量,” Academic Monthly 57(6):123–136 (2025); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
“Ling & Cui 2025” → SRC-LD07-LING-CUI-SILICON-SAMPLES-2025--v1.0 — Ling Wanying & Cui Sizhan, “复现的限度——‘硅基样本’的可能与可为,” Journal of Intelligent Society 4(2):179–201 (2025); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)

Read the originals

SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024: open the original
SRC-LD07-XIE-SSDATABENCH-2026: open the original
SRC-LD07-GONG-HUANG-CGSS-2025: open the original

All 25 canonical Claim fields (raw values and normalized values)

#	Canonical field	Canonical value (verbatim)	Normalized (Claim Value Vocabulary v1.0.2)
1	CLAIM ID	LD07-HQ-C06	—
2	PUBLIC-FACING CLAIM	Synthetic respondents can match selected survey averages while failing to reproduce real-world variance, regression relationships, individual structure and subgroup heterogeneity.	—
3	CLAIM TYPE	FACT / INTERPRETATION	base_types: FACT, INTERPRETATION
4	RESEARCH STRAND	C	—
5	MATURITY	CONTESTED, NEGATIVE EVIDENCE MULTI-STUDY	states: CONTESTED · qualifiers: NEGATIVE_EVIDENCE_MULTI_STUDY
6	SOURCE	Bisbee et al. Political Analysis 2024; Xie et al. PNAS 2026; Wang et al. NMI 2025; Gong & Huang 2025; Ling & Cui 2025	source_ids: SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024, SRC-LD07-XIE-SSDATABENCH-2026, SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025, SRC-LD07-GONG-HUANG-CGSS-2025, SRC-LD07-LING-CUI-SILICON-SAMPLES-2025
7	SOURCE TYPE	Multiple peer-reviewed studies	classes: PEER_REVIEWED_JOURNAL_ARTICLE
8	INSTITUTIONAL POSITION	Independent academic	—
9	SCOPE	ANES, multi-survey statistical benchmark, identity groups, CGSS	—
10	DESIGN	Human-vs-synthetic comparisons; statistical realism benchmarks	—
11	SAMPLE / DATASET	Multiple US/China/UK surveys and identity studies	—
12	MODEL / TOOL / VERSION	GPT-3.5/4 lineages plus multiple newer/open models	—
13	COMPARATOR	Human survey/population data	—
14	MAIN RESULT	Mean/topline match can coexist with compressed variance, altered coefficients, stereotyped/flattened groups and cross-model inconsistency	—
15	INFERENTIAL TARGET	Population representation and downstream associations	—
16	DOES NOT LICENSE	“Synthetic data are never useful”; impossibility of future validated simulators	—
17	VERIFICATION STATE	CLEARED	state: CONTENT_VERIFIED_PRIMARY · verified_by: UNKNOWN
18	LIMITATIONS	Designs and conditioning richness differ across studies	—
19	CONTRADICTORY / SUPPORTING SOURCES	Ashokkumar is positive on a different forecasting estimand	—
20	GEOGRAPHY / LANGUAGE	US, China, UK; English/Chinese	—
21	DISTRIBUTIONAL LEDGER	Marginal/identity groups particularly vulnerable to flattening/underrepresentation	—
22	R01 / FIELD-MAP LINKS	L-A, L-B, D9	—
23	WHAT WOULD CHANGE THIS CLAIM?	Preregistered subgroup-valid synthetic population benchmark with probability-survey gold	—
24	PUBLICATION STATUS	ELIGIBLE	state: ELIGIBLE
25	RECORDED DATE	2026-09-25	—

Field 22 references resolved against Field Map v0.2: L-A (LENS); L-B (LENS); D9 (DOMAIN)

Cited by Living Debate versions

LD-07 v0.1

Radar Updates referencing this ClaimVersion

RADAR-LD07-PEW-SYNTHETIC-SURVEYS-2026: Pew Research Center (30 September 2026) built an AI “digital twin” for each panelist analyzed from three 2026 waves of its American Trends Panel, a nationally representative panel of randomly selected U.S. adults. Each twin was conditioned on that person’s profile and earlier survey answers, and weighted synthetic and human results were compared on nearly 300 questions. With its best-performing configuration (Claude Opus 4.6), synthetic estimates differed from human estimates by an average of 12.4 percentage points. Error was substantial for every subgroup reported, reaching 16.1 points for Republicans and Republican leaners and 15.1 points for Black adults. On 47% of questions at least one answer choice was selected by no synthetic respondent, which happened on no question among humans. Results also differed systematically by AI model, although some toplines matched closely. This strengthens the evidence for LD07-HQ-C06 on selected-average agreement, real-world response variance and subgroup heterogeneity. The report contains no regression or treatment-effect analysis and no individual-level accuracy result for its final configuration, so those parts of the claim are unchanged. The claim wording is unchanged.

Research-of-record provenance

Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 da2589d0c80046281ebe5265eb8fbdd4168d24a03cf90062e8887b9f908894b1.

Frozen Drive-derived inputs

R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED (v1.0) — Drive ID 1xVz_LrQAA8EA1tPhVWIrDgOUU3Zidlg9; revision: NOT_EXPOSED_BY_DRIVE_TEXT_EXPORT; Drive modified 2026-09-29T04:31:14.120Z; block “ENTRY 06 — LD07-HQ-C06” SHA-256 4bcc6149e6f274dab11470cb90500fc248e0b7858fd5016e2abb8fecf89815e1; SocioLens — Programme Status & Authority Register v0.6 status “CURRENT.”; frozen fixture fixtures/frozen/r03-ld07-claim-ledger-v1.0-canonical.

Builder decisions (not Drive data)

VERSION = v1.0 — basis: Claim version taken from the canonical ledger artifact version (R03 — LD-07 Canonical Claim Ledger v1.0); Implementation Architecture Selection §5
CURRENT_POINTER = true — basis: SocioLens — Programme Status & Authority Register v0.6 status for "R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED": "CURRENT."
FIELD_6_SOURCE_REFERENCE = SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Bisbee et al. Political Analysis 2024"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §2; ledger C06; KEY EVIDENCE.
FIELD_6_SOURCE_REFERENCE = SRC-LD07-XIE-SSDATABENCH-2026--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Xie et al. PNAS 2026"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §3; ledger C06; KEY EVIDENCE.
FIELD_6_SOURCE_REFERENCE = SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Wang et al. NMI 2025"; the frozen field-6 segment of this ClaimVersion's ledger block; Central HQ ruling embedded in this issue, resolution ruling §1.
FIELD_6_SOURCE_REFERENCE = SRC-LD07-GONG-HUANG-CGSS-2025--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Gong & Huang 2025"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §4; ledger C06/C08; KEY EVIDENCE.
FIELD_6_SOURCE_REFERENCE = SRC-LD07-LING-CUI-SILICON-SAMPLES-2025--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Ling & Cui 2025"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §5 (distinct study); ledger C06/C08; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Bisbee et al. Political Analysis 2024"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §2; ledger C06; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-LD07-XIE-SSDATABENCH-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Xie et al. PNAS 2026"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §3; ledger C06; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Wang et al. NMI 2025"; Central HQ ruling embedded in this issue, resolution ruling §1.
SOURCE_PROVENANCE_PIN = SRC-LD07-GONG-HUANG-CGSS-2025--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Gong & Huang 2025"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §4; ledger C06/C08; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-LD07-LING-CUI-SILICON-SAMPLES-2025--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Ling & Cui 2025"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §5 (distinct study); ledger C06/C08; KEY EVIDENCE.