Claim LD07-HQ-C06 (v1.0)
Synthetic respondents can match selected survey averages while failing to reproduce real-world variance, regression relationships, individual structure and subgroup heterogeneity.
Provenance and status
- Claim type
- FACT + INTERPRETATION (as printed: “FACT / INTERPRETATION”)
- Publication status
- ELIGIBLE (as printed: “ELIGIBLE”)
- Verification state
- CONTENT_VERIFIED_PRIMARY (as printed: “CLEARED”; verified by: UNKNOWN)
- Does not license
- “Synthetic data are never useful”; impossibility of future validated simulators
- Source (field 6, as printed)
- Bisbee et al. Political Analysis 2024; Xie et al. PNAS 2026; Wang et al. NMI 2025; Gong & Huang 2025; Ling & Cui 2025
- Source type (field 7)
- PEER_REVIEWED_JOURNAL_ARTICLE (as printed: “Multiple peer-reviewed studies”)
- Institutional position
- Independent academic
- Maturity (claim evidence / publication maturity)
- CONTESTED (NEGATIVE_EVIDENCE_MULTI_STUDY) (as printed: “CONTESTED, NEGATIVE EVIDENCE MULTI-STUDY”)
- Recorded date
- 2026-09-25
- Currentness / correction
- This is the current version. No correction or concern relation is recorded on its Sources.
Version
Pinned version v1.0 (entry LD07-HQ-C06--v1.0) — this page never floats to newer evidence.
This pinned version is also the current version.
Pinned Sources (field 6, version-specific)
- “Bisbee et al. Political Analysis 2024” → SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024--v1.0 — Bisbee, Clinton, Dorff, Kenkel & Larson, “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models,” Political Analysis 32(4):401–416 (2024); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
- “Xie et al. PNAS 2026” → SRC-LD07-XIE-SSDATABENCH-2026--v1.0 — Xie et al., “Evaluating the statistical realism of LLM-generated social science data,” PNAS 123(19):e2538145123 (2026); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
- “Wang et al. NMI 2025” → SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025--v1.0 — Wang et al. NMI 2025; class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
- “Gong & Huang 2025” → SRC-LD07-GONG-HUANG-CGSS-2025--v1.0 — Gong Weigang & Huang Siyuan, “整体事实还是有偏样本——基于大语言模型生成数据的测量,” Academic Monthly 57(6):123–136 (2025); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
- “Ling & Cui 2025” → SRC-LD07-LING-CUI-SILICON-SAMPLES-2025--v1.0 — Ling Wanying & Cui Sizhan, “复现的限度——‘硅基样本’的可能与可为,” Journal of Intelligent Society 4(2):179–201 (2025); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
Read the originals
- SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024: open the original
- SRC-LD07-XIE-SSDATABENCH-2026: open the original
- SRC-LD07-GONG-HUANG-CGSS-2025: open the original
All 25 canonical Claim fields (raw values and normalized values)
| # | Canonical field | Canonical value (verbatim) | Normalized (Claim Value Vocabulary v1.0.2) |
|---|---|---|---|
| 1 | CLAIM ID | LD07-HQ-C06 | — |
| 2 | PUBLIC-FACING CLAIM | Synthetic respondents can match selected survey averages while failing to reproduce real-world variance, regression relationships, individual structure and subgroup heterogeneity. | — |
| 3 | CLAIM TYPE | FACT / INTERPRETATION | base_types: FACT, INTERPRETATION |
| 4 | RESEARCH STRAND | C | — |
| 5 | MATURITY | CONTESTED, NEGATIVE EVIDENCE MULTI-STUDY | states: CONTESTED · qualifiers: NEGATIVE_EVIDENCE_MULTI_STUDY |
| 6 | SOURCE | Bisbee et al. Political Analysis 2024; Xie et al. PNAS 2026; Wang et al. NMI 2025; Gong & Huang 2025; Ling & Cui 2025 | source_ids: SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024, SRC-LD07-XIE-SSDATABENCH-2026, SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025, SRC-LD07-GONG-HUANG-CGSS-2025, SRC-LD07-LING-CUI-SILICON-SAMPLES-2025 |
| 7 | SOURCE TYPE | Multiple peer-reviewed studies | classes: PEER_REVIEWED_JOURNAL_ARTICLE |
| 8 | INSTITUTIONAL POSITION | Independent academic | — |
| 9 | SCOPE | ANES, multi-survey statistical benchmark, identity groups, CGSS | — |
| 10 | DESIGN | Human-vs-synthetic comparisons; statistical realism benchmarks | — |
| 11 | SAMPLE / DATASET | Multiple US/China/UK surveys and identity studies | — |
| 12 | MODEL / TOOL / VERSION | GPT-3.5/4 lineages plus multiple newer/open models | — |
| 13 | COMPARATOR | Human survey/population data | — |
| 14 | MAIN RESULT | Mean/topline match can coexist with compressed variance, altered coefficients, stereotyped/flattened groups and cross-model inconsistency | — |
| 15 | INFERENTIAL TARGET | Population representation and downstream associations | — |
| 16 | DOES NOT LICENSE | “Synthetic data are never useful”; impossibility of future validated simulators | — |
| 17 | VERIFICATION STATE | CLEARED | state: CONTENT_VERIFIED_PRIMARY · verified_by: UNKNOWN |
| 18 | LIMITATIONS | Designs and conditioning richness differ across studies | — |
| 19 | CONTRADICTORY / SUPPORTING SOURCES | Ashokkumar is positive on a different forecasting estimand | — |
| 20 | GEOGRAPHY / LANGUAGE | US, China, UK; English/Chinese | — |
| 21 | DISTRIBUTIONAL LEDGER | Marginal/identity groups particularly vulnerable to flattening/underrepresentation | — |
| 22 | R01 / FIELD-MAP LINKS | L-A, L-B, D9 | — |
| 23 | WHAT WOULD CHANGE THIS CLAIM? | Preregistered subgroup-valid synthetic population benchmark with probability-survey gold | — |
| 24 | PUBLICATION STATUS | ELIGIBLE | state: ELIGIBLE |
| 25 | RECORDED DATE | 2026-09-25 | — |
Field 22 references resolved against Field Map v0.2: L-A (LENS); L-B (LENS); D9 (DOMAIN)
Cited by Living Debate versions
Radar Updates referencing this ClaimVersion
- RADAR-LD07-PEW-SYNTHETIC-SURVEYS-2026: Pew Research Center (30 September 2026) built an AI “digital twin” for each panelist analyzed from three 2026 waves of its American Trends Panel, a nationally representative panel of randomly selected U.S. adults. Each twin was conditioned on that person’s profile and earlier survey answers, and weighted synthetic and human results were compared on nearly 300 questions. With its best-performing configuration (Claude Opus 4.6), synthetic estimates differed from human estimates by an average of 12.4 percentage points. Error was substantial for every subgroup reported, reaching 16.1 points for Republicans and Republican leaners and 15.1 points for Black adults. On 47% of questions at least one answer choice was selected by no synthetic respondent, which happened on no question among humans. Results also differed systematically by AI model, although some toplines matched closely. This strengthens the evidence for LD07-HQ-C06 on selected-average agreement, real-world response variance and subgroup heterogeneity. The report contains no regression or treatment-effect analysis and no individual-level accuracy result for its final configuration, so those parts of the claim are unchanged. The claim wording is unchanged.
Research-of-record provenance
Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 da2589d0c80046281ebe5265eb8fbdd4168d24a03cf90062e8887b9f908894b1.
Frozen Drive-derived inputs
- R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED (v1.0) — Drive ID
1xVz_LrQAA8EA1tPhVWIrDgOUU3Zidlg9; revision: NOT_EXPOSED_BY_DRIVE_TEXT_EXPORT; Drive modified 2026-09-29T04:31:14.120Z; block “ENTRY 06 — LD07-HQ-C06” SHA-2564bcc6149e6f274dab11470cb90500fc248e0b7858fd5016e2abb8fecf89815e1; SocioLens — Programme Status & Authority Register v0.6 status “CURRENT.”; frozen fixturefixtures/frozen/r03-ld07-claim-ledger-v1.0-canonical.
Builder decisions (not Drive data)
- VERSION = v1.0 — basis: Claim version taken from the canonical ledger artifact version (R03 — LD-07 Canonical Claim Ledger v1.0); Implementation Architecture Selection §5
- CURRENT_POINTER = true — basis: SocioLens — Programme Status & Authority Register v0.6 status for "R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED": "CURRENT."
- FIELD_6_SOURCE_REFERENCE = SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Bisbee et al. Political Analysis 2024"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §2; ledger C06; KEY EVIDENCE.
- FIELD_6_SOURCE_REFERENCE = SRC-LD07-XIE-SSDATABENCH-2026--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Xie et al. PNAS 2026"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §3; ledger C06; KEY EVIDENCE.
- FIELD_6_SOURCE_REFERENCE = SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Wang et al. NMI 2025"; the frozen field-6 segment of this ClaimVersion's ledger block; Central HQ ruling embedded in this issue, resolution ruling §1.
- FIELD_6_SOURCE_REFERENCE = SRC-LD07-GONG-HUANG-CGSS-2025--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Gong & Huang 2025"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §4; ledger C06/C08; KEY EVIDENCE.
- FIELD_6_SOURCE_REFERENCE = SRC-LD07-LING-CUI-SILICON-SAMPLES-2025--v1.0 — basis: field 6 of ENTRY 06 — LD07-HQ-C06 cites "Ling & Cui 2025"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §5 (distinct study); ledger C06/C08; KEY EVIDENCE.
- SOURCE_PROVENANCE_PIN = SRC-LD07-BISBEE-SYNTHETIC-REPLACEMENTS-2024--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Bisbee et al. Political Analysis 2024"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §2; ledger C06; KEY EVIDENCE.
- SOURCE_PROVENANCE_PIN = SRC-LD07-XIE-SSDATABENCH-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Xie et al. PNAS 2026"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §3; ledger C06; KEY EVIDENCE.
- SOURCE_PROVENANCE_PIN = SRC-R03-WANG-MORGENSTERN-DICKERSON-NMI-2025--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Wang et al. NMI 2025"; Central HQ ruling embedded in this issue, resolution ruling §1.
- SOURCE_PROVENANCE_PIN = SRC-LD07-GONG-HUANG-CGSS-2025--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Gong & Huang 2025"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §4; ledger C06/C08; KEY EVIDENCE.
- SOURCE_PROVENANCE_PIN = SRC-LD07-LING-CUI-SILICON-SAMPLES-2025--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Ling & Cui 2025"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §5 (distinct study); ledger C06/C08; KEY EVIDENCE.