Claim LD07-HQ-C05 (v1.0)

LLM simulations can usefully forecast the relative pattern of treatment effects in some US survey-experiment archives, while systematically overstating effect magnitudes.

Research opportunities this claim motivates

What this claim leaves open for research. Each is a hypothesis about a gap, not a finding.

Can synthetic respondents recover strong treatment-effect heterogeneity?Forecasting the pattern of average effects works in some US archives but overstates magnitudes.
Do silicon samples work outside the North Atlantic, and in which language?Forecasting success comes from US archives; the claim would change with failure on non-US archives.

Provenance and status

Claim type: FACT (as printed: “FACT”)
Publication status: ELIGIBLE (ESTIMAND) (as printed: “ELIGIBLE WITH ESTIMAND QUALIFIER”)
Verification state: CONTENT_VERIFIED_PRIMARY (SOURCE_SPECIFIC) (as printed: “CLEARED WITH QUALIFIER”; verified by: UNKNOWN)
Does not license: Synthetic population substitution; unbiased effect sizes; mechanism recovery; subgroup validity; non-US transport
Source (field 6, as printed): Ashokkumar, Hewitt, Ghezae & Willer, Nature 656:115–122 (2026), doi:10.1038/s41586-026-10742-x
Source type (field 7): PEER_REVIEWED_JOURNAL_ARTICLE (as printed: “Peer-reviewed journal article”)
Institutional position: Independent academic; one author also industry-affiliated
Maturity (claim evidence / publication maturity): CONTESTED (POSITIVE_BOUNDED_USE) (as printed: “CONTESTED / POSITIVE BOUNDED USE”)
Recorded date: 2026-09-25
Currentness / correction: This is the current version. No correction or concern relation is recorded on its Sources.

Version

Current version v1.0, resolved through the explicit current pointer CLAIM--LD07-HQ-C05.

Versions:v1.0 · current

Pinned Sources (field 6, version-specific)

“Ashokkumar, Hewitt, Ghezae & Willer, Nature 656:115–122 (2026), doi:10.1038/s41586-026-10742-x” → SRC-LD07-ASHOKKUMAR-NATURE-2026--v1.0 — Ashokkumar, Hewitt, Ghezae & Willer, “Large language models can predict the results of social science experiments,” Nature 656:115–122 (2026); class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)

Read the originals

SRC-LD07-ASHOKKUMAR-NATURE-2026: open the original

All 25 canonical Claim fields (raw values and normalized values)

#	Canonical field	Canonical value (verbatim)	Normalized (Claim Value Vocabulary v1.0.2)
1	CLAIM ID	LD07-HQ-C05	—
2	PUBLIC-FACING CLAIM	LLM simulations can usefully forecast the relative pattern of treatment effects in some US survey-experiment archives, while systematically overstating effect magnitudes.	—
3	CLAIM TYPE	FACT	base_types: FACT
4	RESEARCH STRAND	C — Synthetic respondents/agents	—
5	MATURITY	CONTESTED / POSITIVE BOUNDED USE	states: CONTESTED · qualifiers: POSITIVE_BOUNDED_USE
6	SOURCE	Ashokkumar, Hewitt, Ghezae & Willer, Nature 656:115–122 (2026), doi:10.1038/s41586-026-10742-x	source_ids: SRC-LD07-ASHOKKUMAR-NATURE-2026
7	SOURCE TYPE	Peer-reviewed journal article	classes: PEER_REVIEWED_JOURNAL_ARTICLE
8	INSTITUTIONAL POSITION	Independent academic; one author also industry-affiliated	—
9	SCOPE	US nationally representative text-based survey experiments	—
10	DESIGN	Forecast/model simulation benchmark against realized experimental effects and human forecasts	—
11	SAMPLE / DATASET	70 experiments; 469 effects; 119,330 participants; 15 secondary megastudies/606 effects	—
12	MODEL / TOOL / VERSION	GPT-4 primary plus prominent open-weight models	—
13	COMPARATOR	Realized human treatment effects; pooled human/expert forecasts	—
14	MAIN RESULT	Strong effect-pattern correlation / human-forecast-comparable accuracy; systematic magnitude overestimation	—
15	INFERENTIAL TARGET	Aggregate effect forecasting / replication triage	—
16	DOES NOT LICENSE	Synthetic population substitution; unbiased effect sizes; mechanism recovery; subgroup validity; non-US transport	—
17	VERIFICATION STATE	CLEARED WITH QUALIFIER	state: CONTENT_VERIFIED_PRIMARY · qualifiers: SOURCE_SPECIFIC · verified_by: UNKNOWN
18	LIMITATIONS	US survey-experiment archive; exact public r=.85 attribution to VOR remains HOLD	—
19	CONTRADICTORY / SUPPORTING SOURCES	Li & Ji shows realism ≠ ATE estimation; Cui/Li/Zhou shows null→significant inflation	—
20	GEOGRAPHY / LANGUAGE	US / English	—
21	DISTRIBUTIONAL LEDGER	Subgroup performance unresolved	—
22	R01 / FIELD-MAP LINKS	D9, R-1, T1	—
23	WHAT WOULD CHANGE THIS CLAIM?	Failure on independent non-US experimental archives	—
24	PUBLICATION STATUS	ELIGIBLE WITH ESTIMAND QUALIFIER	state: ELIGIBLE · qualifiers: ESTIMAND
25	RECORDED DATE	2026-09-25	—

Field 22 references resolved against Field Map v0.2: D9 (DOMAIN); R-1 (LENS); T1 (TENSION)

Cited by Living Debate versions

LD-07 v0.1

Research-of-record provenance

Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 5e1c81162b8ef25d5bed6a5db6d84d8c41dee1ed576f8c4e5ba1c563cab9bf63.

Frozen Drive-derived inputs

R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED (v1.0) — Drive ID 1xVz_LrQAA8EA1tPhVWIrDgOUU3Zidlg9; revision: NOT_EXPOSED_BY_DRIVE_TEXT_EXPORT; Drive modified 2026-09-29T04:31:14.120Z; block “ENTRY 05 — LD07-HQ-C05” SHA-256 1bde3e034b0890c118453eca52b1332ac22679f04989cf11b3a217a52a4a8f8c; SocioLens — Programme Status & Authority Register v0.6 status “CURRENT.”; frozen fixture fixtures/frozen/r03-ld07-claim-ledger-v1.0-canonical.

Builder decisions (not Drive data)

VERSION = v1.0 — basis: Claim version taken from the canonical ledger artifact version (R03 — LD-07 Canonical Claim Ledger v1.0); Implementation Architecture Selection §5
CURRENT_POINTER = true — basis: SocioLens — Programme Status & Authority Register v0.6 status for "R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED": "CURRENT."
FIELD_17_CONTEXT_QUALIFIERS = SOURCE_SPECIFIC — basis: R03 — HQ Cross-Batch Synthesis & Adjudication (v0.1, Drive 1V-HOVaPFwF8BTf60-LGPsJXf59V3kRlQ)
FIELD_6_SOURCE_REFERENCE = SRC-LD07-ASHOKKUMAR-NATURE-2026--v1.0 — basis: field 6 of ENTRY 05 — LD07-HQ-C05 cites "Ashokkumar, Hewitt, Ghezae & Willer, Nature 656:115–122 (2026), doi:10.1038/s41586-026-10742-x"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §1; ledger C05 fields 6/7/8; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-LD07-ASHOKKUMAR-NATURE-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Ashokkumar, Hewitt, Ghezae & Willer, Nature 656:115–122 (2026), doi:10.1038/s41586-026-10742-x"; Drive-derived identity authority (Issue #39 Source inventory): Batch 02 §1; ledger C05 fields 6/7/8; KEY EVIDENCE.