Claim LD07-HQ-C15 (v1.0)

Human–LLM agreement is not a sufficient decision rule for qualitative coding quality, and current evidence does not establish LLM replacement of researcher interpretation in inductive qualitative analysis.

Research opportunities this claim motivates

What this claim leaves open for research. Each is a hypothesis about a gap, not a finding.

How do research ethics boards judge synthetic participants and AI-interpreted data?Agreement metrics do not establish that interpretation can be delegated, which is exactly what boards must weigh.
Does "LLMs can replace coders" depend on which humans you compare with?Human agreement is not a sufficient quality rule, so the choice of human reference matters.

Provenance and status

Claim type: FACT + INTERPRETATION (as printed: “FACT + INTERPRETATION”)
Publication status: ELIGIBLE (QUALIFIED) (as printed: “ELIGIBLE WITH QUALIFIER”)
Verification state: CONTENT_VERIFIED_PRIMARY (PREPRINT, SOURCE_SPECIFIC) (as printed: “CLEARED WITH PREPRINT/DOMAIN QUALIFIER”; verified by: UNKNOWN)
Does not license: Claim LLMs are better interpreters; one expert as universal ground truth
Source (field 6, as printed): Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890; PLOS Digital Health 2026
Source type (field 7): PREPRINT, PEER_REVIEWED_JOURNAL_ARTICLE (as printed: “Preprint + peer-reviewed bounded case”)
Institutional position: Independent academic
Maturity (claim evidence / publication maturity): EMERGING (as printed: “EMERGING”)
Recorded date: 2026-09-25
Currentness / correction: This is the current version. No correction or concern relation is recorded on its Sources.

Version

Current version v1.0, resolved through the explicit current pointer CLAIM--LD07-HQ-C15.

Versions:v1.0 · current

Pinned Sources (field 6, version-specific)

“Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890” → SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026--v1.0 — Liu et al., “Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth,” arXiv:2607.28890; class PREPRINT (explicit public provenance pin)
“PLOS Digital Health 2026” → SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026--v1.0 — “Large language models for thematic analysis in healthcare research: A blinded mixed-methods comparison with human analysts,” PLOS Digital Health 2026; class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)

Read the originals

SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026: open the original
SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026: open the original

All 25 canonical Claim fields (raw values and normalized values)

#	Canonical field	Canonical value (verbatim)	Normalized (Claim Value Vocabulary v1.0.2)
1	CLAIM ID	LD07-HQ-C15	—
2	PUBLIC-FACING CLAIM	Human–LLM agreement is not a sufficient decision rule for qualitative coding quality, and current evidence does not establish LLM replacement of researcher interpretation in inductive qualitative analysis.	—
3	CLAIM TYPE	FACT + INTERPRETATION	base_types: FACT, INTERPRETATION
4	RESEARCH STRAND	E	—
5	MATURITY	EMERGING	states: EMERGING
6	SOURCE	Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890; PLOS Digital Health 2026	source_ids: SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026, SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026
7	SOURCE TYPE	Preprint + peer-reviewed bounded case	classes: PREPRINT, PEER_REVIEWED_JOURNAL_ARTICLE
8	INSTITUTIONAL POSITION	Independent academic	—
9	SCOPE	K-12 educator messages; healthcare thematic analysis	—
10	DESIGN	Blind independent expert verification + inductive comparison	—
11	SAMPLE / DATASET	Liu: 2,560 messages; 5 LLM systems; 3 humans; 855 blind pair comparisons	—
12	MODEL / TOOL / VERSION	Multiple LLM systems; PLOS contemporary frontier systems	—
13	COMPARATOR	Human coders and independent domain expert	—
14	MAIN RESULT	Human–LLM Jaccard .30 vs human–human .52, yet blind quality preference 51.5% human vs 48.5% LLM (p=.537); inductive coding in PLOS more variable than deductive	—
15	INFERENTIAL TARGET	Interpretive quality rather than mere agreement	—
16	DOES NOT LICENSE	Claim LLMs are better interpreters; one expert as universal ground truth	—
17	VERIFICATION STATE	CLEARED WITH PREPRINT/DOMAIN QUALIFIER	state: CONTENT_VERIFIED_PRIMARY · qualifiers: PREPRINT, SOURCE_SPECIFIC · verified_by: UNKNOWN
18	LIMITATIONS	Specific domains; Liu preprint; limited independent verifiers	—
19	CONTRADICTORY / SUPPORTING SOURCES	Than supports supervised augmentation	—
20	GEOGRAPHY / LANGUAGE	Primarily US/English	—
21	DISTRIBUTIONAL LEDGER	Shared human or model priors may both encode group-specific blind spots	—
22	R01 / FIELD-MAP LINKS	D3, D9, R-2	—
23	WHAT WOULD CHANGE THIS CLAIM?	Broad blinded expert replication of autonomous inductive analysis	—
24	PUBLICATION STATUS	ELIGIBLE WITH QUALIFIER	state: ELIGIBLE · qualifiers: QUALIFIED
25	RECORDED DATE	2026-09-25	—

Field 22 references resolved against Field Map v0.2: D3 (DOMAIN); D9 (DOMAIN); R-2 (LENS)

Cited by Living Debate versions

LD-07 v0.1

Research-of-record provenance

Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 86724b97891c856c9fdbc18a994d499abc8675306704dfc44e86b1e018d7e58a.

Frozen Drive-derived inputs

R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED (v1.0) — Drive ID 1xVz_LrQAA8EA1tPhVWIrDgOUU3Zidlg9; revision: NOT_EXPOSED_BY_DRIVE_TEXT_EXPORT; Drive modified 2026-09-29T04:31:14.120Z; block “ENTRY 15 — LD07-HQ-C15” SHA-256 f4c50936c08e550987f95bd60f2fb38437ae11b9d6bbcddc6a706a2e7060e831; SocioLens — Programme Status & Authority Register v0.6 status “CURRENT.”; frozen fixture fixtures/frozen/r03-ld07-claim-ledger-v1.0-canonical.

Builder decisions (not Drive data)

VERSION = v1.0 — basis: Claim version taken from the canonical ledger artifact version (R03 — LD-07 Canonical Claim Ledger v1.0); Implementation Architecture Selection §5
CURRENT_POINTER = true — basis: SocioLens — Programme Status & Authority Register v0.6 status for "R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED": "CURRENT."
FIELD_6_SOURCE_REFERENCE = SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026--v1.0 — basis: field 6 of ENTRY 15 — LD07-HQ-C15 cites "Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §9; ledger C15; KEY EVIDENCE.
FIELD_6_SOURCE_REFERENCE = SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026--v1.0 — basis: field 6 of ENTRY 15 — LD07-HQ-C15 cites "PLOS Digital Health 2026"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §8; ledger C14/C15; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890"; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §9; ledger C15; KEY EVIDENCE.
SOURCE_PROVENANCE_PIN = SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "PLOS Digital Health 2026"; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §8; ledger C14/C15; KEY EVIDENCE.