Claim LD07-HQ-C15 (v1.0)
Human–LLM agreement is not a sufficient decision rule for qualitative coding quality, and current evidence does not establish LLM replacement of researcher interpretation in inductive qualitative analysis.
Provenance and status
- Claim type
- FACT + INTERPRETATION (as printed: “FACT + INTERPRETATION”)
- Publication status
- ELIGIBLE (QUALIFIED) (as printed: “ELIGIBLE WITH QUALIFIER”)
- Verification state
- CONTENT_VERIFIED_PRIMARY (PREPRINT, SOURCE_SPECIFIC) (as printed: “CLEARED WITH PREPRINT/DOMAIN QUALIFIER”; verified by: UNKNOWN)
- Does not license
- Claim LLMs are better interpreters; one expert as universal ground truth
- Source (field 6, as printed)
- Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890; PLOS Digital Health 2026
- Source type (field 7)
- PREPRINT, PEER_REVIEWED_JOURNAL_ARTICLE (as printed: “Preprint + peer-reviewed bounded case”)
- Institutional position
- Independent academic
- Maturity (claim evidence / publication maturity)
- EMERGING (as printed: “EMERGING”)
- Recorded date
- 2026-09-25
- Currentness / correction
- This is the current version. No correction or concern relation is recorded on its Sources.
Version
Pinned version v1.0 (entry LD07-HQ-C15--v1.0) — this page never floats to newer evidence.
This pinned version is also the current version.
Pinned Sources (field 6, version-specific)
- “Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890” → SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026--v1.0 — Liu et al., “Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth,” arXiv:2607.28890; class PREPRINT (explicit public provenance pin)
- “PLOS Digital Health 2026” → SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026--v1.0 — “Large language models for thematic analysis in healthcare research: A blinded mixed-methods comparison with human analysts,” PLOS Digital Health 2026; class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)
Read the originals
- SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026: open the original
- SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026: open the original
All 25 canonical Claim fields (raw values and normalized values)
| # | Canonical field | Canonical value (verbatim) | Normalized (Claim Value Vocabulary v1.0.2) |
|---|---|---|---|
| 1 | CLAIM ID | LD07-HQ-C15 | — |
| 2 | PUBLIC-FACING CLAIM | Human–LLM agreement is not a sufficient decision rule for qualitative coding quality, and current evidence does not establish LLM replacement of researcher interpretation in inductive qualitative analysis. | — |
| 3 | CLAIM TYPE | FACT + INTERPRETATION | base_types: FACT, INTERPRETATION |
| 4 | RESEARCH STRAND | E | — |
| 5 | MATURITY | EMERGING | states: EMERGING |
| 6 | SOURCE | Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890; PLOS Digital Health 2026 | source_ids: SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026, SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026 |
| 7 | SOURCE TYPE | Preprint + peer-reviewed bounded case | classes: PREPRINT, PEER_REVIEWED_JOURNAL_ARTICLE |
| 8 | INSTITUTIONAL POSITION | Independent academic | — |
| 9 | SCOPE | K-12 educator messages; healthcare thematic analysis | — |
| 10 | DESIGN | Blind independent expert verification + inductive comparison | — |
| 11 | SAMPLE / DATASET | Liu: 2,560 messages; 5 LLM systems; 3 humans; 855 blind pair comparisons | — |
| 12 | MODEL / TOOL / VERSION | Multiple LLM systems; PLOS contemporary frontier systems | — |
| 13 | COMPARATOR | Human coders and independent domain expert | — |
| 14 | MAIN RESULT | Human–LLM Jaccard .30 vs human–human .52, yet blind quality preference 51.5% human vs 48.5% LLM (p=.537); inductive coding in PLOS more variable than deductive | — |
| 15 | INFERENTIAL TARGET | Interpretive quality rather than mere agreement | — |
| 16 | DOES NOT LICENSE | Claim LLMs are better interpreters; one expert as universal ground truth | — |
| 17 | VERIFICATION STATE | CLEARED WITH PREPRINT/DOMAIN QUALIFIER | state: CONTENT_VERIFIED_PRIMARY · qualifiers: PREPRINT, SOURCE_SPECIFIC · verified_by: UNKNOWN |
| 18 | LIMITATIONS | Specific domains; Liu preprint; limited independent verifiers | — |
| 19 | CONTRADICTORY / SUPPORTING SOURCES | Than supports supervised augmentation | — |
| 20 | GEOGRAPHY / LANGUAGE | Primarily US/English | — |
| 21 | DISTRIBUTIONAL LEDGER | Shared human or model priors may both encode group-specific blind spots | — |
| 22 | R01 / FIELD-MAP LINKS | D3, D9, R-2 | — |
| 23 | WHAT WOULD CHANGE THIS CLAIM? | Broad blinded expert replication of autonomous inductive analysis | — |
| 24 | PUBLICATION STATUS | ELIGIBLE WITH QUALIFIER | state: ELIGIBLE · qualifiers: QUALIFIED |
| 25 | RECORDED DATE | 2026-09-25 | — |
Field 22 references resolved against Field Map v0.2: D3 (DOMAIN); D9 (DOMAIN); R-2 (LENS)
Cited by Living Debate versions
Research-of-record provenance
Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 86724b97891c856c9fdbc18a994d499abc8675306704dfc44e86b1e018d7e58a.
Frozen Drive-derived inputs
- R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED (v1.0) — Drive ID
1xVz_LrQAA8EA1tPhVWIrDgOUU3Zidlg9; revision: NOT_EXPOSED_BY_DRIVE_TEXT_EXPORT; Drive modified 2026-09-29T04:31:14.120Z; block “ENTRY 15 — LD07-HQ-C15” SHA-256f4c50936c08e550987f95bd60f2fb38437ae11b9d6bbcddc6a706a2e7060e831; SocioLens — Programme Status & Authority Register v0.6 status “CURRENT.”; frozen fixturefixtures/frozen/r03-ld07-claim-ledger-v1.0-canonical.
Builder decisions (not Drive data)
- VERSION = v1.0 — basis: Claim version taken from the canonical ledger artifact version (R03 — LD-07 Canonical Claim Ledger v1.0); Implementation Architecture Selection §5
- CURRENT_POINTER = true — basis: SocioLens — Programme Status & Authority Register v0.6 status for "R03 — LD-07 Canonical Claim Ledger v1.0 — LOCKED": "CURRENT."
- FIELD_6_SOURCE_REFERENCE = SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026--v1.0 — basis: field 6 of ENTRY 15 — LD07-HQ-C15 cites "Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §9; ledger C15; KEY EVIDENCE.
- FIELD_6_SOURCE_REFERENCE = SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026--v1.0 — basis: field 6 of ENTRY 15 — LD07-HQ-C15 cites "PLOS Digital Health 2026"; the frozen field-6 segment of this ClaimVersion's ledger block; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §8; ledger C14/C15; KEY EVIDENCE.
- SOURCE_PROVENANCE_PIN = SRC-LD07-LIU-AGREEMENT-IS-NOT-QUALITY-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "Liu et al., “Agreement Is Not Quality,” arXiv:2607.28890"; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §9; ledger C15; KEY EVIDENCE.
- SOURCE_PROVENANCE_PIN = SRC-LD07-PLOS-THEMATIC-ANALYSIS-2026--v1.0 — basis: Issue #5 proof rendering (Architecture Proof Protocol v0.1 AP-08 / Amendment v1.2 §6): explicit version-pinned public provenance of this ClaimVersion — the pinned field-6 Source record "PLOS Digital Health 2026"; Drive-derived identity authority (Issue #39 Source inventory): Batch 04 §8; ledger C14/C15; KEY EVIDENCE.