Claim LD04-SL-C07 (v1.0)

In a 2025 randomized trial with 16 experienced open-source developers working on 246 tasks in their own projects, allowing AI tools increased completion time by 19%, although the developers estimated it had cut their time by 20%; METR later called its 2026 follow-up data an unreliable signal because of selection.

Research opportunities this claim motivates

What this claim leaves open for research. Each is a hypothesis about a gap, not a finding.

Does skill compression reverse when AI acts as an agent rather than an assistant?Highly experienced developers were slower with early-2025 AI tools while believing they were faster, and METR's later data were too selected to read: expert gains from newer tools are unmeasured.

Provenance and status

Claim type: ESTIMATE (as printed: “ESTIMATE”)
Publication status: ELIGIBLE (PREPRINT) (as printed: “ELIGIBLE WITH PREPRINT QUALIFIER”)
Verification state: CONTENT_VERIFIED_PRIMARY (PREPRINT) (as printed: “CONTENT VERIFIED PREPRINT”; verified by: UNKNOWN)
Does not license: A general claim that AI slows experts; any estimate for current tools
Source (field 6, as printed): Becker, Rush, Barnes & Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089, July 2025; METR, “We are Changing our Developer Productivity Experiment Design,” 24 February 2026
Source type (field 7): PREPRINT, INDEPENDENT_RESEARCH_REPORT (as printed: “Preprint + research nonprofit post”)
Institutional position: Independent research nonprofit
Maturity (claim evidence / publication maturity): CONTESTED (as printed: “CONTESTED”)
Recorded date: 2026-10-14
Currentness / correction: This is the current version. No correction or concern relation is recorded on its Sources.

Version

Current version v1.0, resolved through the explicit current pointer CLAIM--LD04-SL-C07.

Versions:v1.0 · current

Pinned Sources (field 6, version-specific)

“Becker, Rush, Barnes & Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089, July 2025” → SRC-SL-BECKER-RUSH-BARNES-REIN-ARXIV-2507-09089--v1.0 — Becker, Rush, Barnes & Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089, July 2025; class PREPRINT (explicit public provenance pin)
“METR, “We are Changing our Developer Productivity Experiment Design,” 24 February 2026” → SRC-SL-METR-DEVELOPER-STUDY-DESIGN-2026-02-24--v1.0 — METR, “We are Changing our Developer Productivity Experiment Design,” 24 February 2026; class INDEPENDENT_RESEARCH_REPORT (explicit public provenance pin)

All 25 canonical Claim fields (raw values and normalized values)

#	Canonical field	Canonical value (verbatim)	Normalized (Claim Value Vocabulary v1.0.2)
1	CLAIM ID	LD04-SL-C07	—
2	PUBLIC-FACING CLAIM	In a 2025 randomized trial with 16 experienced open-source developers working on 246 tasks in their own projects, allowing AI tools increased completion time by 19%, although the developers estimated it had cut their time by 20%; METR later called its 2026 follow-up data an unreliable signal because of selection.	—
3	CLAIM TYPE	ESTIMATE	base_types: ESTIMATE
4	RESEARCH STRAND	A — Performance while the tool is in use	—
5	MATURITY	CONTESTED	states: CONTESTED
6	SOURCE	Becker, Rush, Barnes & Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089, July 2025; METR, “We are Changing our Developer Productivity Experiment Design,” 24 February 2026	source_ids: SRC-SL-BECKER-RUSH-BARNES-REIN-ARXIV-2507-09089, SRC-SL-METR-DEVELOPER-STUDY-DESIGN-2026-02-24
7	SOURCE TYPE	Preprint + research nonprofit post	classes: PREPRINT, INDEPENDENT_RESEARCH_REPORT
8	INSTITUTIONAL POSITION	Independent research nonprofit	—
9	SCOPE	Mature open-source repositories; developers with about five years on their projects; early-2025 tools	—
10	DESIGN	Randomized controlled trial at task level	—
11	SAMPLE / DATASET	16 developers, 246 tasks	—
12	MODEL / TOOL / VERSION	Early-2025 AI coding tools	—
13	COMPARATOR	Tasks on which AI use was not allowed	—
14	MAIN RESULT	Completion time +19% (CI +2% to +39%) with AI allowed; developers estimated a 20% reduction; in 2026 METR described its new data as only very weak evidence because developers unwilling to work without AI opted out and others withheld tasks	—
15	INFERENTIAL TARGET	Assisted speed of highly experienced developers on familiar codebases	—
16	DOES NOT LICENSE	A general claim that AI slows experts; any estimate for current tools	—
17	VERIFICATION STATE	CONTENT VERIFIED PREPRINT	state: CONTENT_VERIFIED_PRIMARY · qualifiers: PREPRINT · verified_by: UNKNOWN
18	LIMITATIONS	Very small sample; one kind of setting; tools have changed; METR believes developers are now more sped up, and its own 2026 redesign post treats later estimates as unreliable	—
19	CONTRADICTORY / SUPPORTING SOURCES	Contrasts with large speed gains in lab coding tasks; consistent with meta-analytic evidence that controlled-experiment gains shrink in real settings	—
20	GEOGRAPHY / LANGUAGE	Not checked / English	—
21	DISTRIBUTIONAL LEDGER	The most experienced may gain least, or lose, on their own familiar work	—
22	R01 / FIELD-MAP LINKS	LD-04, D2	—
23	WHAT WOULD CHANGE THIS CLAIM?	A larger, well-selected trial with current tools	—
24	PUBLICATION STATUS	ELIGIBLE WITH PREPRINT QUALIFIER	state: ELIGIBLE · qualifiers: PREPRINT
25	RECORDED DATE	2026-10-14	—

Field 22 references resolved against Field Map v0.2: LD-04 (LIVING_DEBATE); D2 (DOMAIN)

Cited by Living Debate versions

LD-04 v0.1

Research-of-record provenance

Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 7abe93a263e9b6337e9e9e3bbcaac2c415822013d9dc45730a68188c5778d643.

Repository-authored inputs (not Drive exports)

SocioLens claim — LD04-SL-C07 v1.0 (v1.0) — written in the SocioLens repository by Project Lead (Claude, AI) on 2026-10-14; no Drive identity exists. Authority: AGENTS.md "Authority" (the Project Lead decides research content within the research-integrity rules) and the Owner's choice in the SocioLens project chat on 14 October 2026 to build the LD-04 debate; a new Claim, first version, written from the project's LD-04 source notes, checked against full texts or official abstracts. Block “ENTRY 07 — LD04-SL-C07” SHA-256 34bf84fff7760b7961fac666a0aaa1c78f2b796fe3190dd2b257ca057e636091; fixture fixtures/authored/claims/LD04-SL-C07--v1.0.

Builder decisions (not Drive data)

VERSION = v1.0 — basis: Claim version taken from the repository-authored claim fixture version (SocioLens claim — LD04-SL-C07 v1.0); the first version of a new Claim of the LD-04 debate
CURRENT_POINTER = true — basis: repository-authored claim "SocioLens claim — LD04-SL-C07 v1.0" (Project Lead (Claude, AI), 2026-10-14); basis: AGENTS.md "Authority" (the Project Lead decides research content within the research-integrity rules) and the Owner's choice in the SocioLens project chat on 14 October 2026 to build the LD-04 debate; a new Claim, first version, written from the project's LD-04 source notes, checked against full texts or official abstracts
FIELD_6_SOURCE_REFERENCE = SRC-SL-BECKER-RUSH-BARNES-REIN-ARXIV-2507-09089--v1.0 — basis: field 6 of ENTRY 07 — LD04-SL-C07 cites "Becker, Rush, Barnes & Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089, July 2025"; the first field-6 segment of this ClaimVersion's repository-authored ledger block.
FIELD_6_SOURCE_REFERENCE = SRC-SL-METR-DEVELOPER-STUDY-DESIGN-2026-02-24--v1.0 — basis: field 6 of ENTRY 07 — LD04-SL-C07 cites "METR, “We are Changing our Developer Productivity Experiment Design,” 24 February 2026"; the second field-6 segment of this ClaimVersion's repository-authored ledger block.
SOURCE_PROVENANCE_PIN = SRC-SL-BECKER-RUSH-BARNES-REIN-ARXIV-2507-09089--v1.0 — basis: explicit version-pinned public provenance of this ClaimVersion (Amendment v1.2 §6): the pinned field-6 Source record "Becker, Rush, Barnes & Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089, July 2025".
SOURCE_PROVENANCE_PIN = SRC-SL-METR-DEVELOPER-STUDY-DESIGN-2026-02-24--v1.0 — basis: explicit version-pinned public provenance of this ClaimVersion (Amendment v1.2 §6): the pinned field-6 Source record "METR, “We are Changing our Developer Productivity Experiment Design,” 24 February 2026".