Research opportunity · Organizational absorption: Why micro gains do not become macro outcomes

How much of the time AI saves its users do colleagues spend checking and fixing?

Question

In organizations where some staff produce work with generative AI, how much time do reviewers, managers and recipients spend checking, correcting and redoing that work, and once both are counted, does the downstream cost offset the time the author saved?

Not yet measuredOne PhD chapter or paperLow to medium costPartly addressed

Why it matters

The time a tool saves its user is measured; the time it costs everyone downstream mostly is not. In open-source projects that adopted an AI editor, complexity and warnings rose and stayed high and slowed later work; METR's maintainers could merge none of the agent's pull requests as they were; German organizations reported that validation effort offset part of the gain. If the downstream cost is large, the overstated-micro-gains position (P3) explains much of the gap; if it is small, it does not.

The debate lists this among what we still do not know: How much of the measured gain is spent on checking and repairing AI output elsewhere in the organization?

Evidence that motivates it

In 806 open-source projects that adopted the Cursor AI editor, compared with 1,380 matched projects, development velocity rose sharply but only for one or two months, while static-analysis warnings rose 30% and code complexity 42% and stayed higher; the added complexity and warnings in turn slowed later development.
After adopting an AI editor, open-source projects saw a short burst of velocity, then persistently higher warnings and complexity that slowed later development.
On 18 real tasks from two open-source repositories, an AI agent's code passed the maintainers' tests 38% of the time, but none of the 15 pull requests METR reviewed by hand could be merged as is; METR roughly estimated 26 minutes of human repair even for those that passed, about a third of the original human time.
None of the 15 agent pull requests METR reviewed by hand was mergeable as is, with repair estimated at about a third of the original human time even for those that passed the tests.
In a case study of three German organizations, generative AI was used mainly as a personal assistant rather than changing team processes, the effort of validating its output partly offset efficiency gains, and policies written in response to regulation without regard to actual use led to gaps between policy and practice and to shadow IT.
In three German organizations the effort of validating generative-AI output partly offset the efficiency gains.

A possible design

Two linked studies. (1) In an organization with a recorded review workflow (software, legal drafting, consulting or policy units), link authors' AI-use logs to reviewers' time and to rework (review threads, tracked changes, reopened tickets, revision rounds), using a staggered rollout as a difference-in-differences or randomizing AI access at the author level; report the author's time saved and the reviewers' added time on the same items. (2) Outside software, a blind experiment in which managers review drafts produced with or without AI on the same brief and record time to acceptance and the edits they make.

Data needed

Access to review and ticketing systems with author identifiers and timestamps; consented AI-use logs; for the experiment, matched briefs and a pool of reviewers.

Feasibility

One PhD chapter or paper. Low for the blind-review experiment; medium for an in-firm study that needs data engineering on review systems. Software review systems already record most of what is needed, so a first study there is the cheapest. The harder and more novel part is documents and analyses outside software, where review time is rarely logged.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).

Industrial software work has begun to measure the reviewer's side: one study of 3.52 million code changes at a large company reports increased review effort for AI-generated code, and a multi-firm working paper reportedly finds pull-request review time rising after adoption of coding agents. No study was found that charges reviewers' added time against authors' time saved on the same work to give a net organizational effect, and none that measures downstream checking and rework of AI-drafted documents, emails or analyses outside software in real organizations; the 'workslop' evidence there is a self-reported survey of recipients.

How strong this evidence is

Who searched and checked: an independent AI reviewer with fresh context (not a person), 15 October 2026 (review of PR #77): verdicts, query counts and 13 abstract-read labels checked; label slips corrected. Every check so far was done by AI; no person or domain expert has checked it yet.
What was read: 6 of the 6 nearest works below were judged from an abstract, summary or search listing only, and 1 has details that could not be confirmed; each is marked. Not every entry says how much of the work was read.
Where the search could not reach: Screened by web search (7 queries), two arXiv API searches and arXiv and NBER abstract pages on the date shown; ICSE, FSE and MSR 2026 proceedings and SSRN were not searched directly, so industrial case studies may be missing. The clinical-documentation literature, where clinicians' rework of AI scribe notes is studied, was not searched. Absence of a hit is not proof that the work does not exist.
Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

Tran et al., "Characterizing the Quality Profile of AI-Generated C++ in Production," arXiv:2608.06640 (2026) Judged from an abstract, summary or search listing only
Abstract read. 3.52 million code changes over a year at a large enterprise that the abstract does not identify: AI-generated C++ carries interface and coupling burdens and leads to increased review effort and more compute. The strongest in-firm evidence on review burden found; it does not net review time against authors' time saved.
Chen & Stratton, "Artificial Intelligence in the Firm: Bottlenecks in Software Production," Harvard working paper (August 2026) Judged from an abstract, summary or search listing only
Seen only in secondary press coverage; the paper itself was not read and no primary page was found. Reportedly an engineering-analytics panel of about 718 firms with staggered difference-in-differences: more code per worker, no significant rise in finished tasks and longer pull-request review times. If confirmed, the closest match; the figures need checking against the paper.
Demirer, Musolff & Yang, "Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools," NBER Working Paper 35275 (2026) Judged from an abstract, summary or search listing only
Abstract read. Over 500,000 GitHub developers with AI-use telemetry: commits rise much more than releases across tool generations, which the authors read as human bottlenecks. Evidence of a downstream bottleneck, without a direct measure of reviewers' time.
BetterUp Labs and Stanford Social Media Lab, "workslop" survey, Harvard Business Review (September 2025; authors not confirmed) Judged from an abstract, summary or search listing onlySome details not confirmed
Judged from search listings and a news report; not read directly. Reportedly a survey of 1,150 US workers on AI-generated work they received and the time it took to deal with it. Recipient-side, but self-reported and not peer-reviewed.
Gao, Banyongrakkul, Guan, Zahedi & Treude, "On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source," arXiv:2601.13754 (2026) Judged from an abstract, summary or search listing only
Abstract read. In open source, AI co-authored pull requests are merged faster with little feedback, and most from non-owners are merged without explicit review. Suggests review is skipped and the cost deferred, rather than paid; open source, not firms.
Zhong, Noei, Adams & Zou, "From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality," arXiv:2607.13196 (2026) Judged from an abstract, summary or search listing only
Abstract read. 1.02 million pull requests from 207 GitHub projects: AI agents in review speed decisions without improving review quality. Concerns AI as reviewer more than the burden on human reviewers.

Search queries used

AI coding assistant adoption code review time firm pull request review burden empirical study
Chen Stratton AI coding agents pull request merge time review comments firms working paper
"workslop" study AI-generated work burden coworkers measurement
rework AI-generated code company study code churn reviewer effort industrial case study LLM-generated code maintenance burden
arXiv AI-generated C++ code changes Google review effort blocking review threads compute 3.52 million
"Artificial Intelligence in the Firm: Bottlenecks in Software Production" Chen Stratton
experiment recipients evaluate AI-drafted documents time spent verifying editing managers burden generative AI organization
arXiv API: all:"AI-generated" AND all:C++ AND all:review AND all:compute
arXiv API: abs:"AI-generated" AND abs:code AND abs:"review effort" AND abs:industrial

Provenance

RO-LD03-03 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-03 debate on 2026-10-15. Checked by an independent AI reviewer with fresh context (not a person), 15 October 2026 (review of PR #77): verdicts, query counts and 13 abstract-read labels checked; label slips corrected. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.

All research opportunities