Preview: not announced. Search engines asked not to index.

Proposal pack · RO-LD03-03 v1

How much of the time AI saves its users do colleagues spend checking and fixing?

To keep this as a PDF or print it, use your browser's Print command (Ctrl+P, or Cmd+P on a Mac) and choose "Save as PDF". Links print with their addresses. This pack gathers what the opportunity page already holds; nothing in it is new.

Read this first. This is a research hypothesis drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-03 debate on 15 October 2026, not a finding. Checked by an independent AI reviewer with fresh context (not a person), 15 October 2026 (review of PR #77): verdicts, query counts and 13 abstract-read labels checked; label slips corrected. Every check so far was done by AI; no person or domain expert has checked it yet. The literature screen below was done by AI web search and can miss relevant work. Version v1, permanent address https://sociolens-web.pages.dev/opportunities/ro-ld03-03/v1/.

Not yet measured One PhD chapter or paper Low to medium cost Partly addressed

Research question

In organizations where some staff produce work with generative AI, how much time do reviewers, managers and recipients spend checking, correcting and redoing that work, and once both are counted, does the downstream cost offset the time the author saved?

Why it matters

The time a tool saves its user is measured; the time it costs everyone downstream mostly is not. In open-source projects that adopted an AI editor, complexity and warnings rose and stayed high and slowed later work; METR's maintainers could merge none of the agent's pull requests as they were; German organizations reported that validation effort offset part of the gain. If the downstream cost is large, the overstated-micro-gains position (P3) explains much of the gap; if it is small, it does not.

The debate lists this among what we still do not know: How much of the measured gain is spent on checking and repairing AI output elsewhere in the organization?

Evidence that motivates it

  1. In 806 open-source projects that adopted the Cursor AI editor, compared with 1,380 matched projects, development velocity rose sharply but only for one or two months, while static-analysis warnings rose 30% and code complexity 42% and stayed higher; the added complexity and warnings in turn slowed later development. (LD03-SL-C11 v1.0)
    After adopting an AI editor, open-source projects saw a short burst of velocity, then persistently higher warnings and complexity that slowed later development.
    Cites: He et al. (2026), “Speed at the Cost of Quality: How…”
  2. On 18 real tasks from two open-source repositories, an AI agent's code passed the maintainers' tests 38% of the time, but none of the 15 pull requests METR reviewed by hand could be merged as is; METR roughly estimated 26 minutes of human repair even for those that passed, about a third of the original human time. (LD03-SL-C12 v1.0)
    None of the 15 agent pull requests METR reviewed by hand was mergeable as is, with repair estimated at about a third of the original human time even for those that passed the tests.
    Cites: Rein (2025), “Research Update: Algorithmic vs. Holistic…”
  3. In a case study of three German organizations, generative AI was used mainly as a personal assistant rather than changing team processes, the effort of validating its output partly offset efficiency gains, and policies written in response to regulation without regard to actual use led to gaps between policy and practice and to shadow IT. (LD03-SL-C08 v1.0)
    In three German organizations the effort of validating generative-AI output partly offset the efficiency gains.
    Cites: Neumann et al. (2026), “Between Policy and Practice…”

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Two linked studies. (1) In an organization with a recorded review workflow (software, legal drafting, consulting or policy units), link authors' AI-use logs to reviewers' time and to rework (review threads, tracked changes, reopened tickets, revision rounds), using a staggered rollout as a difference-in-differences or randomizing AI access at the author level; report the author's time saved and the reviewers' added time on the same items. (2) Outside software, a blind experiment in which managers review drafts produced with or without AI on the same brief and record time to acceptance and the edits they make.

Data needed

Access to review and ticketing systems with author identifiers and timestamps; consented AI-use logs; for the experiment, matched briefs and a pool of reviewers.

Feasibility

One PhD chapter or paper. Low for the blind-review experiment; medium for an in-firm study that needs data engineering on review systems. Software review systems already record most of what is needed, so a first study there is the cheapest. The harder and more novel part is documents and analyses outside software, where review time is rarely logged.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).

Industrial software work has begun to measure the reviewer's side: one study of 3.52 million code changes at a large company reports increased review effort for AI-generated code, and a multi-firm working paper reportedly finds pull-request review time rising after adoption of coding agents. No study was found that charges reviewers' added time against authors' time saved on the same work to give a net organizational effect, and none that measures downstream checking and rework of AI-drafted documents, emails or analyses outside software in real organizations; the 'workslop' evidence there is a self-reported survey of recipients.

Where the search could not reach. Screened by web search (7 queries), two arXiv API searches and arXiv and NBER abstract pages on the date shown; ICSE, FSE and MSR 2026 proceedings and SSRN were not searched directly, so industrial case studies may be missing. The clinical-documentation literature, where clinicians' rework of AI scribe notes is studied, was not searched. Absence of a hit is not proof that the work does not exist.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

References

Sources behind the motivating claims:

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld03_03_v1,
  author = {{SocioLens project}},
  title = {How much of the time AI saves its users do colleagues spend checking and fixing?},
  year = {2026},
  date = {2026-10-15},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld03-03/v1/}},
  note = {Research opportunity RO-LD03-03, version v1}
}

@misc{ro_ld03_03_nearest_1,
  title = {Tran et al., "Characterizing the Quality Profile of AI-Generated C++ in Production," arXiv:2608.06640 (2026)},
  url = {https://arxiv.org/abs/2608.06640},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_03_nearest_2,
  title = {Chen \& Stratton, "Artificial Intelligence in the Firm: Bottlenecks in Software Production," Harvard working paper (August 2026)},
  url = {https://fourweekmba.com/ai-coding-agents-add-30-more-code-not-more-finished-work/},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_03_nearest_3,
  title = {Demirer, Musolff \& Yang, "Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools," NBER Working Paper 35275 (2026)},
  url = {https://www.nber.org/papers/w35275},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_03_nearest_4,
  title = {BetterUp Labs and Stanford Social Media Lab, "workslop" survey, Harvard Business Review (September 2025; authors not confirmed)},
  url = {https://the-decoder.com/ai-generated-workslop-is-costing-companies-millions-and-hurting-team-morale-study-finds/},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only; Some details not confirmed}
}

@misc{ro_ld03_03_nearest_5,
  title = {Gao, Banyongrakkul, Guan, Zahedi \& Treude, "On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source," arXiv:2601.13754 (2026)},
  url = {https://arxiv.org/abs/2601.13754},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_03_nearest_6,
  title = {Zhong, Noei, Adams \& Zou, "From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality," arXiv:2607.13196 (2026)},
  url = {https://arxiv.org/abs/2607.13196},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{src_sl_he_miller_agarwal_kastner_vasilescu_msr_2026,
  title = {He, Miller, Agarwal, Kästner \& Vasilescu, “Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects,” Proceedings of MSR 2026},
  url = {https://doi.org/10.1145/3793302.3793349},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_rein_metr_algorithmic_holistic_2025_08_13,
  title = {Rein, “Research Update: Algorithmic vs. Holistic Evaluation,” METR, 13 August 2025},
  url = {https://metr.org/blog/2025-08-12-research-update-towards-reconciling-slowdown-with-time-horizons/},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_neumann_et_al_xp_2026,
  title = {Neumann, Bischof, Hinz et al., “Between Policy and Practice: GenAI Adoption in Agile Software Development Teams,” XP 2026, LNBIP},
  url = {https://doi.org/10.1007/978-3-032-22375-3_18},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

Back to the opportunity · All research opportunities