Research opportunity · Levelling or hollowing? Skill compression and the reproduction of expertise
Do professionals trained with AI lose the skill without it a year later?
Question
Among people learning a professional or technical skill (clinical reasoning, programming, legal drafting), are those randomly assigned to learn with generative AI less able to do the work without it 6 to 12 months later than comparable learners trained without it, measured against an unassisted baseline test?
Not yet measuredA PhD projectLow to medium costPartly addressed
Why it matters
The hollowing worry is "never-skilling", that novices who learn with AI never build the skill AI does for them. The randomized evidence stops at a single session or three months, and the only professional trial has no baseline skill test. Longer-horizon evidence comes from school pupils and is observational. Without a randomized professional test over a year, training programmes cannot tell whether AI-assisted training is a shortcut or a debt.
The debate lists this among what we still do not know: Do people who learn a task with AI become less able to do it without AI over months or years ("never-skilling")?
Evidence that motivates it
- In a three-month randomized trial with 133 patent lawyers, AI assistance raised graded drafting quality by 0.38 standard deviations at 90 days; on a later task done without AI, treated seniors scored 0.45 standard deviations higher than control seniors, treated juniors no higher (−0.03), and the senior–junior difference was statistically significant.
After three months of AI-assisted patent drafting, seniors improved on an unaided task and juniors did not; the study has no baseline skill test and stops at three months. - In a field experiment with nearly 1,000 Turkish high-school students, practising with a standard GPT-4 interface raised practice scores by 48% but left students 17% worse than controls on a closed-book exam later in the same session, while a tutor version with safeguards raised practice by 127% and left exam scores indistinguishable from controls.
Students who practised with a standard GPT-4 interface did 17% worse on a closed-book exam in the same session, while a tutor version with safeguards caused no detectable loss; the test was short-term only. - In a randomized experiment with 52 Python programmers learning an unfamiliar library, those given an AI assistant scored 17% lower on a later quiz of concepts, code reading and debugging, with no significant speed gain; participants who delegated code to the AI scored lowest.
Programmers learning a new library with AI scored 17% lower on a quiz straight afterwards, and those who delegated code scored lowest; one session only. - In an online experiment with 1,174 Argentine adults, AI access shrank the performance gap between education groups on a business problem-solving task from 0.548 to 0.139 standard deviations; in a follow-up module without AI in the same session, a gap of 0.200 standard deviations remained.
A module done without AI immediately after an AI-assisted task showed no loss, and the authors call it a diagnostic rather than a test of skill formation.
A possible design
Randomized controlled trial inside an existing training programme, for example a university programming course, a law-school drafting clinic or a residency simulation curriculum. Give every participant an unassisted baseline skill test, then randomize cohorts or sections to three arms: no AI, unrestricted AI, and AI with learning guardrails (hints and explanations rather than answers). Test without AI at the end of training and again at 6 and 12 months, using tasks graded blind by assessors who do not know the arm. Record how AI was used (logs where consented) so that delegation and explanation-seeking can be compared.
Data needed
Access to a training cohort large enough to randomize by section; validated unassisted skill tests with parallel forms; consented AI-use logs; retention of participants over a year (incentives for follow-up tests).
Feasibility
A PhD project. Low to medium; the main costs are participant incentives for follow-up tests and grader time; the AI tools themselves are cheap. Randomizing AI access within a course raises fairness concerns, and one semester-long trial had to merge its arms after student protests, so randomizing by cohort or section, with the guardrailed arm as an acceptable alternative, is safer. Contamination (control learners using AI privately) must be measured, not assumed away.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 14 October 2026; not yet checked in depth).
No randomized or matched study was found that assigns people learning a professional or technical skill to learn with or without generative AI, gives a baseline unassisted test, and repeats unassisted tests at 6 to 12 months or later. Randomized delayed tests found stop at 45 days (students who studied with ChatGPT scored lower on a surprise test) and one week. The longest professional field trial, in patent drafting, runs three months with no baseline test. Longer-horizon evidence is observational and about school pupils: a reported 30-month panel of Chinese secondary students and a ten-year US maths-platform panel. A medical-education perspective states that direct evidence from medical training is absent.
How strong this evidence is
- Who searched and checked: an independent AI reviewer with fresh context (not a person), 14 October 2026 (review of PR #72): verdicts, nearest-work reading labels and limits checked; query lists corrected to the 8 queries run where they differed. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 6 of the 6 nearest works below were judged from an abstract, summary or search listing only, and 1 has details that could not be confirmed; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search (8 queries) and arXiv abstract pages on the date shown; Google Scholar, ERIC, PubMed, Scopus and medical-education journals were not searched directly. Two nearest works were seen only through listings and press coverage. A small trial with medical students and a one-semester observational programming-course study appeared in listings and were not opened. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Strömberg, Lei & Wu, "The Generative AI Learning Penalty: Evidence from Chinese Secondary Education," CEPR Discussion Paper DP21577 (2026) Judged from an abstract, summary or search listing onlySome details not confirmed
Judged from search listings and press summaries; not read directly (the CEPR page could not be opened). Reportedly a 30-month panel of 26,811 secondary students in which AI adoption raises homework scores but lowers closed-book exam scores by about 20% within six months and entrance-exam scores by 18–24% after about two years. Observational and about school pupils, not professional training; it is the closest long-horizon evidence. - Rismanchian, Uzun, Matayoshi, Cosyn & Kurd-Misto, "Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build," arXiv:2605.21629 (2026) Judged from an abstract, summary or search listing only
Abstract read. Ten-year panel of an online maths platform: after ChatGPT, time on problems susceptible to AI falls and the odds of answering proctored retention items correctly fall 25%. Population-level before-and-after comparison; individuals are not assigned to learn with or without AI. - Barcaui, "ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention," Social Sciences & Humanities Open 12, 102287 (2025) Judged from an abstract, summary or search listing only
Judged from search listings and press coverage; not read directly, and the DOI was not confirmed. A randomized trial with 120 business undergraduates in which those who studied with ChatGPT scored lower on a surprise test 45 days later. One study session and no professional skill; the longest randomized delayed test found. - Contractor & Reyes, "Experimental Evidence on the Learning Impact of Generative AI," arXiv:2607.08849 (2026) Judged from an abstract, summary or search listing only
Abstract read. Proctored randomized experiment with undergraduates: AI raises scores, a smaller gain persists in an unaided test a week later, and gains remain for tutor-style use and fade when AI writes for the student. One week only. - Benedek & Sziklai, "Impact of AI Tools on Learning Outcomes: Decreasing Knowledge and Over-Reliance," arXiv:2510.16019 (2025) Judged from an abstract, summary or search listing only
Abstract read. A semester-long course experiment in which AI was allowed or barred; the arms were merged before the final exam after student protests, so there is no clean unassisted end-of-term comparison. Shows the practical difficulty this design must solve. - Fan, "Toward Measuring AI's Effects on Skill Formation: The Stock-Formation Gap," arXiv:2605.16283 (2026) Judged from an abstract, summary or search listing only
Abstract read. Conceptual synthesis proposing to link consented usage records to independent assessments; states that existing measurement cannot yet tell whether AI erodes skill formation. No data of its own.
Search queries used
longitudinal study students learned with generative AI unassisted test months later skill retention semesterrandomized semester-long programming course ChatGPT access final exam without AI learning outcomes 2025medical residents trainees AI assistance deskilling never-skilling longitudinal unassisted performance study 2026randomized experiment ChatGPT learning "delayed post-test" OR "delayed retention test" weeks later without AICheatGPT how using AI for programming homework influences exam results first-semester 800 students26,000 Chinese secondary students AI homework tools monthly exam scores fell paper 2026Barcaui "ChatGPT as a cognitive crutch" randomized controlled trial knowledge retention 45 days Social Sciences & Humanities OpenCEPR DP21577 Strömberg Lei Wu generative AI students homework exam title
Provenance
RO-LD04-01 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-04 debate on 2026-10-14. Checked by an independent AI reviewer with fresh context (not a person), 14 October 2026 (review of PR #72): verdicts, nearest-work reading labels and limits checked; query lists corrected to the 8 queries run where they differed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.