Research opportunity · Regulation-in-Practice: Rights, Compliance & Developmental Steering
Did signing the EU's voluntary AI code change what model providers actually do?
Question
Between before and after August 2025, did providers that signed the EU General-Purpose AI Code of Practice change their documented safety, transparency and copyright practices more than providers that signed only part of it or did not sign, when the same documents are scored with the same rubric over time?
Causal mechanism not isolatedOne PhD chapter or paperLow costPartly addressed
Why it matters
The Code is voluntary, but the Commission treats adherence as a route to show compliance. If signing is mostly a statement of existing practice, the Code adds compliance paperwork; if signatories change practice relative to others, the Code is doing regulatory work. The debate cannot currently tell these apart, and nobody can yet say whether either changes actual risk.
The debate lists this among what we still do not know: substantive risk reduction from voluntary-code adherence
Evidence that motivates it
- The EU GPAI Code of Practice is voluntary, but Commission guidance recognizes adherence as a route to demonstrate compliance; non-signatories must demonstrate compliance by other adequate means.
Adherence is recognized as a route to demonstrate compliance, but the claim explicitly does not show that signatories are substantively compliant or safer. - The EU AI Office began using information-request powers after its GPAI enforcement powers became applicable; the exact report that more than 30 providers received initial RFIs remains press-relayed.
The AI Office has begun sending information requests, so the regulator's own view of provider practice may later provide an outside check on document-based scores; the claim does not establish non-compliance by anyone.
A possible design
Assemble a panel of public documents (safety frameworks, model cards, system cards, training-data summaries, copyright policies) for 15–25 providers, collected from about January 2024 to the present using archived versions (Internet Archive) where needed. Group providers by status using the Commission's signatory list (22 full signatories when checked on 12 October 2026, last updated 7 October 2026): full signatories; a partial signatory (xAI, which the Commission lists as signing only the Safety and Security chapter); and non-signatories, meaning providers absent from the list, such as Meta, DeepSeek and Alibaba. Record each provider's signing date, because the list is updated as signatures are confirmed. Score each document version with a fixed rubric derived from the Code's measures, extending existing rubrics (Stelling et al.; Wang et al.; the Foundation Model Transparency Index), with two coders and reported agreement. Estimate a difference-in-differences comparison around July–August 2025, with an event-study check for changes that started before signing. Be explicit that the outcome is documented practice, not reduced harm: the study can show whether the Code changed what providers commit to and disclose, which is a precondition for, not evidence of, risk reduction.
Data needed
Public provider documents and their archived versions; the Code's final text and signatory list; a coding rubric; second coder time.
Feasibility
One PhD chapter or paper. Low: researcher and coder time only. Few providers and self-selection into signing limit causal claims; the event-study and a matched comparison on pre-period scores partly address this. Documents can be edited silently, so archive dates must be recorded.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026, checked again on 13 October 2026).
Existing studies score providers' public documents at one point in time (Wang et al. on the 2023 White House commitments; Stelling et al. 2026 on frontier safety frameworks; Blankvoort et al. 2026 on five training-data summaries) or map existing practice onto a draft of the Code (Stelling et al. 2025). The 2025 Foundation Model Transparency Index compares signatories with non-signatories cross-sectionally, finding that signatories score slightly higher, with 2025 evidence gathered between April and September 2025, around the time the Code was signed; it does not analyse change around signing dates. Zhu (2026) traces every public version of twelve developers' safety frameworks and compares revisions before and after California's TFAIA took effect (1 January 2026), but not by Code signatory status. No study was found that measures change in documented practice before and after signing against a comparison group of non-signatories. Whether any of this reduces risk remains unmeasured and this design would not settle it.
How strong this evidence is
- Who searched and checked: Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert; an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 2 of the 6 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search on the date shown; Google Scholar, SSRN and paywalled venues were only partly reachable, and civil-society trackers published in late 2026 may have been missed. The signatory grouping was checked at review against the Commission's own signatory page (last updated 7 October 2026). The Foundation Model Transparency Index comparison was found at review, not in the original screen. Absence of a hit is not proof that the work does not exist. Literature check of 2026-10-13: Civil-society trackers and law-firm reviews of Code compliance were not systematically read; the last part of the Zhu paper (about 10,000 characters, including the full developer list in an appendix) was not read. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Wang, Huang, Klyman & Bommasani, "Do AI Companies Make Good on Voluntary Commitments to the White House?," Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(3), 2625–2636 (2025); arXiv:2508.08345 Judged from an abstract, summary or search listing only
Scores 16 companies' public disclosures against the eight 2023 White House commitments (average 53%, weakest on model-weight security). Cross-sectional, US voluntary commitments, no comparison group of non-signers. Read from the arXiv abstract page; venue confirmed on the AIES proceedings page. - Stelling, Murray, Galizzi, Schaffelder, Campos & Papadatos, "Evaluating AI Providers' Frontier AI Safety Frameworks," arXiv:2512.01166v3 (revised 26 Mar 2026)
Scores 12 providers' safety frameworks on 65 criteria (median 18%). Measures published commitments, not outcomes, and notes that commitments may not reflect internal practice; makes no comparison of Code signatories with non-signatories; a footnote suggests aligning its criteria with the Code as future work. - Stelling, Yang, Gipiškis, Staufer, Chin, Campos, Gil & Chen, "Mapping Industry Practices to the EU AI Act's GPAI Code of Practice Safety and Security Measures," Oxford Martin AI Governance Initiative working paper (Jul 2025)
For the majority of the safety and security measures in the Code's third draft, finds relevant passages in the documents of at least five companies, which is evidence that signing may partly codify existing practice. Explicitly not an assessment of compliance and not a before-and-after comparison. - Wan, Klyman, Kapoor, Maslej, Longpre, Xiong, Liang & Bommasani, "The 2025 Foundation Model Transparency Index," arXiv:2512.10169 (Dec 2025)
Scores 13 developers on transparency and reports that Code of Practice signatories score slightly higher than non-signatories, mostly on downstream transparency; six companies have been scored in all three editions (2023–2025). The 2025 evidence was gathered from April to September 2025, so it cannot separate signing effects from earlier differences; no before-and-after analysis around signing. Added at review. - Zhu, "Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers," arXiv:2609.08789 (submitted 8 Sep 2026)
Builds a corpus of every public version of the safety frameworks of the twelve developers that have published one, traces 710 commitment instances across twelve consecutive version pairs, and reports that 67% of material changes are not disclosed by the developer (strict reading) and 77% weaken or remove a commitment. Its before-and-after split is around 1 January 2026, when California's TFAIA took effect, not around Code signing; it mentions the Code's framework-update duty but does not compare signatories with non-signatories. Longitudinal and close in method; does not do the proposed design. Opened on the arXiv abstract and HTML pages (first 100,000 characters). - Blankvoort, Pandit & Gahntz, "Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)," arXiv:2603.13270 (Feb 2026); DOI 10.1145/3805689.3806755 Judged from an abstract, summary or search listing only
Proposes a quality-assessment framework and applies it to the 5 public training-data summaries found as of 12 January 2026. No comparison by signatory status and no change over time. Relevant to scoring the training-data part of the proposed rubric. Opened on the arXiv abstract page; the DOI was seen in search results only.
Literature check
Stands as first published. No study was found that compares change in documented practice before and after signing the GPAI Code of Practice against non-signatories. The closest new work, Zhu's "Silent Revision" (arXiv:2609.08789, Sep 2026), traces every public version of twelve developers' safety frameworks and splits version pairs around 1 January 2026 (California TFAIA), but does not compare by Code signatory status or around August 2025. Blankvoort et al. (2026) assess only five public training-data summaries with no signatory comparison. The design stands, and Zhu's corpus is a usable data source and an outcome measure.
Searched: web (extended); arxiv.org; ssrn.com; osf.io; dl.acm.org; cambridge.org; academic.oup.com; tandfonline.com; sciencedirect.com; ojs.aaai.org; link.springer.com. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert.
The screen before this check
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026).
Existing studies score providers' public documents at one point in time (Wang et al. on the 2023 White House commitments; Stelling et al. 2026 on frontier safety frameworks) or map existing practice onto a draft of the Code (Stelling et al. 2025). The 2025 Foundation Model Transparency Index does compare signatories with non-signatories, finding that signatories score slightly higher, and has scored six companies in each of 2023, 2024 and 2025; but that comparison is cross-sectional, its 2025 evidence was gathered between April and September 2025, around the time the Code was signed, and it does not analyse change around signing dates. No study was found that measures change in documented practice before and after signing against a comparison group of non-signatories. Whether any of this reduces risk remains unmeasured and this design would not settle it.
Search queries used
GPAI Code of Practice signatories versus non-signatories comparison safety documentation transparency study 2026Do AI companies make good on voluntary commitments White House evaluation study arXivFrontier AI Safety Commitments Seoul compliance assessment companies safety frameworks published evaluationempirical analysis Safety and Security chapter Code of Practice signatories model documentation published model cards compliance before after August 2025
Literature check, 13 October 2026:
General-Purpose AI Code of Practice signatories effect change in model documentation before after signing empirical [arxiv.org, ssrn.com, papers.ssrn.com, osf.io]GPAI Code of Practice compliance assessment model cards system cards signatories 2026 report evaluation transparency safety chapterlongitudinal analysis frontier AI developers safety frameworks revisions over time voluntary commitments change"Foundation Model Transparency Index" 2026 OR "Code of Practice" signatories transparency change developers difference-in-differencessoft law voluntary code AI providers signatories self-selection effect disclosure regulation empirical general-purpose AI [dl.acm.org, cambridge.org, academic.oup.com, tandfonline.com, sciencedirect.com, ojs.aaai.org, link.springer.com]"Do AI Companies Make Good on Voluntary Commitments" follow-up 2026 tracking commitments over timeanalysis of published training data summaries AI Act Article 53 template providers compliance study copyright policies GPAI 2026
Provenance
RO-LD12-03 v2. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-12 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.
Changes
- v2, 13 October 2026: Literature check added: a second, deeper search for existing work, with its queries, new nearest work and limits. The question and verdict stand. See v1 as published.