Research opportunity · Regulation-in-Practice: Rights, Compliance & Developmental Steering
Did signing the EU's voluntary AI code change what model providers actually do?
Earlier version (v1)
This is RO-LD12-03 as it was published at v1, kept unchanged. The current version is v2; its changes are listed there.
Question
Between before and after August 2025, did providers that signed the EU General-Purpose AI Code of Practice change their documented safety, transparency and copyright practices more than providers that signed only part of it or did not sign, when the same documents are scored with the same rubric over time?
Causal mechanism not isolatedOne PhD chapter or paperLow costPartly addressed
Why it matters
The Code is voluntary, but the Commission treats adherence as a route to show compliance. If signing is mostly a statement of existing practice, the Code adds compliance paperwork; if signatories change practice relative to others, the Code is doing regulatory work. The debate cannot currently tell these apart, and nobody can yet say whether either changes actual risk.
The debate lists this among what we still do not know: substantive risk reduction from voluntary-code adherence
Evidence that motivates it
- The EU GPAI Code of Practice is voluntary, but Commission guidance recognizes adherence as a route to demonstrate compliance; non-signatories must demonstrate compliance by other adequate means.
Adherence is recognized as a route to demonstrate compliance, but the claim explicitly does not show that signatories are substantively compliant or safer. - The EU AI Office began using information-request powers after its GPAI enforcement powers became applicable; the exact report that more than 30 providers received initial RFIs remains press-relayed.
The AI Office has begun sending information requests, so the regulator's own view of provider practice may later provide an outside check on document-based scores; the claim does not establish non-compliance by anyone.
A possible design
Assemble a panel of public documents (safety frameworks, model cards, system cards, training-data summaries, copyright policies) for 15–25 providers, collected from about January 2024 to the present using archived versions (Internet Archive) where needed. Group providers by status using the Commission's signatory list (22 full signatories when checked on 12 October 2026, last updated 7 October 2026): full signatories; a partial signatory (xAI, which the Commission lists as signing only the Safety and Security chapter); and non-signatories, meaning providers absent from the list, such as Meta, DeepSeek and Alibaba. Record each provider's signing date, because the list is updated as signatures are confirmed. Score each document version with a fixed rubric derived from the Code's measures, extending existing rubrics (Stelling et al.; Wang et al.; the Foundation Model Transparency Index), with two coders and reported agreement. Estimate a difference-in-differences comparison around July–August 2025, with an event-study check for changes that started before signing. Be explicit that the outcome is documented practice, not reduced harm: the study can show whether the Code changed what providers commit to and disclose, which is a precondition for, not evidence of, risk reduction.
Data needed
Public provider documents and their archived versions; the Code's final text and signatory list; a coding rubric; second coder time.
Feasibility
One PhD chapter or paper. Low: researcher and coder time only. Few providers and self-selection into signing limit causal claims; the event-study and a matched comparison on pre-period scores partly address this. Documents can be edited silently, so archive dates must be recorded.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026; not yet checked in depth).
Existing studies score providers' public documents at one point in time (Wang et al. on the 2023 White House commitments; Stelling et al. 2026 on frontier safety frameworks) or map existing practice onto a draft of the Code (Stelling et al. 2025). The 2025 Foundation Model Transparency Index does compare signatories with non-signatories, finding that signatories score slightly higher, and has scored six companies in each of 2023, 2024 and 2025; but that comparison is cross-sectional, its 2025 evidence was gathered between April and September 2025, around the time the Code was signed, and it does not analyse change around signing dates. No study was found that measures change in documented practice before and after signing against a comparison group of non-signatories. Whether any of this reduces risk remains unmeasured and this design would not settle it.
How strong this evidence is
- Who searched and checked: an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 1 of the 4 nearest works below was judged from an abstract, summary or search listing only; it is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search on the date shown; Google Scholar, SSRN and paywalled venues were only partly reachable, and civil-society trackers published in late 2026 may have been missed. The signatory grouping was checked at review against the Commission's own signatory page (last updated 7 October 2026). The Foundation Model Transparency Index comparison was found at review, not in the original screen. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Wang, Huang, Klyman & Bommasani, "Do AI Companies Make Good on Voluntary Commitments to the White House?," Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(3), 2625–2636 (2025); arXiv:2508.08345 Judged from an abstract, summary or search listing only
Scores 16 companies' public disclosures against the eight 2023 White House commitments (average 53%, weakest on model-weight security). Cross-sectional, US voluntary commitments, no comparison group of non-signers. Read from the arXiv abstract page; venue confirmed on the AIES proceedings page. - Stelling, Murray, Galizzi, Schaffelder, Campos & Papadatos, "Evaluating AI Providers' Frontier AI Safety Frameworks," arXiv:2512.01166v3 (revised 26 Mar 2026)
Scores 12 providers' safety frameworks on 65 criteria (median 18%). Measures published commitments, not outcomes, and notes that commitments may not reflect internal practice; makes no comparison of Code signatories with non-signatories; a footnote suggests aligning its criteria with the Code as future work. - Stelling, Yang, Gipiškis, Staufer, Chin, Campos, Gil & Chen, "Mapping Industry Practices to the EU AI Act's GPAI Code of Practice Safety and Security Measures," Oxford Martin AI Governance Initiative working paper (Jul 2025)
For the majority of the safety and security measures in the Code's third draft, finds relevant passages in the documents of at least five companies, which is evidence that signing may partly codify existing practice. Explicitly not an assessment of compliance and not a before-and-after comparison. - Wan, Klyman, Kapoor, Maslej, Longpre, Xiong, Liang & Bommasani, "The 2025 Foundation Model Transparency Index," arXiv:2512.10169 (Dec 2025)
Scores 13 developers on transparency and reports that Code of Practice signatories score slightly higher than non-signatories, mostly on downstream transparency; six companies have been scored in all three editions (2023–2025). The 2025 evidence was gathered from April to September 2025, so it cannot separate signing effects from earlier differences; no before-and-after analysis around signing. Added at review.
Search queries used
GPAI Code of Practice signatories versus non-signatories comparison safety documentation transparency study 2026Do AI companies make good on voluntary commitments White House evaluation study arXivFrontier AI Safety Commitments Seoul compliance assessment companies safety frameworks published evaluationempirical analysis Safety and Security chapter Code of Practice signatories model documentation published model cards compliance before after August 2025
Provenance
RO-LD12-03 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-12 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.