Research opportunity · What kind of thing is generative AI, and does the answer predict anything?
Does the EU's training-compute criterion track how general a model is?
Question
Across open-weight and documented models released from 2020 to 2026, how closely does training compute predict measured generality (the range of distinct tasks performed competently), and how many models fall on the wrong side of the 10²³ FLOP indicative criterion that the European Commission uses for a general-purpose AI model?
Wrong or missing validation targetOne PhD chapter or paperNeeds a budget (medium or higher)No study found
Why it matters
The EU has turned "what kind of thing is it?" into law: a model is general-purpose if it displays significant generality, and the Commission's guidelines adopt an indicative criterion under which a model above 10²³ FLOP with generative ability likely is general-purpose, while calling compute an imperfect proxy. If the proxy misclassifies many models, the legal ontology diverges from the technical one, and obligations fall on the wrong models. No published study measuring this was found.
The debate lists this among what we still do not know: Does training compute track the generality that the EU's criterion is meant to capture?
Evidence that motivates it
- The European Commission’s guidelines note that the AI Act’s definition of a “general-purpose AI model” “does not set out specific criteria” and adopt an indicative criterion: training compute above 10²³ FLOP and the ability to generate language, images or video, while calling compute “an imperfect proxy for generality and capabilities.”
The Commission adopts 10²³ FLOP plus generative ability as an indicative criterion and calls compute an imperfect proxy. - Bommasani and colleagues named “foundation models” for their “critically central yet incomplete character,” argued that their significance lies in “emergence and homogenization,” warned that homogenization “creates single points of failure,” and predicted that the barrier to entry for developing them “will continue to rise.”
The foundation-model idea of a general base adapted downstream, which the legal category resembles.
A possible design
Assemble a dataset of models with documented or estimated training compute and run a fixed battery of distinct task families to measure generality, defining competence thresholds in advance. Estimate the relation between compute and generality, the rates of models above the threshold that lack generality and below it that have it, and how these rates change with time as training becomes more efficient. Compare alternative proxies such as benchmark breadth.
Data needed
Model inventories with training-compute estimates; open weights or API access; a task battery covering distinct task families.
Feasibility
One PhD chapter or paper. Medium; compute for evaluating open models and API fees for closed ones. Training compute is often estimated rather than disclosed; carry the uncertainty through. "Significant generality" has no legal operationalization, so the study should report results under several definitions.
Has this been done?
The search found no study that does the core design (first-pass screen of 16 October 2026; not yet checked in depth).
No published study was found that measures how well training compute predicts generality around the EU threshold. Search results were legal commentary describing the criterion as a presumption.
How strong this evidence is
- Who searched and checked: .
- What was read: none of the 1 nearest works below is recorded as judged from an abstract only. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by one web search and the LD-01 source scan on the date shown; policy-research papers on compute thresholds were not read. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- European Commission, guidelines on general-purpose AI models (LD01-SL-C36)
Full text read for this debate. Sets the criterion and its rationale; reports no measurement.
Search queries used
does training compute threshold 10^23 FLOP track model generality empirical evaluation general-purpose AI model definition AI ActLD-01 source scan for legal classification of general-purpose AI models (16 October 2026)training compute thresholds regulation proxy capabilities validation study
Provenance
RO-LD01-05 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-01 debate on 2026-10-16. Not yet reviewed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.