Research opportunity · Regulation-in-Practice: Rights, Compliance & Developmental Steering
Do AI hiring-audit rules cost small employers more than large ones?
Earlier version (v1)
This is RO-LD12-06 as it was published at v1, kept unchanged. The current version is v2; its changes are listed there.
Question
Under New York City's bias-audit law for automated hiring tools, how do the costs and forms of compliance (commissioning an audit, relying on a vendor-supplied audit, narrowing the tool's use to fall outside the law, or doing nothing visible) differ by employer size, and do these differences favour large employers and large tool vendors?
Not yet measuredMaster's thesisLow costPartly addressed
Why it matters
Whether AI rules entrench incumbents is a recurring argument for and against regulation, but current figures are either modelled estimates or surveys commissioned by interested parties. New York City's law has been in effect since 2023, so it offers a rare case where real compliance choices can be observed and related to firm size, which may inform other jurisdictions considering audit requirements.
The debate lists this among what we still do not know: robust SME compliance costs or incumbent advantage
Evidence that motivates it
- In an audit-style study of 391 NYC employers after Local Law 144 took effect, 18 posted bias-audit reports and 13 posted transparency notices; employer scope discretion means those counts are not a violation rate.
Only 18 of 391 employers studied posted audit reports and 13 posted notices, and because employers decide whether their tool is covered these counts are not a violation rate; how this discretion is used across firm sizes is exactly what is unknown. - The New York State Comptroller concluded NYC's enforcement of Local Law 144 was ineffective; 75% of test calls were misdirected, and DCWP identified one issue where auditors identified at least 17 potential issues in the same 32-company review.
The state Comptroller found the city's enforcement ineffective, so compliance choices are being made with little enforcement pressure, which the design must take into account; the claim says nothing about other regulators.
A possible design
Three components. (1) A size-stratified sample of 400–600 NYC employers that recruit online, drawn from job-board postings and linked to employer size from business registers or disclosures; record whether each posts an audit or notice, whether the audit is employer-specific or supplied by the tool vendor, and which auditor signed it. (2) Interviews or a short survey of 20–30 auditors and HR-technology vendors on pricing by client size and on vendor-level audits that spread fixed costs across clients; ask auditors openly for price ranges rather than posing as clients. (3) A survey of HR managers in small and large employers (via professional associations) on how they decided whether the law applies to them and what compliance cost in money and staff time. Relate visible compliance, audit form and stated cost to firm size, and compare with the hiring-cost effects reported in difference-in-differences work.
Data needed
Job-board postings and employer websites; firm-size data (business registers, LinkedIn company pages, public filings); auditor and vendor interviews; an HR-manager survey; ethics approval.
Feasibility
Master's thesis. Low: researcher time and small survey incentives. Use of automated hiring tools is not observed directly, so non-posting cannot be read as non-compliance (as the original audit study warned); the analysis should compare forms of visible compliance, not violation rates. HR-survey response rates will be low; report them.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026; not yet checked in depth).
The law's visible compliance has been documented (Wright et al.), auditors have been interviewed about how the regime works (Groves et al.), and a difference-in-differences study presented in June 2026 reports hiring effects and higher HR staffing at affected firms (Ma). EU-focused cost figures are modelled (Intellera) or come from an industry survey. No study was found that relates compliance costs or forms of compliance to employer size, or tests for an advantage to large employers or large vendors.
How strong this evidence is
- Who searched and checked: an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 2 of the 4 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search on the date shown; the full Ma paper was not found, SSRN and Google Scholar were only partly reachable, and law-review and HR-industry literature was not searched systematically. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Wright, Muenster, Vecchione, Qu, Cai, COMM/INFO 2450 Student Investigators, Metcalf & Matias, "Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability," Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, 1701–1713 (Jun 2024); arXiv:2406.01399 Judged from an abstract, summary or search listing only
The 391-employer audit behind the debate's claim; documents posting rates and employer discretion but does not analyse compliance by firm size or measure costs. Read from the abstract page; venue confirmed with Crossref. - Groves, Metcalf, Kennedy, Vecchione & Strait, "Auditing Work: Exploring the New York City algorithmic bias audit regime," arXiv:2402.08101 (Feb 2024)
Interviews with 16 experts and practitioners on how audits are done and who does them, including auditors' wider consultancy work, which two participants said was more lucrative than the audit itself. Reports no audit prices and no analysis of costs by firm size. - Ma (Stockholm School of Economics), "Auditing Effects on Employment Hiring: Evidence from the New York City Algorithmic Bias Audit Law," seminar, Renmin University of China Business School (26 Jun 2026) Judged from an abstract, summary or search listing only
Difference-in-differences study: audit requirements reduced the share of male hires by almost two percentage points (no significant effect on white hiring), and affected firms hired more HR staff and took longer to fill vacancies. Only a seminar abstract was seen; no paper text, and no breakdown by firm size is mentioned. - Pellegrino, Mancini, Gerardi & Mumeni Urbani, "The AI Act: help or hindrance for SMEs? An analysis of the cost of compliance with the AI Act for SMEs," Intellera Consulting (c. late 2022)
Modelled compliance costs for one representative 150-employee firm under the draft AI Act (from about 1% to 17% of revenue depending on the scenario). A theoretical model checked with a few SMEs; the authors say further research is needed to validate it empirically.
Search queries used
NYC Local Law 144 bias audit employers compliance study "Null compliance" Wright audit-style 391 employersAuditing work Exploring the New York City algorithmic bias audit regime Groves FAccT 2024 auditors interviewsAI regulation compliance costs small and medium enterprises empirical survey AI Act SMEs cost estimate firm size incumbent advantage studybias audit cost price Local Law 144 auditors fees vendor-level audit employers firm size analysisMa "Auditing Effects on Employment Hiring" New York City bias audit law SSRN working paper
Provenance
RO-LD12-06 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-12 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.