Research opportunity · Regulation-in-Practice: Rights, Compliance & Developmental Steering
Do AI hiring-audit rules cost small employers more than large ones?
Question
Under New York City's bias-audit law for automated hiring tools, how do the costs and forms of compliance (commissioning an audit, relying on a vendor-supplied audit, narrowing the tool's use to fall outside the law, or doing nothing visible) differ by employer size, and do these differences favour large employers and large tool vendors?
Not yet measuredMaster's thesisLow costPartly addressed
Why it matters
Whether AI rules entrench incumbents is a recurring argument for and against regulation, but current figures are either modelled estimates or surveys commissioned by interested parties. New York City's law has been in effect since 2023, so it offers a rare case where real compliance choices can be observed and related to firm size, which may inform other jurisdictions considering audit requirements.
The debate lists this among what we still do not know: robust SME compliance costs or incumbent advantage
Evidence that motivates it
- In an audit-style study of 391 NYC employers after Local Law 144 took effect, 18 posted bias-audit reports and 13 posted transparency notices; employer scope discretion means those counts are not a violation rate.
Only 18 of 391 employers studied posted audit reports and 13 posted notices, and because employers decide whether their tool is covered these counts are not a violation rate; how this discretion is used across firm sizes is exactly what is unknown. - The New York State Comptroller concluded NYC's enforcement of Local Law 144 was ineffective; 75% of test calls were misdirected, and DCWP identified one issue where auditors identified at least 17 potential issues in the same 32-company review.
The state Comptroller found the city's enforcement ineffective, so compliance choices are being made with little enforcement pressure, which the design must take into account; the claim says nothing about other regulators.
A possible design
Three components. (1) A size-stratified sample of 400–600 NYC employers that recruit online, drawn from job-board postings and linked to employer size from business registers or disclosures; record whether each posts an audit or notice, whether the audit is employer-specific or supplied by the tool vendor, and which auditor signed it. (2) Interviews or a short survey of 20–30 auditors and HR-technology vendors on pricing by client size and on vendor-level audits that spread fixed costs across clients; ask auditors openly for price ranges rather than posing as clients. (3) A survey of HR managers in small and large employers (via professional associations) on how they decided whether the law applies to them and what compliance cost in money and staff time. Relate visible compliance, audit form and stated cost to firm size, and compare with the hiring-cost effects reported in difference-in-differences work.
Data needed
Job-board postings and employer websites; firm-size data (business registers, LinkedIn company pages, public filings); auditor and vendor interviews; an HR-manager survey; ethics approval.
Feasibility
Master's thesis. Low: researcher time and small survey incentives. Use of automated hiring tools is not observed directly, so non-posting cannot be read as non-compliance (as the original audit study warned); the analysis should compare forms of visible compliance, not violation rates. HR-survey response rates will be low; report them.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026, checked again on 13 October 2026).
The law's visible compliance has been documented (Wright et al.; about 5% posting in Jansen 2026), auditors have been interviewed about how the regime works (Groves et al.), and the published audits themselves have been analysed, showing vendor-level duplicate audits and a few dominant auditors (Gerchick et al., FAccT 2025). Difference-in-differences studies report hiring effects and higher HR staffing at affected firms (Aobdia, Ma & Zhang) or null employment effects attributed to non-compliance (Jansen). EU-focused cost figures are modelled (Intellera) or come from an industry survey. No study was found that relates compliance costs or forms of compliance to employer size, or tests for an advantage to large employers or large vendors.
How strong this evidence is
- Who searched and checked: Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert; an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 2 of the 7 nearest works below were judged from an abstract, summary or search listing only, and 1 has details that could not be confirmed; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search on the date shown; the full Ma paper was not found, SSRN and Google Scholar were only partly reachable, and law-review and HR-industry literature was not searched systematically. Absence of a hit is not proof that the work does not exist. Literature check of 2026-10-13: The SSRN page and PDF of Aobdia, Ma & Zhang could not be opened, so any firm-size heterogeneity in it is unchecked and should be verified before relying on this outcome. HR-industry pricing sources (auditor price lists) were seen only as vendor marketing pages and not used. A Taylor & Francis article on bias audit laws (doi:10.1080/13600869.2024.2403053) was seen only as a listing. Google Scholar was not searched. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Wright, Muenster, Vecchione, Qu, Cai, COMM/INFO 2450 Student Investigators, Metcalf & Matias, "Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability," Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, 1701–1713 (Jun 2024); arXiv:2406.01399 Judged from an abstract, summary or search listing only
The 391-employer audit behind the debate's claim; documents posting rates and employer discretion but does not analyse compliance by firm size or measure costs. Read from the abstract page; venue confirmed with Crossref. - Groves, Metcalf, Kennedy, Vecchione & Strait, "Auditing Work: Exploring the New York City algorithmic bias audit regime," arXiv:2402.08101 (Feb 2024); Proceedings of FAccT 2024, doi:10.1145/3630106.3658959
Interviews with 16 experts and practitioners on how audits are done and who does them, including auditors' wider consultancy work, which two participants said was more lucrative than the audit itself. Reports no audit prices and no analysis of costs by firm size. - Aobdia, Ma & Zhang (presented by Ma, Stockholm School of Economics), "Auditing Effects on Employment Hiring: Evidence from the New York City Algorithmic Bias Audit Law," seminar, Renmin University of China Business School (26 Jun 2026) Judged from an abstract, summary or search listing only
Difference-in-differences study: audit requirements reduced the share of male hires by almost two percentage points (no significant effect on white hiring), and affected firms hired more HR staff and took longer to fill vacancies. Only a seminar abstract was seen; no paper text, and no breakdown by firm size is mentioned. - Pellegrino, Mancini, Gerardi & Mumeni Urbani, "The AI Act: help or hindrance for SMEs? An analysis of the cost of compliance with the AI Act for SMEs," Intellera Consulting (c. late 2022)
Modelled compliance costs for one representative 150-employee firm under the draft AI Act (from about 1% to 17% of revenue depending on the scenario). A theoretical model checked with a few SMEs; the authors say further research is needed to validate it empirically. - Gerchick, Encarnación, Tanigawa-Lau, Armstrong, Gutiérrez & Metaxa, "Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits," FAccT 2025
Collects 116 bias audits in 44 reports (July 2023 to Nov 2024) covering 31 employers, 22 tool vendors and 11 auditors: pervasive missing demographic data, 53% with an impact ratio below 0.8, reliance on test or pooled data, and 'silent duplicate' results across employers' audits of the same vendor tools, mostly from one auditor. Partly touches vendor-level audits and auditor shares, but does not analyse employer size or audit costs. Read from the FAccT archival PDF. - Jansen, "The Employment Effects of AI Hiring Transparency: Evidence from New York City's Local Law 144," SSRN 6465063 (posted 9 Apr 2026; UC Davis undergraduate honors thesis)
Triple difference-in-differences with New Jersey as comparison: no significant effect on employment or racial employment and wage ratios, attributed to near-universal non-compliance (about 5% of employers posted audits; two complaints in two years). No firm-size, cost or audit-form analysis. - Aobdia, Ma & Zhang, "Auditing Effects on Employment Hiring: Evidence from the New York City Algorithmic Bias Audit Law," SSRN 5703842 Some details not confirmed
The full paper behind the 'Ma' seminar entry, identified from SSRN search listings (title, authors, identifier); also presented at Shanghai Jiao Tong University (seminar dated 22 Dec 2025) and Renmin (Jun 2026). The SSRN page and PDF could not be opened, so whether it reports heterogeneity by firm size or compliance costs is unconfirmed.
Literature check
Stands as first published. Seven new queries found more work on Local Law 144 but none that relates compliance costs or forms of compliance to employer size. Gerchick et al. (FAccT 2025) analyse 116 published bias audits from 31 employers, 22 vendors and 11 auditors, documenting vendor-level duplicate audits and auditor shares, but do not analyse employer size or prices; Jansen (SSRN, Apr 2026) finds null employment effects and attributes them to about 5% posting compliance; and the Ma seminar paper is on SSRN as Aobdia, Ma & Zhang, though its text could not be opened. The firm-size, audit-pricing and large-vendor-advantage questions remain unstudied.
Searched: web (extended and standard); arxiv.org; ssrn.com; papers.ssrn.com; osf.io; nber.org; dl.acm.org; facctconference.org; law-review domains (NYU, Harvard, Columbia, Yale); journals.sagepub.com; academic.oup.com; sciencedirect.com; tandfonline.com; cambridge.org. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert.
The screen before this check
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026).
The law's visible compliance has been documented (Wright et al.), auditors have been interviewed about how the regime works (Groves et al.), and a difference-in-differences study presented in June 2026 reports hiring effects and higher HR staffing at affected firms (Ma). EU-focused cost figures are modelled (Intellera) or come from an industry survey. No study was found that relates compliance costs or forms of compliance to employer size, or tests for an advantage to large employers or large vendors.
Search queries used
NYC Local Law 144 bias audit employers compliance study "Null compliance" Wright audit-style 391 employersAuditing work Exploring the New York City algorithmic bias audit regime Groves FAccT 2024 auditors interviewsAI regulation compliance costs small and medium enterprises empirical survey AI Act SMEs cost estimate firm size incumbent advantage studybias audit cost price Local Law 144 auditors fees vendor-level audit employers firm size analysisMa "Auditing Effects on Employment Hiring" New York City bias audit law SSRN working paper
Literature check, 13 October 2026:
Local Law 144 bias audit compliance employer size small firms cost of audit vendorsNYC automated employment decision tools law compliance empirical study audits firms [allowed_domains: arxiv.org, ssrn.com, papers.ssrn.com, osf.io, nber.org]"Null Compliance" Local Law 144 cited OR extends algorithmic audit law employers transparency [allowed_domains: dl.acm.org, arxiv.org, law.nyu.edu, harvardlawreview.org, scholarship.law.columbia.edu, yalelawjournal.org, ssrn.com]Aobdia Ma Zhang "Auditing Effects on Employment Hiring" New York City algorithmic bias audit law abstract (standard)algorithmic hiring audit regulation compliance burden small and medium employers economies of scale vendor audits incumbents empirical [allowed_domains: dl.acm.org, journals.sagepub.com, academic.oup.com, sciencedirect.com, tandfonline.com, cambridge.org, papers.ssrn.com]"Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits" (standard)
Provenance
RO-LD12-06 v2. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-12 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.
Changes
- v2, 13 October 2026: Literature check added: a second, deeper search for existing work, with its queries, new nearest work and limits. The question and verdict stand. See v1 as published.