Research opportunity · Persuasion, Intermediation & the Information Order
Do AI search answers stay faithful to their cited sources in other languages and markets?
Earlier version (v1)
This is RO-LD10-03 as it was published at v1, kept unchanged. The current version is v2; its changes are listed there.
Question
For matched public-interest queries (news, health, elections) issued in several languages and country markets, how often do claims in Google AI Overviews and AI Mode lack support in the pages they cite, how often do they cite sources in the user's own language and country, and do these rates differ from the US English baseline?
Geographic or language coverageMaster's thesisLow to medium costPartly addressed
Why it matters
Citation fidelity is one of the few directly measurable qualities of AI intermediation, and it matters because citations raise trust even when they are wrong. The only large claim-level audit of AI Overviews is US-localized, and multilingual audits of chatbots check sourcing by journalist judgement rather than claim by claim. If fidelity is worse in smaller languages, the users with the fewest alternative sources may get the least reliable intermediation.
The debate lists this among what we still do not know: stronger non-English empirical coverage
Evidence that motivates it
- A 2026 Google AI Overviews audit found 13.7% overall activation, 64.7% on question-form queries, nearly 30% of cited domains absent from the co-displayed first results page, and 11.0% of atomic claims unsupported by cited pages.
Reports 11.0% of atomic claims unsupported by cited pages in a 2026 AI Overviews audit; a cross-market replication is named as what would change it, and it does not license claims about user beliefs. - A 2025 EBU/BBC-led audit across 18 countries and 14 languages found significant issues in 45% of AI-assistant news answers, serious sourcing problems in 31% and major accuracy issues in 20%.
A multilingual audit of AI assistants (18 countries, 14 languages) found serious sourcing problems in 31% of news answers, but covered chatbots rather than answers built into search and does not give a platform-wide error rate. - In a preregistered U.S. experiment, citations increased trust in AI-search answers even when incorrect or hallucinated, while uncertainty highlighting reduced trust and sharing.
Citations increased trust in AI-search answers even when incorrect, which is why unsupported citations matter; the claim does not establish real-world reliance levels.
A possible design
Build a matched query set (about 500 queries per language) from trending searches and fixed public-interest topics, translated and localized by native speakers, in four to six languages spanning high- and low-resource tiers (for example English, German, Spanish, Polish, Hindi, Swahili). Collect AI Overviews and AI Mode answers with their cited pages from in-country vantage points (residential proxies or volunteers) over four to eight weeks. Extract atomic claims and judge support against cited pages with an LLM pipeline validated by native-speaker annotators on a stratified sample in every language, reporting per-language agreement. Compare unsupported-claim rates, activation rates, share of local-language and local-country sources, and overlap with the co-displayed results, using the same pipeline on a US English arm as the benchmark.
Data needed
In-country collection points; native-speaker annotators (paid) for translation and validation; page archiving for cited sources; an LLM or natural-language-inference model for claim checking.
Feasibility
Master's thesis. Low to medium: proxies and model calls a few hundred to a few thousand US dollars; annotator pay is the main cost (roughly US$2,000 to 6,000 for two to three languages). A two-language version suits a master's thesis. Automated collection may conflict with platform terms; check the legal position and prefer volunteer or research-access routes. Claim-checking models are weaker in low-resource languages, so human validation per language is essential, not optional.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 12 October 2026; not yet checked in depth).
The claim-level fidelity audit (Xu et al.) uses trending queries from the US-localized Google Trends dashboard, collected from a US cloud server. A 243-country audit (Aral et al.) measures activation and source mix for English queries only, without claim-level support checks. A multilingual mental-health audit (Nguyen et al.) classifies citations across platforms and found fewer and less language-appropriate citations for non-English queries, but only three questions were translated and it does not check claims against cited pages. The EBU/BBC study covers chatbots, not AI answers in search. A human-validated, claim-level fidelity comparison of AI answers in search across languages and markets was not found.
How strong this evidence is
- Who searched and checked: an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 1 of the 4 nearest works below was judged from an abstract, summary or search listing only; it is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search in English on the date shown; searches in other languages were not run, so audits published only in those languages (for example by national regulators or newsrooms) may have been missed. Google Scholar and paywalled journals were not searched. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Xu, Iqbal & Montgomery, "Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact," arXiv:2605.14021 (13 May 2026)
55,393 trending queries from US-localized Google Trends over 40 days (March to April 2026), collected from a US server; 98,020 atomic claims checked by an LLM verifier, validated by two annotators on a sample of 100 verdicts; 11.0% unsupported. Single market; does not test other languages or countries. - Aral, Li & Zuo, "The Rise of AI Search: Implications for Information Markets and Human Judgement at Scale," arXiv:2602.13415 (13 Feb 2026)
Collects AI and traditional results in 243 countries (location set through a search API) for queries with non-English ones excluded; compares response variety, source popularity, credibility and lean. No claim-level check against cited pages and no non-English queries. - Nguyen, Noorily, Flathers, Notsu, Ospina-Pinillos, Nguyen, Clark, Keane, Thompson & Torous, "Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries," arXiv:2609.00319 (31 Aug 2026)
15,942 citations from ChatGPT, Perplexity and Google AI Overview on 20 English mental-health questions, a subset of three translated into six further languages; non-English queries got fewer citations and were less often directed to language-appropriate resources. Classifies citation types; the abstract does not describe checking whether claims are supported by cited pages. - European Broadcasting Union & BBC, "News Integrity in AI Assistants: An International PSM Study" (EBU, 21 Oct 2025) Judged from an abstract, summary or search listing only
Journalists from 22 public-service media organisations in 18 countries rated over 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity in 14 languages. Covers chatbots, not AI answers in search, and judges sourcing overall rather than claim by claim. Read through the EBU report page and a summary, not the full report PDF.
Search queries used
audit Google AI Overviews citations unsupported claims atomic claims 2026 activation ratemultilingual audit AI search answers citation accuracy non-English languages Perplexity AI Overviews sources faithfulnessEBU BBC News Integrity in AI Assistants report 18 countries 14 languages 45%AI Overviews audit German OR Spanish OR Hindi OR Portuguese language cross-country comparison sources accuracy study
Provenance
RO-LD10-03 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-10 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.