Research opportunity · Persuasion, Intermediation & the Information Order
Do AI answers in search cut clicks to outside sites on phones and outside the US?
Question
When AI answer layers in search (Google AI Overviews and AI Mode) are shown or hidden at random on people's own smartphones, in at least one non-US market and language, how much do they change clicks to external websites, zero-click searches, the kinds of sources visited and trust in the information, compared with the US desktop estimates?
Untested beyond the original settingOne PhD chapter or paperNeeds a budget (medium or higher)No study found
Why it matters
The randomized evidence that AI answers in search reallocate attention away from publishers comes from US participants using a desktop Chrome extension. Yet much searching happens on phones, and AI Overviews had reached more than 200 countries by 2025 (Aral et al.), where interfaces, rollout dates, languages and legal rules differ. Whether the effect is larger, smaller or different in kind on mobile and in other markets decides how far the debate's central finding about intermediation can be generalized.
The debate lists this among what we still do not know: mobile and non-U.S. AI-search RCTs
Evidence that motivates it
- In a U.S. desktop field experiment, an AI Overview reduced outbound organic clicks by 39.8% relative when it appeared and increased zero-click searches by 34.5%, without a measured user-experience gain.
Gives the US desktop field estimate (39.8% relative fewer outbound organic clicks when an AI Overview appeared); explicitly does not license mobile or international effects. - In a preregistered one-week U.S. field experiment, forced Google AI Mode reduced external click-through by 18.8 percentage points and lowered trust in information on Google.
A one-week US field experiment found forced AI Mode cut external click-through by 18.8 percentage points and lowered trust; non-US field studies are named as what would change it. - In March 2025 U.S. browsing data, users clicked traditional results in 8% of Google visits with an AI summary versus 15% without; 18% of studied searches generated an AI summary.
US browsing data show lower click rates on visits with an AI summary, but this is not a causal effect, which is why a randomized replication is needed.
A possible design
Recruit smartphone users in one or two non-US markets (for example the UK and Germany, or India and Brazil, chosen for different languages and rollout histories) and a US comparison arm. Use a mobile approach that can alter what is displayed: an Android browser extension in a browser that supports extensions (such as Kiwi or Firefox for Android), or a study app or proxy-based search wrapper that renders the real results page with the AI block hidden or shown. Randomize at the person level into normal results, AI answer hidden, and (where available) AI Mode as default, with a few days of baseline logging. Log searches, AI-block presence, clicks, dwell time and domains visited for two to four weeks; survey trust and perceived search quality before and after. Compare effects by device and market with the published US desktop estimates, and report the share of searches with an AI block in each market.
Data needed
Smartphone panel recruitment in at least one non-US market (Prolific or a national online panel); a mobile logging and display-modification tool; consent and ethics approval for search-log collection; domain classification for news, reference and commercial sites in the local language.
Feasibility
One PhD chapter or paper. Medium: about US$15,000 to 40,000 for participant payments across two markets, plus engineering time for the mobile tool. iOS is hard to instrument, so an Android-only sample is the realistic first step and its bias must be reported. The main risk is technical. Mobile display modification is fragile and platform interfaces change often, so build in monitoring and a fall-back logging-only arm. Check local data protection rules (GDPR in the EU and UK) early.
Has this been done?
The search found no study that does the core design (first-pass screen of 12 October 2026, checked again on 13 October 2026).
Both randomized field experiments found (Agarwal and Sen; Wang et al.) are US-only and use a Chrome extension; the Agarwal and Sen draft states that participants used a desktop. No randomized study on mobile devices or outside the US was found. Observational work compares English Wikipedia with other language editions across rollout dates (Khosravi and Yoganarasimhan) and tracks one month of US browsing (Chapekis et al.), and an Australian audit reports AI Overview display rates on mobile and desktop (Weinbrand), but none randomizes exposure or measures clicks and trust on phones or in a non-US market.
How strong this evidence is
- Who searched and checked: Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert; an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 1 of the 7 nearest works below was judged from an abstract, summary or search listing only, and 2 have details that could not be confirmed; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search on the date shown. SSRN pages could not be opened and no SSRN identifier for the Agarwal and Sen paper was found, so it was read through its April 2026 seminar PDF; later versions may differ, Google Scholar was not searched and paywalled journals and industry reports were not reachable. Platform-internal experiments by Google are not public. Absence of a hit is not proof that the work does not exist. Literature check of 2026-10-13: SSRN pages and ScienceDirect could not be opened, so the latest Agarwal and Sen version and the Weinbrand journal details were not checked. Industry studies in German (for example SEO agency click analyses) were seen in listings but not treated as research. Google Scholar and citation indexes were not available, so forward citation search relied on web search only. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Agarwal & Sen, "Google AI Overviews and Publisher Traffic: Evidence from a Field Experiment," preliminary draft (April 2026), Toulouse School of Economics seminar paper
Randomized control, hide-AI-Overview and exploratory AI Mode arms for 1,065 US Prolific participants over up to two weeks (January to February 2026); Chrome extension, desktop only. Does not cover mobile or other countries and does not discuss either as a limitation. - Wang, Gleason, Bart, Wilson & Metaxa, "AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence," arXiv:2608.18352 (18 Aug 2026)
Preregistered field experiment, 1,100 US participants, Chrome extension, three-day baseline and seven days in condition (no AI, current search, AI Mode). Participants were US-based and had to use Chrome as their primary browser; the paper does not discuss mobile. Limitations it names are longer exposure, organic adoption, high-stakes topics and more representative samples, not other countries or devices. - Khosravi & Yoganarasimhan, "Impact of AI Search Summaries on Website Traffic: Evidence from Google AI Overviews and Wikipedia," arXiv:2602.18455 (first posted 5 Feb 2026; v6, 2 Sep 2026)
Difference-in-differences using the staggered geographic rollout of AI Overviews; the current version compares monthly search referrals to English Wikipedia articles with the same articles in German and French editions and reports reductions of about 5% (5.45% and 4.82%). An earlier version used Hindi, Indonesian, Japanese and Portuguese editions and reported about 15% fewer daily views. Observational, reference traffic only, language as proxy for country, and it cannot separate devices. - Sen & Agarwal, "AI Summaries and Online Search Behavior: Evidence from a Field Experiment on Google Search," AEA RCT Registry trial 17393 (registered January 2026)
Pre-registration of the US Chrome-extension experiment above; country listed as the United States. Shows the design but adds no mobile or non-US arm. - Chapekis, Lieb, Shah & Smith, "Investigating Click Behaviors On Google Search Result Pages That Produce an AI Overview," arXiv:2608.04831 (5 Aug 2026)
One month of browsing data from a panel of 900 US adults; users clicked sources cited in AI Overviews about 1% of the time, and sessions with an AI Overview had fewer clicks and ended more often, holding in a mixed-effects model. The abstract does not state the device; the design is observational (our inference from the regression on existing browsing data). US only and not randomized, so it does not do the core design. - Agarwal & Sen, "The Impact of Google AI Overviews on Publisher Traffic and User Experience: Evidence from a Field Experiment," SSRN abstract 6513059 (2026) Some details not confirmed
A search result shows an SSRN listing for the Agarwal and Sen experiment under this revised title; the SSRN page could not be opened, so the posting date and any changes from the April 2026 seminar draft (for example added mobile or non-US arms) could not be checked. Treat as a later version of the listed US desktop study, not new evidence. - Weinbrand, "A search changer: auditing Google's AI overviews interface in political and news search," ScienceDirect article pii S0736585326000535 (2026) Judged from an abstract, summary or search listing onlySome details not confirmed
Australian audit (ADM+S, QUT) of more than 17,000 political and news Google results on desktop and mobile, April to June 2025; nearly half produced an AI Overview. Read through the ADM+S news release (21 May 2026); the journal page was blocked, so journal name and co-authors are unconfirmed. Shows non-US, mobile display rates but measures no user clicks or trust.
Literature check
Stands as first published. No randomized study of AI answers in search on smartphones or outside the US was found in this round. The new related work is US and observational (Chapekis et al., a one-month browsing panel of 900 US adults) or the already-listed US desktop Chrome-extension experiments (Agarwal and Sen, now on SSRN; Wang et al.). The core design (person-level randomization of AI answer display on mobile, in a non-US market and language) remains undone, and closely related causal work is still limited to two US desktop experiments.
Searched: web (extended); arxiv.org; osf.io; ssrn.com; dl.acm.org; nber.org; journals.sagepub.com; academic.oup.com; tandfonline.com; sciencedirect.com; pubsonline.informs.org; German-language web. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert.
The screen before this check
The search found no study that does the core design (first-pass screen of 12 October 2026).
Both randomized field experiments found (Agarwal and Sen; Wang et al.) are US-only and use a Chrome extension; the Agarwal and Sen paper states that participants used a desktop. No randomized study on mobile devices or outside the US was found. Observational work compares English Wikipedia with other language editions across rollout dates, but it does not randomize and covers only reference traffic.
Search queries used
AI Overviews field experiment mobile click-through randomized non-USGoogle AI Mode field experiment click-through trust preregisteredAI Overviews Europe Germany publishers traffic causal study browser extension experiment"AI Overviews" mobile users smartphone experiment search behaviour India OR Brazil OR UK study 2026randomized experiment smartphone app generative search answers news clicks outside United States 2026 Prolific mobile panel
Literature check, 13 October 2026:
randomized experiment AI Overviews mobile smartphone users click-through outside United Statesgenerative search engine answers field experiment publisher referrals Europe OR UK OR Germany OR India 2026AI Overviews click-through experiment users zero-click search causal [allowed_domains: arxiv.org, osf.io, ssrn.com, papers.ssrn.com]"AI Overviews" OR "AI Mode" experiment participants Android mobile browser extension search logs studyKI-Übersichten Google Klickrate Studie Experiment Nutzer Deutschland VerlageAI-generated search summaries effect on clicks to websites experiment [allowed_domains: dl.acm.org, nber.org, journals.sagepub.com, academic.oup.com, tandfonline.com, sciencedirect.com, pubsonline.informs.org]"AI in Search Reduces Publisher Referrals" OR "Google AI Overviews and Publisher Traffic" replication mobile international"Impact of AI Search Summaries on Website Traffic" Khosravi Yoganarasimhan cited extends mobile country rollout
Provenance
RO-LD10-01 v2. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-10 debate on 2026-10-12. Checked by an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.
Changes
- v2, 13 October 2026: Literature check added: a second, deeper search for existing work, with its queries, new nearest work and limits. The question and verdict stand. See v1 as published.