Whose conversations do data donations capture?

Research question

How do people who donate their chatbot conversations to researchers differ from the developers' user bases, and how much do findings from donated data change once donors are weighted to the user population?

Why it matters

Data donation is one of the few ways researchers study chatbot use without the developer's permission, alongside public logs, and a possible independent check on vendor statistics. But donors are volunteers, and no benchmark of who they are against the real user base was found. If donated data cannot be validated against the population, independent research swaps one unknown sample for another.

The debate lists this among what we still do not know: Are donated or public conversation datasets representative enough to check vendor usage statistics?

Evidence that motivates it

1. Researchers studied ChatGPT health advice without the developer’s involvement by collecting 202,590 donated conversations from 1,252 users in India, Nigeria, Brazil and Pakistan (December 2022 to February 2026); an AI judge applying a published rubric rated roughly one in five health conversations high-risk. (LD08-SL-C23 v1.0)
   Donated conversations from 1,252 users in four countries, collected without the developer.
   Cites: Roy Chowdhury & Garimella (2026), “How Safe Is ChatGPT…”
2. Analysing 37,845 Australian conversations from WildChat, a public dataset of ChatGPT logs collected outside OpenAI, university researchers classed 61.1% as work-related against 37.3% in OpenAI’s published global figures; to classify them they adopted the classifier prompts and categories of OpenAI-co-authored work. (LD08-SL-C22 v1.0)
   A public log dataset gives a work share very different from the developer's global figure; the two are not comparable.
   Cites: Ma et al. (2026), “How People Use ChatGPT in Australia: A…”

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Recruit donors through a probability-based panel in at least two countries, record demographics and usage frequency, and collect donated histories through data-access rights. Compare donors with non-donors in the same panel and with any published user-base statistics, then reweight key measures (topic shares, work share, sensitive-domain share) and report how far they move.

Data needed

A probability-based survey panel; data-donation tooling under data-access rights; ethics approval; published user-base statistics.

Feasibility

One PhD chapter or paper. Medium to high; panel costs and secure handling of donated data. Developers publish few user-base demographics, so the comparison with the platform population may rest on survey estimates of who uses chatbots.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 16 October 2026; not yet checked in depth).

Donation studies and public datasets exist, but no study was found that measures donor selection against a probability sample and reweights the findings.

Where the search could not reach. Screened by web searches and arXiv record lookups on the date shown; the data-donation methods literature on social media was not searched. Absence of a hit is not proof that the work does not exist.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

• Donated ChatGPT health conversations in four countries (LD08-SL-C23)
  Full text read for this debate. Donations from 1,252 users; no comparison with the user base.
• Karnam, Dash, Gummadi, Mukherjee, Weber & Zannettou, "Bowling with ChatGPT: On the Evolving User Interactions with Conversational AI Systems," arXiv:2602.01114 (2026) Judged from an abstract, summary or search listing only
  Abstract read. 825,000 interactions donated by 300 users through data-access rights; no representativeness benchmark is described in the abstract.

References

Sources behind the motivating claims:

• Roy Chowdhury & Garimella, “How Safe Is ChatGPT Health Advice? Evidence from Real User Conversations in Four Global South Countries,” arXiv:2610.18465, 2026 (SRC-SL-ROY-CHOWDHURY-GARIMELLA-ARXIV-2610-18465, original)
• Ma, Gero, Canonne, Jin & Thilakarathna, “How People Use ChatGPT in Australia: A WildChat Analysis,” arXiv:2609.28990, 2026 (SRC-SL-MA-ET-AL-WILDCHAT-AUSTRALIA-ARXIV-2609-28990, original)

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld08_06_v1,
  author = {{SocioLens project}},
  title = {Whose conversations do data donations capture?},
  year = {2026},
  date = {2026-10-16},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld08-06/v1/}},
  note = {Research opportunity RO-LD08-06, version v1}
}

@misc{ro_ld08_06_nearest_1,
  title = {Donated ChatGPT health conversations in four countries (LD08-SL-C23)},
  url = {https://arxiv.org/abs/2610.18465},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld08_06_nearest_2,
  title = {Karnam, Dash, Gummadi, Mukherjee, Weber \& Zannettou, "Bowling with ChatGPT: On the Evolving User Interactions with Conversational AI Systems," arXiv:2602.01114 (2026)},
  url = {https://arxiv.org/abs/2602.01114},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{src_sl_roy_chowdhury_garimella_arxiv_2610_18465,
  title = {Roy Chowdhury \& Garimella, “How Safe Is ChatGPT Health Advice? Evidence from Real User Conversations in Four Global South Countries,” arXiv:2610.18465, 2026},
  url = {https://arxiv.org/abs/2610.18465},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_ma_et_al_wildchat_australia_arxiv_2609_28990,
  title = {Ma, Gero, Canonne, Jin \& Thilakarathna, “How People Use ChatGPT in Australia: A WildChat Analysis,” arXiv:2609.28990, 2026},
  url = {https://arxiv.org/abs/2609.28990},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}