in10x / lab / health-citations-study
Which sources AI assistants cite for health questions
The question. Which sources do AI assistants cite for health questions in different countries, and do they name companies?
what I built
A pipeline that asks the same questions in English, Danish and German, stores every raw answer and sorts every cited link by source type. The pages read their numbers from the data file, so no number is typed by hand.
Python · SQLite · OpenAI Responses API with web search · Next.js · Vercel
what happened
- 238 answers from ChatGPT with 747 cited links.
- 60 percent of citations went to public health authorities. 0 went to pharma sites, Wikipedia or forums.
- 0 company names in 238 answers.
- Answers lean local: 71 percent .uk sources in the UK, 64 percent .dk in Denmark and 49 percent .de in Germany.
- Only 44 percent of cited sources stayed the same between two identical runs.
- Unclassified links fell from 19.7 to 1.2 percent after 47 new rules, with no new API calls.
what broke
- The API credits ran out and 62 of 300 calls failed. The Gemini key was dead and there was no Claude key, so the study covers ChatGPT only.
decision
Published as ChatGPT only, with the scope stated on the page. The run across three models is still to do.
Every number on this page comes from the project's own logs, analytics or database. Back to the lab.