AI answered your health question. Here’s how to check it first.
A chatbot’s health answer arrives sounding certain. Certainty is not evidence. Run a five-minute check — sources, one trusted cross-check, a note for your clinician — and know the moments when you skip the check entirely and seek care straight away.
It’s late at night. Your child feels hot, and there’s a rash you don’t recognise. The clinic is closed, and you’re trying not to worry. So you do what millions of people now do: you ask a chatbot.
Within seconds you have an answer. It sounds confident. It names a likely cause, explains it in plain language, and tells you what to do. The knot in your stomach loosens a little. That’s exactly the moment to pause — not because the answer is necessarily wrong, but because a confident tone is not evidence.
One thing to be plain about first: An AI chatbot is not a doctor, and it cannot give medical advice. It can explain general information, but it cannot diagnose you or tell you what treatment you need. Only a clinician who knows you and your history can do that.
Why a confident answer is the trap
A chatbot doesn’t look things up the way a doctor, or even a search engine, does. It generates text by predicting which words are most likely to follow each other, based on patterns in the enormous amount of text it was trained on. That is why its answers read so smoothly — and why smoothness tells you nothing about truth. The same machinery that produces a perfect explanation can produce a perfect-sounding invention.
The World Health Organization has said this plainly: responses from these systems can appear authoritative and plausible while being completely incorrect or containing serious errors, especially on health questions. There are also documented cases of chatbots inventing studies, journal articles and links that look real but don’t exist — which means even a nicely referenced answer needs checking, not just admiring.
Two common mistakes follow from this. The first is treating a second chatbot as a second opinion: two systems trained on similar material agreeing with each other is not verification, it’s an echo. The second is treating detail as proof. A long, specific answer feels more trustworthy than a short one. It isn’t. Length is a property of the writing, not of the evidence.
None of this means you should never ask. It means you should treat the answer as a lead, not a verdict. Here’s a check that takes about five minutes.
The five-minute check
| Step | What to do | What “good” looks like |
|---|---|---|
| 1. Make it show its work | Ask the chatbot for its sources, then actually open them. | The links work, and the source genuinely says what the chatbot claimed it says. A dead link, a page that says something different, or a source that doesn’t exist is a failed check — stop there. |
| 2. Cross-check one authoritative source | Look the question up with one trusted health publisher — the World Health Organization, your country’s national health service, or a major medical institution such as Mayo Clinic or Cleveland Clinic — and compare. | The two accounts agree on the important points. If they disagree, trust the authoritative source, not the chatbot, and treat the disagreement itself as a reason to ask a clinician. |
| 3. Look for uncertainty | Read the chatbot’s answer again and ask: does it admit what it doesn’t know? | Real health information comes with caveats — it depends on age, medical history, other symptoms, and so on. An answer with no caveats, no “it depends,” and no mention of when to seek care is a warning sign, not a strength. |
| 4. Check the date | Look at the date on the source you opened, and be wary of guidance presented as current when it isn’t. | Medical guidance changes as evidence grows. If the source is years old, or the chatbot can’t say where its information comes from, treat the answer as a starting point for a conversation with a clinician, not as instructions. |
| 5. Write it down | Note what you asked, what the chatbot said, and what the authoritative source said. | You now have something concrete to show a clinician or pharmacist, instead of trying to reconstruct a late-night chat from memory. This one step turns a worrying night into a useful appointment. |
If the answer fails step 1 or step 2, that’s your answer: don’t act on it. Go to the authoritative source, or better, to a clinician or pharmacist, and say exactly that — “I asked an AI tool, it said this, and I couldn’t verify it.” Clinicians hear this more and more, and it gives them something real to work with.
And the reverse matters just as much: even if an answer passes every step of the check, that only means the general information held up. It still isn’t personal medical advice, and it doesn’t turn the chatbot into a doctor. Decisions about diagnosis or treatment belong to a clinician, not to an answer that passed a checklist.
Skip the check entirely when it matters
The five-minute check is for the ordinary, non-urgent question — the rash you’re watching, the symptom you want to understand, the advice you want to sanity-check. Some situations don’t get a check at all. If you or someone near you has any of these, skip the chatbot and seek urgent care straight away, through your local emergency number or nearest emergency department:
The same applies in quieter form to anything serious, worsening, or worrying: a symptom that is getting worse rather than better, something that keeps coming back, or anything about a very young child, a pregnancy, or someone with a serious existing condition. Those go to a clinician or pharmacist, not through an AI check. The chatbot can wait. Care can’t.
What AI is actually good for here
Used in its proper place, a chatbot is genuinely useful for health questions — just not as the decider. It’s good at explaining a medical term in plain language after a clinician has used it. It’s good at helping you phrase the questions you want to ask at an appointment. It’s good at turning a confusing leaflet into a short summary you can actually follow up on — but check the summary against the leaflet itself before relying on it, since a summary can quietly drop a warning. Notice what these have in common: the chatbot is helping you understand and prepare, not telling you what is wrong or what to do.
One more habit worth keeping: don’t paste identifying details or a full medical history into a general chatbot. The WHO has flagged that sensitive health information shared with these systems may not be protected the way you’d expect. A question phrased generally — “what can cause a rash like this in children, and when should a child be seen?” — gets you the same kind of information with far less exposure.
The takeaway
An AI health answer is a starting point that arrives dressed as a conclusion. The check is simple: make it show its sources, open them, compare with one source you trust, look for honest uncertainty, check the date, and write down what you found. Five minutes, and you end up in one of two good places: either you understand the general information better, or you have a clear reason — and a clear note — to take to a professional. Either way, nothing here ends with you acting on a chatbot’s say-so. The goal was never to prove the chatbot wrong, and never to let it decide. It was to make sure you walk into a professional’s room better informed.
Sources consulted: World Health Organization statements on large language models and health (May 2023 caution; January 2024 guidance on large multi-modal models); study of reference integrity in AI-generated surgical information, Annals of the Royal College of Surgeons of England (2026); standard emergency guidance on heart attack and stroke warning signs (American Heart Association / FAST). No accuracy statistics are cited in this article, deliberately: published error rates vary widely by model, question type and study design, and any single number would mislead more than it informs.
Free newsletter
Get guides like this in your inbox
Practical AI for parents, teachers, and everyday people. Copy-paste ready. Always free.