How to tell when an AI answer is wrong

Lawyers keep learning this the expensive way. A public database run by researcher Damien Charlotin had logged more than 1,500 court decisions involving AI-invented citations by mid-2026, and judges have answered with fines and suspensions. The fake cases looked perfect: real-sounding names, plausible courts, tidy quotes. That polish is the problem. A wrong AI answer rarely looks wrong.

The pattern to watch for: confident, specific, unsourced

AI chat tools predict likely-sounding text. Most of the time that lands on the truth. When it misses, it misses smoothly, with the same calm tone as a correct answer. Be most suspicious when an answer is:

• Oddly precise. An exact percentage, a page number, a quote with quotation marks, a statute section. Precision feels like proof, but it's the easiest thing for a model to invent.
• Perfectly on your side. If you asked a leading question ("isn't it true that..."), many tools will agree with you.
• About something recent or local. Last month's rule change, your county's deadline, a small business's hours. Thin training data means more guessing.
• Missing a source, or citing one you can't find.

Treat a specific claim from AI as a lead to check, not a fact to repeat.

Five checks that take two minutes

1. Ask for sources, then open them. Not just "does it have links" but "does the page actually say this." Fabricated or mismatched links are common. If the link is dead or the page says something different, the claim is unverified.
2. Check every number and date against the original. Search the exact figure plus the organization's name. Official numbers live on official sites (.gov, the company's own page, the published study).
3. Re-ask it differently. Open a fresh chat and phrase the question neutrally, or ask "what's the strongest case that this is wrong?" If the answer changes shape, the first one wasn't solid.
4. Ask it to separate known from guessed. "Which parts of this are you unsure about?" won't catch everything, but it often surfaces the weak spots.
5. Cross-check with one human-written source. A reputable reference, a professional association, or the agency itself. Two independent sources agreeing beats one fluent paragraph.

These habits are the same ones we describe in what we mean by practical AI: use the tool for the draft, keep a person on the facts.

When not to rely on AI at all

Some questions carry consequences that one wrong digit can't undo. For these, use AI only to understand the vocabulary, then get the answer from the source:

• Medication doses and drug interactions. Use the label, a pharmacist, or your doctor. A decimal in the wrong place is a real risk.
• Legal deadlines and filing rules. Statutes of limitations, court dates and appeal windows vary by state and change. Go to the court's or agency's own site, or ask a lawyer.
• Tax and benefits rules. IRS.gov, SSA.gov, Medicare.gov and HealthCare.gov publish the actual numbers each year.
• Emergencies. If someone might be hurt, call your local emergency number, not a chatbot.
• Anything you'll sign or pay for. Read the actual contract, not a summary of it.

A quick habit that sticks

Before you act on an AI answer, ask yourself one question: "If this is wrong, what happens?" If the answer is "I'll pick a different recipe," move on. If it's "I'll miss a deadline, take the wrong dose, or lose money," spend the two minutes to verify.

The tools are getting better at citing sources, which makes checking them easier, not optional.