Cogloom · Notes

How to Check an AI Answer Before You Trust It

The dangerous thing about a chatbot is not that it is sometimes wrong. Every source is sometimes wrong. The dangerous thing is that it is wrong in exactly the same tone it uses when it is right. A search engine that does not know shows you fewer results. A model that does not know writes you a confident paragraph.

So the skill worth having is not "prompting." It is a fast, boring verification habit you run before you act on anything that matters. Here is the one I use. The whole thing takes about a minute, and the last step is the one most people have never tried.

1. Make it show the reasoning, not just the answer

An answer you cannot inspect is an answer you cannot check. Ask for the working:

Before you answer, list the steps you'll use and the facts each step depends on.
Then answer. Keep the steps visible.

Two things happen. First, you can see where a wrong answer went wrong — usually one bad assumption three steps in, not the whole chain. Second, models are measurably more accurate on numerical and logical questions when they reason out loud instead of jumping to a conclusion, which is why "show your work" appears in essentially every serious prompting guide.

Read the steps for one thing in particular: a fact it asserted without being given it. That is where the error usually lives.

2. Ask for sources — then actually check the sources exist

Asking for citations is half the move. The other half is opening them. A model can produce a citation that is perfectly formatted, plausibly titled, attributed to a real author in a real journal, and completely invented. This is the single most embarrassing failure mode in practice, because a fake citation looks more credible than a real one — real references are messy.

The check is mechanical: copy the title, search for it, confirm it exists and says what the model said it says. If a chatbot has live web access, ask it to give you links and then click them. A link that 404s or lands somewhere unrelated is your answer about the whole response.

3. Ask again in a brand-new chat

This one costs ten seconds and catches a surprising amount. Open a fresh conversation and ask the same question, worded differently, with no reference to the earlier discussion.

Why it works: inside a long conversation the model is strongly influenced by everything already said — including its own earlier claims and any assumption you accidentally planted. It will defend a wrong answer it has already given far more readily than it will invent the same wrong answer twice from a cold start. If the cold answer matches, your confidence should go up. If it does not, you have found the soft spot, and it is worth asking the model directly why the two differ.

4. Invert the question

Models are agreeable. Ask "why is X true?" and you will usually get reasons X is true, whether or not it is. So ask the other one:

Make the strongest case that the answer you just gave is wrong.
What evidence would I expect to see if it were wrong?

A solid answer survives this and the counter-case comes back thin. A shaky one collapses immediately, and you will often watch the model quietly abandon its original position. That reversal is the signal. Note that this cuts both ways — a model that flips under mild pressure is not proving the new answer is right either; it is proving it was never anchored to anything.

5. Do the arithmetic and the units yourself

If there are numbers, check them by hand or in a spreadsheet. Not because the model cannot calculate — it often can, especially when it writes and runs code — but because the failure is rarely the arithmetic. It is the setup: a percentage applied to the wrong base, an annual figure treated as monthly, a rate compounded when it should be simple, a unit conversion done once too often.

Rules of thumb that catch most of it:

• Sanity-check the magnitude first. Should this answer be in the hundreds or the millions? Order-of-magnitude errors are the easiest to spot and the most damaging to miss.
• Check the units on every line, not just the last one.
• Re-derive one number independently. If one checks out and the method is visible, the rest usually follows.

6. Ask what would make the answer wrong

The closing question, and the one almost nobody asks:

What are you least confident about in that answer, and why?
What would I need to verify myself before relying on it?

Given explicit permission to be uncertain, models will generally tell you where the soft ground is — and it is usually the exact spot you were about to build on. The reason this works is that the default behavior is to be helpful and complete, and hedging reads as unhelpful. Change what counts as helpful and the hedging appears.

The three question types where you should assume it is wrong

Verification is cheap but not free, so spend it where the failure rate is highest:

• Anything very recent. A model's knowledge has an end date, and without live search it will answer a question about last month using the world as it was earlier. Worse, it may not flag the gap. Ask directly: "what is your information cutoff, and could this have changed since?"
• Anything very specific and numeric. Exact prices, exact statistics, exact dates, exact quotes, exact section numbers. Recall of a specific figure is precisely where a generative model is weakest and most confident.
• Anything very local. A small business's hours, a municipal rule, a niche product's specifications. Thin coverage in training data produces fluent, plausible, invented detail.

Broad, conceptual, explain-it-to-me questions are where these tools are genuinely excellent. Narrow, factual, look-it-up questions are where they are at their worst — and that is the opposite of how most people use them.

The sixty-second version

1. Ask for the steps, and read them for an unsupported assumption.
2. Ask for sources, and open one.
3. Re-ask cold in a new chat.
4. Ask it to argue the opposite.
5. Re-derive one number yourself.
6. Ask what it is least sure about.

None of this requires knowing anything technical about how the models work. It requires treating a chatbot the way you would treat a fast, well-read, extremely confident colleague who has never once said "I don't know" out loud: useful, worth listening to, and never the last word on anything that costs you money.