Verification and Limits

Every serious user of these tools converges on the same discipline: trust the fluency, verify the facts. This chapter makes that concrete, because 'always check everything' is advice nobody follows.

Verify in proportion to consequence

Not everything needs checking. A brainstorm does not. A rephrased sentence does not. The question is what happens if this is wrong.

• Low stakes, easily reversible — a draft, an idea, a rewrite of your own words: skim it, move on
• Medium — anything a colleague or customer will read: check every name, number, date and claim
• High — money, law, medicine, safety, anything published under your name: check against primary sources, and have a qualified human review it

The failure modes worth naming

Knowing the shapes these failures take makes them much easier to spot.

• Fabricated specifics — a citation, statistic, case or quotation that does not exist, delivered in the same confident tone as everything true
• Plausible-but-wrong detail — mostly-right output with one incorrect number or name buried in it, which is harder to catch than a wholly wrong answer
• Outdated information — pricing, features, rules and law that changed after training; especially dangerous because it was once true
• Silent assumption — filling a gap in your prompt with a guess rather than asking, so you never learn the gap existed
• Agreeable drift — following your framing rather than challenging it, which makes it a poor check on your own reasoning unless you explicitly ask it to argue against you

A verification pass that takes five minutes

1. Circle every proper noun, number and date in the output
2. Check each one against a primary source — the vendor's own page, the actual document, the official record
3. Open every cited link. A link that does not resolve is a fabricated citation
4. Ask yourself which single claim the whole piece depends on, and check that one hardest
5. Delete anything you could not verify. Softening it is not enough — unverifiable claims that survive as hedged statements are still claims

Where not to use these tools at all

Some uses are not a matter of careful verification — they are simply the wrong application.

• As a substitute for professional advice on law, medicine or finance. Use it to prepare questions for a professional, not to replace one
• For decisions about real people — hiring, grading, credit, discipline — without a human making and owning the decision
• As a detector of AI-written text. Detection tools are unreliable and disproportionately misjudge writing by people working in a second language
• Anywhere confidential data would be sent to a service whose terms you have not read

The competence that matters

Notice that every safeguard in this chapter requires you to know something about the subject. You cannot spot a wrong number in a field you do not understand.

That is the durable conclusion: these tools raise the value of expertise rather than replacing it. The person who knows the domain gets faster. The person who does not gets confident output they cannot evaluate — which is worse than no output at all.