Verification and Iterative Improvement
Check AI output with evidence, diagnose the kind of failure, and improve the workflow without hiding uncertainty.
By the end
You will be able to
- Match verification strength to claim type, volatility, and consequence.
- Decompose a response into claims and attach the nearest practical evidence.
- Diagnose whether a miss comes from the task, context, prompt, model, tool, or review process.
- Version and compare prompt changes against representative cases.
Verify claims, not confidence
Break a response into checkable units: factual claims, calculations, quotations, classifications, code behavior, recommendations, and actions. Each needs evidence suited to what it asserts.
Use current authoritative sources for volatile facts, recompute calculations, run code and tests, compare quotations with originals, and obtain domain review for consequential judgments.
Scale verification to consequence and uncertainty
Low-consequence brainstorming may need only a usefulness review. Moderate-consequence work benefits from source comparison, examples, tests, and edge cases. High-consequence decisions require current authoritative evidence, independent review, domain expertise, and a safe approval boundary.
Verification should be planned before generation so time, tools, and responsible reviewers are available.
Output use: [DECISION OR ACTION]
Claims to check: [FACTS, MATH, QUOTES, CODE, JUDGMENT]
Consequence if wrong: [LOW, MODERATE, HIGH]
Evidence per claim: [SOURCE, TEST, CALCULATION, REVIEW]
Independent check: [WHO OR WHAT]
Pass criteria: [OBSERVABLE RESULT]
Failure action: [CORRECT, ESCALATE, OR STOP]Diagnose before changing the prompt
A failed result may come from an unclear objective, missing or bad context, contradictory instructions, unsuitable examples, a weak output contract, provider capability, tool failure, or an inadequate review method. Adding more words to the prompt will not fix every cause.
Record the failure category and the smallest change that addresses it. Keep safety and approval boundaries stable while testing changes.
Improve with representative cases
Save a small evaluation set containing normal, boundary, missing-data, conflicting-source, privacy-sensitive, and adversarial-like cases. Compare the previous and revised workflow on the same cases.
Version the prompt, context recipe, output contract, model or product, and evaluation result. A change is an improvement only when it raises the intended quality without unacceptable regressions.
Practice activity
Verify a response and improve the workflow
- Choose an AI response containing at least three different claim types.
- List each claim, its consequence if wrong, and the nearest practical evidence needed to check it.
- Record every failed check and classify its likely cause as task, context, prompt, model, tool, or review.
- Change one controllable element, rerun the original case plus two boundary cases, and compare results.
What to produce
- A claim-to-evidence table with consequence, verification result, and failure action.
- A before-and-after comparison that records one change, three cases, improvements, regressions, and remaining uncertainty.
Reflect before continuing
Which failure initially looked like a prompt problem but came from another part of the workflow?
Evidence
Sources and verification
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileNIST · verified 2026-07-25
- Prompt engineeringOpenAI · verified 2026-07-25
- Prompt engineering overviewAnthropic · verified 2026-07-25
Knowledge check
Make it stick.
Choose the strongest answer for each question. Your attempts become part of your device-local transcript.