Module 2 of 7 · 48 min

Prompt Claude with Testable Contracts

Turn a real task into a clear, structured, testable Claude prompt and improve it from evaluation evidence instead of guesswork.

Anthropic

By the end

You will be able to

  • Define success criteria and evaluation cases before tuning a Claude prompt.
  • Structure instructions, context, examples, variable input, and output requirements clearly.
  • Use relevant and diverse examples without leaking secrets or treating untrusted content as instructions.
  • Diagnose whether a failed result needs prompt, context, model, tool, or application changes.
01

Define success before changing wording

Prompt engineering begins with a clear success definition, representative examples, and a way to evaluate results. Write the required outcome, unacceptable behavior, output contract, and pass threshold before optimizing phrasing.

Not every failure is a prompt failure. Missing source data, an unsuitable model or surface, unavailable tools, excessive latency, insufficient permissions, or broken application validation require changes outside the prompt.

02

Be clear, direct, and explicit

State the task, audience, trusted context, constraints, required steps, output shape, and verification. If order or completeness matters, use explicit numbered instructions. Explain why a constraint matters when that context helps resolve ambiguity.

Do not rely on Claude to infer hidden organizational norms. A useful test is whether a capable colleague who knows little about the task could follow the same instruction and produce a checkable result.

03

Structure context and trust boundaries

Separate instructions from reference documents, examples, and user-supplied variable input. Descriptive XML tags can make complex prompts easier to inspect and reduce accidental mixing of roles, especially when several documents or examples are included.

Tags organize content; they do not make untrusted content safe. Tell the workflow which material is authoritative, treat retrieved or user-provided text as data, and enforce permissions and output validation outside the model.

Testable Claude prompt shape
xml
<instructions>
  Produce a decision memo using only supported claims.
</instructions>
<success_criteria>
  Include recommendation, tradeoffs, evidence links, and unresolved risks.
</success_criteria>
<context trust="authoritative">
  [APPROVED SOURCE MATERIAL]
</context>
<input trust="untrusted">
  [VARIABLE USER REQUEST]
</input>
<output_contract>
  Markdown with the required headings; label unsupported claims.
</output_contract>
<verification>
  Check every material claim against the approved context before finishing.
</verification>
04

Use examples deliberately

Examples can demonstrate the desired format, tone, classification boundary, and edge-case behavior more precisely than abstract instructions. Use examples that are relevant to the real distribution and diverse enough to avoid teaching an accidental pattern.

Keep examples synthetic or approved, remove secrets and sensitive data, and distinguish example inputs from ideal outputs. Evaluate cases that are not shown in the prompt so improvement does not merely memorize the demonstrations.

05

Iterate from failure evidence

Run the original and candidate prompt against the same versioned case set. Record outputs, rubric scores, critical failures, latency, cost, model and prompt versions, and reviewer disagreement. Change one meaningful hypothesis at a time when practical.

Classify each failure before editing: ambiguous instruction, missing context, misleading example, output-contract violation, model limitation, tool failure, policy boundary, or application defect. Promote passing failures into regression cases and preserve the prior version for rollback.

Practice activity

Build and evaluate a Claude prompt contract

  1. Choose a recurring task and define a rubric with at least four criteria, one unacceptable failure, and an 80 percent pass threshold.
  2. Create six versioned cases: three typical, one boundary, one adversarial or instruction-conflict case, and one prior failure.
  3. Write a prompt that separates instructions, authoritative context, untrusted input, examples, output contract, and verification.
  4. Compare a baseline and candidate on the same cases, classify every miss, and recommend prompt, context, model, tool, or application action from the evidence.

What to produce

  • A secret-free prompt template, rubric, and six-case evaluation set with trust boundaries and expected behavior.
  • A baseline-versus-candidate report with per-case scores, failure classifications, version information, and a ship, revise, or reject decision.

Reflect before continuing

Which failed case could not be repaired safely by prompt wording alone, and what system boundary needed to change?

Evidence

Sources and verification

Knowledge check

Make it stick.

Pass at 80%

Choose the strongest answer for each question. Your attempts become part of your device-local transcript.

01What should exist before prompt wording is optimized?
02What is the main purpose of XML tags in a complex Claude prompt?
03Which example set is most useful?
04A response lacks facts that were never supplied or retrievable. What is the first failure classification?
05How should a candidate prompt be compared with its baseline?