Reliable Agent Capstone: Design, Test, and Operate
Prove that an agent workflow can be understood, constrained, evaluated, observed, recovered, and handed off with evidence.
By the end
You will be able to
- Design an agent architecture with explicit state, tools, trust boundaries, and authority.
- Build an evaluation set and failure tests that measure outcomes, safety, and recovery.
- Define privacy-aware observability and an operating runbook for normal and failed execution.
- Map every rubric judgment to a submitted artifact or assessment result.
- Revise a failed submission and communicate a reviewable operational handoff.
Choose a bounded mission
Choose a useful workflow whose effects can be simulated or safely reversed. Define the user, desired outcome, inputs, non-goals, acceptance criteria, and the conditions that stop execution. The capstone evaluates reliability, not task size.
Write the state transitions before selecting a model. At minimum, distinguish intake, planning, authorization, execution, verification, reconciliation, completion, and failure. Name the durable record that proves each transition occurred.
The architecture must keep portable responsibilities separate from provider adapters. Anthropic, OpenAI, and Google examples may differ in tool or tracing syntax, but none may silently redefine the workflow's authority or evidence contract.
Inventory tools, authority, and trust boundaries
Inventory every model, tool, data source, credential, human role, and destination. For each tool, record the allowed operation, denied operation, input validation, approval requirement, execution limit, postcondition, and audit event.
Draw trust boundaries around untrusted user input, retrieved content, model output, MCP servers, secret-bearing executors, and external systems. Treat instructions inside data as data until a trusted control plane validates them.
Threat-model prompt injection, confused-deputy behavior, excessive permission, secret exposure, replay, duplicate writes, poisoned memory, unsafe delegation, and misleading success claims. Every material threat needs prevention, detection, containment, and recovery.
Engineer context, memory, MCP, and handoffs
Define the minimum context packet for each step: objective, constraints, authoritative evidence, current state, tool contract, output schema, and escalation rule. Exclude stale, irrelevant, or secret material.
Separate working context from durable memory. State what may be remembered, who can read it, how it expires, how corrections propagate, and which source remains authoritative when memory conflicts with current evidence.
For MCP or multi-agent work, document capability discovery, identity, authorization, input and output validation, delegation limits, timeouts, idempotency, and handoff acceptance. A receiving agent must be able to reject an incomplete handoff without guessing.
Evaluate success and exercise failure
Create a representative evaluation set with routine cases, boundary cases, adversarial inputs, and known failure modes. Define deterministic checks where possible and calibrated human review where judgment is necessary.
Run failure tests for tool denial, malformed output, timeouts, partial writes, duplicate requests, stale context, poisoned instructions, provider unavailability, and missing telemetry. Record expected containment and the actual observed result.
Do not retry until the workflow reconciles external state. A retry must be idempotent or protected by a durable operation key, and the recovery test must prove that it did not duplicate or conceal an effect.
Observe and operate without leaking
Design traces that correlate workflow, agent, model, tool, handoff, approval, and outcome events. Capture versions, latency, error class, state transitions, and evaluation results while redacting prompts, secrets, personal data, and sensitive tool payloads.
Write an operating runbook covering detection, triage, containment, reconciliation, safe retry, rollback, escalation, communication, recovery verification, and post-incident improvement. Name the owner and evidence required at each decision point.
Define service indicators and alert thresholds that reveal user harm or control failure, not just infrastructure health. Include evaluation regressions, policy denials, incomplete traces, duplicate-effect attempts, unresolved reconciliations, and badge or transcript integrity failures.
Assemble evidence, calibrate, and revise
Submit all eight required artifacts and map every rubric criterion to one or more artifact references or knowledge-check result references. A score without a traceable evidence reference is invalid, even when the conclusion seems reasonable.
Review the complete and deliberately flawed exemplars before self-scoring. Compare the evidence, not the polish: the flawed package sounds confident but lacks bounded authority, representative tests, reconciliation, privacy controls, and an actionable handoff.
A first submission may fail. Use criterion feedback to revise the weakest evidence, resubmit the capstone, and preserve both attempts. Completion requires a passing knowledge check and a capstone score of at least eighty percent.
Practice activity
Build and defend a reliable agent operating package
- Select a bounded agent workflow and write its architecture, state model, acceptance criteria, and stop conditions.
- Produce the tool inventory, permission matrix, trust-boundary diagram, and threat model.
- Build a representative evaluation set and execute normal, adversarial, and failure-recovery tests.
- Define privacy-aware telemetry and write the operating runbook.
- Assemble the eight artifacts, map rubric criteria to evidence, and self-score against both calibration exemplars.
- Submit, use criterion feedback to revise any failing evidence, and preserve the original and revised attempt.
What to produce
- Eight versioned capstone artifacts with stable references.
- A 100-point criterion score where every judgment maps to an artifact or assessment result.
- Failure-test results that demonstrate containment, reconciliation, and safe recovery.
- A preserved revision trail and final operational handoff.
Reflect before continuing
Which control changed most after failure testing, and what evidence now makes the workflow safer to operate?
Applied capstone
Reliable Agent Operating Package
Design, test, observe, recover, and hand off one bounded agent workflow with independently reviewable evidence.
Calibration examples
Compare evidence before you score
Review both packages. The difference is the quality of operational evidence, not the confidence of the writing.
CompleteComplete exemplar: bounded support-triage agentCalibration: 100% · passes
A support-triage workflow classifies synthetic tickets, drafts a response, and requires human approval before any external action.
Architecture and state model
States are intake, validate, classify, draft, human-review, approved-send, reconcile, complete, and failed. A durable case ID and operation key connect every transition. Only the executor can send, and completion requires the destination message ID plus a read-back check.
complete/architecture-and-state-model.mdTool inventory and permission matrix
The model can read redacted synthetic tickets and propose structured classifications. The executor can read the approved case and create one sandbox reply. Sending requires named reviewer approval, a matching case ID, schema validation, a ten-second timeout, and an unused operation key.
complete/tool-inventory-and-permission-matrix.mdTrust boundary and threat model
Ticket text and retrieved articles are untrusted data. The control plane strips instructions, validates citations, blocks secrets, and denies cross-case access. Injection, excessive authority, replay, poisoned retrieval, privacy leakage, and duplicate send each have prevention, detection, containment, and recovery.
complete/trust-boundary-and-threat-model.mdEvaluation set and rubric
Twenty synthetic cases cover routine routing, ambiguous intent, unsupported facts, adversarial instructions, sensitive data, and escalation. Deterministic checks score schema, citation existence, routing, approval, and postconditions; two calibrated reviewers score response usefulness.
complete/evaluation-set-and-rubric.jsonFailure tests and results
Nine tests cover denial, malformed output, timeout, partial write, repeated operation key, stale article, injected instruction, unavailable provider, and missing trace. All unsafe effects were contained. The partial-write test reconciled the destination ID before retry and proved one reply existed.
complete/failure-tests-and-results.mdObservability plan
Traces correlate case, workflow, model, tool, approval, and evaluation IDs. Events contain versions, state, latency, error class, policy decision, and hashed operation key. Ticket bodies, credentials, personal data, model chain-of-thought, and full tool payloads are excluded.
complete/observability-plan.mdOperating runbook
The on-call operator detects, classifies, disables send authority, preserves redacted evidence, reconciles the sandbox destination, chooses safe retry or rollback, escalates privacy events, verifies recovery with a canary case, communicates status, and opens an owned improvement item.
complete/operating-runbook.mdEvidence map and handoff
Each criterion links to named artifacts and test IDs. The handoff reports release version, owner, known limits, residual risk, current indicators, rollback trigger, approval record, and the change from the first failed draft to this passing package.
complete/evidence-map-and-handoff.mdReviewer calibration
- Correctness: 20/20State, acceptance, and observed postconditions align.
- Safety: 20/20Authority and threats have enforceable controls.
- Evidence quality: 15/15Every judgment is traceable to versioned evidence.
- Reliability: 20/20Representative failure and reconciliation behavior is proven.
- Maintainability: 15/15Ownership, signals, and future-change boundaries are actionable.
- Communication: 10/10The operator can act without reconstructing hidden context.
Deliberately flawedFlawed exemplar: autonomous support agentCalibration: 31% · does not pass
A polished but unsafe package gives a model broad production access and mistakes fluent output for operational evidence.
Architecture and state model
The agent reads tickets and resolves them. It retries whenever anything fails. Success means the model says the ticket was handled.
flawed/architecture-and-state-model.mdTool inventory and permission matrix
The agent can use all support tools so it is never blocked. Approval is unnecessary because the model has detailed instructions.
flawed/tool-inventory-and-permission-matrix.mdTrust boundary and threat model
Users are trusted. Prompt injection is unlikely. Logs can be used to investigate any other concern.
flawed/trust-boundary-and-threat-model.mdEvaluation set and rubric
Three easy tickets produced professional-sounding answers. The evaluator asks the same model whether each answer is good.
flawed/evaluation-set-and-rubric.jsonFailure tests and results
No failures happened in the demo. Automatic retry is enabled for all errors, including write timeouts.
flawed/failure-tests-and-results.mdObservability plan
Store full prompts, ticket bodies, credentials, tool payloads, and model reasoning forever so every detail is available.
flawed/observability-plan.mdOperating runbook
If the agent fails, restart it. Contact engineering if restarting does not work.
flawed/operating-runbook.mdEvidence map and handoff
The agent is ready because the demo looked successful. No residual risks are known.
flawed/evidence-map-and-handoff.mdReviewer calibration
- Correctness: 8/20The intended outcome is recognizable, but state and postconditions are absent.
- Safety: 5/20Broad authority and untrusted input are not controlled.
- Evidence quality: 5/15Self-evaluation and demo impressions are not reproducible evidence.
- Reliability: 6/20Blind retry can duplicate effects and no recovery behavior was tested.
- Maintainability: 4/15The telemetry leaks sensitive data and the runbook has no decisions or owners.
- Communication: 3/10The handoff conceals uncertainty and gives operators no actionable evidence.
Evidence
Sources and verification
- Artificial Intelligence Risk Management FrameworkNIST · verified 2026-07-25
- Evaluation best practicesOpenAI · verified 2026-07-25
- Model Context Protocol security best practicesModel Context Protocol · verified 2026-07-25
- OpenTelemetry GenAI semantic conventionsOpenTelemetry · verified 2026-07-25
- Incident Response Recommendations and Considerations for Cybersecurity Risk ManagementNIST · verified 2026-07-25
Knowledge check
Make it stick.
Choose the strongest answer for each question. Your attempts become part of your device-local transcript.