Project 42Start learning
Module 11 of 12 · 56 min

Operate, Recover, and Improve Agent Systems

Classify failures across the full agent stack, contain impact, reconcile uncertain actions, recover safely, and turn incidents into verified improvements.

Core concept

By the end

You will be able to

  • Distinguish model, prompt, tool, retrieval, data, policy, orchestration, and infrastructure failures.
  • Triage severity and contain unsafe action, data exposure, cost, and cascading retries.
  • Recover with reconciliation, idempotency, rollback, replay, and human escalation.
  • Convert operational evidence into tests, controls, runbook updates, and measured improvement.
01

Classify the failed boundary before changing the model

Model failures include unsupported or incorrect generation. Prompt failures encode ambiguous or conflicting behavior. Tool failures include bad contracts, authorization, execution, or postconditions. Retrieval and data failures include missing, stale, poisoned, malformed, or cross-tenant evidence.

Policy failures allow forbidden action or block required escalation. Orchestration failures misroute, loop, race, or lose state. Infrastructure failures include capacity, network, dependency, credential, storage, and deployment faults. Several classes can occur in one incident, so record evidence instead of forcing one convenient cause.

02

Triage impact and contain first

Assess affected users, tenants, data, permissions, external actions, money, availability, and legal or safety obligations. Set severity from actual and credible potential impact, not from how fluent the agent output looks.

Contain by pausing the workflow or one tool, revoking credentials, disabling a route, reducing concurrency, switching to a safe fallback, or requiring manual approval. Preserve secret-safe traces, configuration versions, request IDs, and postcondition evidence before mutable state disappears.

03

Reconcile before retry, rollback, or replay

A transport timeout does not prove that a write failed. Query the system of record with an idempotency key or correlation ID, compare the intended and actual postcondition, then choose accept, compensate, retry, rollback, or escalate.

Retry only transient failures with exponential backoff, jitter, provider guidance, and a total budget. Do not retry validation, authorization, policy, or deterministic conflict failures unchanged. Roll back through a tested reversible path and verify the restored state.

Recovery decision
text
if outcome == UNKNOWN: reconcile system_of_record
if unauthorized or policy_blocked: stop + escalate
if transient and retry_budget > 0: backoff + idempotent retry
if harmful change and rollback_tested: rollback + verify
otherwise: preserve evidence + human decision
04

Give responders a decision-ready runbook

A runbook names triggers, severity, owner, communication channel, dashboards and queries, containment choices, credentials and approvals, reconciliation steps, rollback or fallback, validation, escalation, and closure criteria. Commands should be safe to copy and explicit about targets.

Status updates distinguish confirmed facts, hypotheses, actions, impact, and next decision. Notify affected people under the organization's incident and privacy obligations without exposing secrets or unsupported blame.

05

Close only after recovery evidence and prevention work

Verify user-visible recovery, external postconditions, queues and retries, permissions, data isolation, cost, and monitoring. A green health endpoint alone may miss incomplete work or a duplicated side effect.

Write a blameless timeline and contributing factors, add the smallest reproducing case to the evaluation suite, improve the control at the failed boundary, rehearse the changed runbook, and measure recurrence. Track follow-up work to completion rather than treating the post-incident review as closure.

Practice activity

Triage and recover a multi-boundary agent incident

  1. Use a scenario where an agent times out after a consequential tool call, retrieves stale data, and produces an unsupported completion claim.
  2. Build a timeline and classify model, prompt, tool, retrieval, data, policy, orchestration, and infrastructure evidence without assuming one root cause.
  3. Write the severity, containment, reconciliation, retry, rollback, communication, escalation, validation, and closure steps.
  4. Convert the incident into regression cases, a control change, a runbook rehearsal, an owner, and a recurrence metric.

What to produce

  • A failure-classification matrix, incident timeline, impact and severity decision, and executable triage/recovery runbook.
  • Reconciliation and rollback evidence plus tracked evaluation, control, rehearsal, and monitoring improvements.

Reflect before continuing

Which symptom looked like a model failure until the trace exposed a different failed boundary?

Evidence

Sources and verification

Knowledge check

Make it stick.

Pass at 80%

Choose the strongest answer for each question. Your attempts become part of your device-local transcript.

01An agent gives the wrong answer because the retriever returned an expired policy. What is the primary failed boundary?
02A write request times out after reaching the server. What should happen before retry?
03Which failure should not be retried unchanged?
04What is the first priority when an agent may still be taking unsafe actions?
05Which failure map correctly separates system boundaries?