Agent Run Observability Checklist

# Agent Run Observability Checklist

Define and inspect an agent trace around outcomes, costs and recoverable failures.

1Overview

A trace field map, failure example and monitoring checklist.

2Requirements

Requires your agent runtime and actual trace/log access; instrumentation is not automatically installed.

3Steps

1. 1. Define run, step, tool-call and outcome identifiers and their parent/child relationships.

2. 2. Record latency, status and known usage/cost without logging secret tool inputs.

3. 3. Capture retry and failure relationships so a successful final response cannot hide earlier failures.

4. 4. Inspect one actual or supplied trace for missing spans, duplicate calls and uncertain side effects.

5. 5. Choose alert conditions and a recovery action based on observed behavior, then test them before enabling paging.

4Worked example

6. Input

```text Run r1 has tool t1 timeout, retry t2 success, and final response success. t1 delivery outcome is unknown. ```

7. Expected result

```text Trace ledger retains t1 timeout and t2 retry, links both to r1 and marks t1’s external delivery unresolved. Final success does not erase that uncertainty. Alert proposal: flag unresolved side effects for review; test with a sandbox trace first. ```

5Acceptance checks

Retain failed attempts and relationships; do not report a fully clean run from final status alone.

6Limits

No hosted monitoring account, instrumentation or pager integration is bundled.

7Suggested tools

• OpenTelemetry-compatible tracing: Use an existing tracing-compatible instrumented runtime
• Your AI assistant: Review the trace and recovery questions
• LibreOffice or your document editor: Document the field and alert contract