To monitor an AI agent in production, capture the user’s goal, the agent’s answer, its tool and retrieval evidence, and the real outcome of the task. Traces tell you what executed. Effective agent monitoring adds checks for whether the answer was supported, whether the requested action actually happened, whether the user had to repeat themselves, and whether a candidate fix improves failing runs without breaking healthy ones.
That distinction matters because many expensive agent failures return HTTP 200, contain no exception, and look normal in a trace viewer. The tool call completed. The model produced fluent text. The user still received the wrong answer.
What should an AI agent monitoring tool detect?
An agent monitor should separate four kinds of failure instead of collapsing them into a generic error rate.
| Failure layer | Example | Evidence needed |
|---|---|---|
| Execution | Tool timed out or model request failed | Status, exception, latency, retry history |
| Evidence | Retriever returned an outdated policy | Retrieved document, version, scope and timestamp |
| Outcome | Agent said a refund completed but it remained pending | Agent claim plus authoritative business state |
| Experience | User repeated the request three times and gave up | Conversation identity, turns, corrections and abandonment |
Traditional application monitoring is strongest at the first row. LLM tracing helps with the first two. Production agent monitoring must cover all four.
For a reproducible demonstration, our agent monitoring detection benchmark compares exception checks, tool-call heuristics and explicit outcome contracts across 17 synthetic cases. It is deliberately small and does not claim to measure real-world prevalence; it shows why execution success and task success are different variables.
What is the minimum telemetry contract?
Start with one record per user-visible interaction. Give it enough identity to connect the public answer to the work that produced it.
| Field | Why it matters |
|---|---|
interaction_id |
Joins the answer to traces, tools and feedback |
conversation_id |
Reveals repeated questions, corrections and context loss |
user_id or privacy-safe account key |
Distinguishes one-off errors from recurring user impact |
agent_name and release |
Shows which agent version produced the behavior |
| User input and public output | Preserves the actual product interaction |
| Tool name, arguments, result and status | Explains actions and evidence used by the answer |
| Retrieved source identifiers and versions | Detects stale, wrong-scope and contradictory evidence |
| Outcome or receipt | Verifies that promised actions reached the required state |
| Feedback and follow-up behavior | Captures rejection even when users never click thumbs down |
Do not collect hidden reasoning or unrestricted personal data merely because the SDK permits arbitrary properties. Capture the smallest approved evidence that can explain and reproduce a failure. Redact before export and keep authorization enforcement in application code.
A five-step monitoring workflow
1. Capture the interaction boundary
For a custom Python application, the basic Agnost SDK boundary is:
import agnost
agnost.init("your-org-id")
interaction = agnost.begin(
user_id="u-123",
agent_name="support-agent",
input="Please refund order 42",
)
# Run the agent and its tools.
answer = "Your refund has been submitted."
interaction.set_properties({
"release": "2026.10.28",
"requested_action": "refund",
"entity_type": "order",
})
interaction.end(output=answer)This creates the conversation-level record. Add framework or OpenTelemetry instrumentation for nested model, retrieval and tool spans. Use one instrumentation route at first; wrapping the same call with two exporters can double-count it. See the Agnost quickstart and OpenTelemetry setup for supported paths.
2. Attach evidence, not just activity
“The refund tool ran” is activity. “Refund r_42 for order 42 reached settled” is evidence.
Record stable identifiers and normalized states at the application boundary. For high-risk actions, compare the agent’s claim with the authoritative system after the operation. Our false-success detection guide contains a concrete receipt contract for this check.
3. Define failure rules from product behavior
Begin with rules engineers and product owners can verify from the recorded evidence:
- Agent claims an action completed without a matching terminal receipt.
- Final answer contradicts the retrieved policy version.
- User asks the same unresolved question again within the conversation.
- The same read-only tool succeeds more than twice with equivalent arguments.
- Agent promises a human handoff but no handoff is accepted.
- A business error is embedded inside a technically successful tool response.
Natural-language rules are useful for classification and investigation. Deterministic authorization, money movement, deletion and safety constraints must remain enforced in code.
4. Trend the failure, then inspect affected runs
A single bad interaction is an incident. A recurring pattern is a product priority.
For each failure class, track:
- affected interactions and unique users;
- rate among eligible interactions, not all traffic;
- release, model, framework and tool breakdowns;
- time to detection and time to resolution;
- recurrence after a fix ships.
Always preserve a path from the aggregate trend to the underlying evidence. A percentage without inspectable examples is difficult to trust; a handful of examples without a denominator is difficult to prioritize.
5. Convert production failures into regression tests
Save the failing input, the evidence available at the time and the expected behavior. Replace side-effecting tools with strict fixtures, run the current and candidate versions, and include healthy controls that the change could damage.
The full process is in Turn Production Agent Failures Into Regression Tests. The release decision should compare cases, not only averages: a candidate that fixes the headline failure while breaking authorization or a common healthy workflow is not an improvement.
Signals for wrong answers that traces do not mark as errors
You rarely have a universal “wrong answer” boolean. Use multiple observable signals and inspect their precision.
| Signal | What it may indicate | Important caveat |
|---|---|---|
| Immediate re-ask | Answer did not resolve the request | User may be adding detail |
| Explicit correction | Wrong entity, fact or interpretation | Sarcasm and quoted text need review |
| Manual rewrite | Output required cleanup | Editing may be normal for the workflow |
| Tool result ignored | Answer conflicts with tool evidence | Tool evidence itself may be stale |
| Unsupported completion claim | Agent promised an action without proof | Missing telemetry is not proof of failure |
| Abandonment after clarification | Question created excessive friction | User may have been interrupted |
| Repeated equivalent tool calls | Loop, uncertainty or bad stopping logic | Retries can be justified after transient errors |
Start alerts at a volume where a person can inspect examples. Label false positives, tighten the evidence contract, and only then increase coverage.
Which monitoring stack should you choose?
No single product is automatically best for every team.
| Team need | Start with |
|---|---|
| Debug individual LLM calls, latency and token cost | LLM tracing and OpenTelemetry |
| Run offline and online evaluators over traces | An evaluation platform with datasets and experiments |
| Self-host an open observability stack | An OpenTelemetry-compatible open-source platform |
| Find recurring user-facing failures and turn them into reviewed changes | Production interaction analysis plus a failure-to-fix workflow |
| Enforce permissions and transaction invariants | Application code and policy controls, regardless of monitoring vendor |
Our agent monitoring tools comparison documents LangSmith, Langfuse, Braintrust, Arize Phoenix and Agnost using their official product documentation. Use the same failure fixtures and healthy controls in every pilot rather than comparing marketing checklists.
A production-readiness checklist
Before calling monitoring complete, confirm that you can answer yes to each question:
- Can we find the exact interaction behind an alert?
- Can we see the answer, tool evidence and authoritative outcome together?
- Can we distinguish failure from missing telemetry?
- Can we group repeated failures without losing inspectable examples?
- Can we reproduce a failure without replaying real side effects?
- Can we compare a candidate change with the deployed baseline?
- Can we identify healthy cases the change must preserve?
- Can we require human review before a generated change is merged?
- Can we measure whether the failure returns after deployment?
- Can we explain what data is collected, redacted and retained?
If the answer is no, the next improvement is usually not another dashboard. It is a stronger link between production behavior, supporting evidence and the test used to approve a fix.
Frequently asked questions
Is tracing enough to monitor an AI agent?
Tracing is necessary but not sufficient. It explains execution: model calls, tools, retrieval, latency and errors. It does not automatically establish that the user’s goal was resolved or that a promised business action completed. Add explicit outcome evidence and conversation-level signals.
Should an LLM judge every production response?
Not by default. Use deterministic checks for structured invariants, sample subjective evaluations according to risk and volume, and calibrate judges against human-reviewed examples. Monitor judge drift and disagreement like any other model behavior.
Can monitoring prevent every hallucination?
No. Monitoring can detect defined failure patterns, surface suspicious behavior and provide evidence for investigation. Prevention also requires retrieval design, tool contracts, authorization, model behavior and product constraints.
How does Agnost differ from a trace viewer?
Agnost starts from production conversations, events, tool calls, intents and behavior violations, then supports investigation through the dashboard, Slack, Claude, Codex and Cursor. Its Auto Improve workflow connects a production finding to a proposed, reviewable change. Existing trace and evaluation tools can remain part of the stack.