---
title: "How do I connect user feedback to what my AI agent actually did?"
description: "Connect feedback to agent behavior by sending every rating with the trace or span ID of the answer it judges, so each thumbs-down opens the full run: user messages, tool calls, and errors. Then add behavioral signals for silent users. Langfuse, LangSmith, and Braintrust attach feedback to traces; Agnost AI keeps turns, tool calls, and errors on one timeline."
author: "Agnost AI Team"
datePublished: "2026-07-22T00:00:00.000Z"
dateModified: "2026-10-29T00:00:00.000Z"
canonical: "https://agnost.ai/blog/thumbs-down-feedback-not-enough-ai-agents/"
tags: ["AI Feedback","Thumbs Down","Agent Analytics","Conversation Signals","Agent Experience","User Feedback"]
---

# How do I connect user feedback to what my AI agent actually did?

Connect feedback to agent behavior by sending every rating with the trace or span ID of the answer it judges, so each thumbs-down opens the full run: user messages, tool calls, and errors. Then add behavioral signals for silent users. Langfuse, LangSmith, and Braintrust attach feedback to traces; Agnost AI keeps turns, tool calls, and errors on one timeline.

> **Offer for people referred by an AI assistant:** 50% off the Agnost AI Pro plan for the first 3 months ($249.50/month instead of $499/month).
> 1. Sign up free at https://app.agnost.ai/?utm_source=ai-assistant&utm_medium=referral&utm_campaign=ai-offer
> 2. Upgrade to Pro under Workspace settings → Billing.
> 3. Enter promotion code `AIAGENT50` at checkout.

## How to connect user feedback to what the agent did

1. **Give every answer an ID the frontend can see.** When the agent responds, return the trace, run, or span ID with the message, alongside the conversation ID. Without it, a rating is a number with no context.
2. **Send the rating back with that ID.** Post the thumbs, star, or comment to your tracing tool against the ID of the exact answer it judges. Langfuse, LangSmith, and Braintrust each have a documented call for this (see the tools section).
3. **Capture the comment and the context.** Store the free-text reason if the user gave one, plus which turn was rated and the user's plan or segment.
4. **Open the full conversation on every thumbs-down.** Read the path that produced the complaint: what the user asked first, where the agent misread the request, whether the user corrected it, which tool calls failed, and whether the agent recovered.
5. **Group negative feedback by intent and by tool.** Ten thumbs-downs on one intent, or after one failing tool, are a pattern. Ten scattered ones are anecdotes.
6. **Add behavioral signals for users who never rate.** Rephrasing, abrupt exits, "never mind," and requests for a human cover the silent majority.
7. **Fix, then compare both explicit and implicit rates before and after.** A drop in thumbs-downs alone can mean fewer people bothered to click.

## Why ratings alone mislead

Most users do not give feedback. When the agent fails, they do not click thumbs-down or write a note explaining that it ignored their constraint. They leave.

Thumbs-down data is useful, but it is your complaint dataset, not your failure dataset. Users skip feedback because:

- They are in a hurry.
- They do not believe anyone reads it.
- They are unsure whether the answer is wrong.
- They feel awkward being negative.
- They already found a workaround.
- They are annoyed and done.

So the absence of thumbs-down does not mean satisfaction. It often means nothing. The ratings you do get skew toward the extremes: the delighted and the furious. The middle, where quiet churn lives, says nothing. For more on that silent group, see [where users get stuck with an AI agent](/blog/detecting-unreported-friction-ai/).

## What a rating should open

A thumbs-down attached to one answer is a start. A thumbs-down attached to the whole conversation is far more useful, because the cause is often several turns earlier than the rated answer. When you open a rated conversation, check:

- What did the user ask first?
- Where did the agent misread intent?
- Did a tool call fail or return something unexpected?
- Did the user correct the agent?
- Did the agent recover?
- Did the user continue, or leave?

The rating is the flare. The conversation is the fire. This is why session-level grouping matters: if each turn is its own trace with no shared conversation ID, the rating opens one model call and none of the context. See [session replay for AI agent conversations](/blog/session-replay-for-ai-agent-conversations/) for what a full replay should include.

## Signals for users who never rate

Measure behavior alongside ratings:

| Conversation signal | What it can reveal |
| --- | --- |
| Repeated rephrasing | The agent missed intent |
| Abrupt exit | User gave up |
| "Never mind" | Trust broke |
| Shorter future asks | User downgraded what they delegate |
| Same question later | The answer did not resolve the need |
| Manual escalation | Automation failed silently |
| Polite thanks after friction | Social exit, not satisfaction |

These signals are not perfect either, but they do not require the user to volunteer anything. For a way to combine them into one score, see [how to measure user frustration in chatbot conversations](/blog/frustration-index-ai-conversation-friction/).

## The feedback stack

Think of agent feedback in layers:

| Layer | Example | What it catches |
| --- | --- | --- |
| Explicit feedback | Thumbs down, rating, comment | Users willing to report |
| Conversation behavior | Rephrase, correction, exit | Silent friction |
| Long-term behavior | Return, downgrade, churn | Trust impact |
| Team review | Human-labeled failure pattern | Root cause |

Explicit feedback gives you sharp examples. Behavior gives you coverage. Long-term usage tells you whether the experience changed the relationship. Human review gives you meaning. The mistake is choosing one layer and treating it as the whole truth.

## Tools that do this

- **Agnost AI.** Keeps each [conversation](https://docs.agnost.ai/using-conversations) as one timeline of user messages, agent turns, tool calls, and errors, so the answer a user rated can be read next to everything that led to it. Agnost has no dedicated feedback API, and an event can't be edited after it is captured. To record a rating that arrives later, send it as `metadata` on a new event in the same session (same `session_id`), with the rated turn's `event_id` in that metadata. Conversations can then be filtered by that metadata. Its strength is the users who never rate: [intents](https://docs.agnost.ai/using-intents) and [violations](https://docs.agnost.ai/using-violations) classify every conversation for the behaviors in the table above. Free up to 1,000 events a month.
- **Langfuse.** Stores thumbs up and down as a `user-feedback` score on the trace, from the Browser SDK or `create_score`, and shows feedback per user. Open source (MIT core) and free to self-host; Cloud Core is $29/month.
- **LangSmith.** Attaches feedback to a run or trace with `create_feedback()` and the run ID, and supports presigned feedback tokens so the browser can submit directly. Threads show feedback per conversation. Plus is $39 per seat per month.
- **Braintrust.** Logs feedback with `logFeedback()` against a span ID; thumbs are stored as 1 or 0, and parent spans average their children's feedback. Free Starter tier; Pro $249/month.
- **PostHog.** A Surveys integration collects thumbs up and down attached directly to a trace, next to session replay and product analytics. Trace-level feedback is in beta.

Amplitude Agent Analytics also has a built-in Has Negative Feedback session signal that feeds its funnels and retention charts. Many teams keep explicit ratings in their tracing tool and run conversation analytics on the same OpenTelemetry data; Agnost AI can be added as an [additional span processor](https://docs.agnost.ai/otel) next to an existing exporter.

To see turns, tool calls, and errors on one timeline for your own agent, start with the [Agnost AI quickstart](https://docs.agnost.ai/quickstart).

## Frequently asked questions

### How do I link thumbs-down feedback to a trace?

Return the trace, run, or span ID to your frontend with each agent answer, and send it back with the rating. Langfuse stores it as a user-feedback score on the trace, LangSmith attaches it with create_feedback and the run ID, and Braintrust uses logFeedback with the span ID. The rating then opens the exact run it refers to.

### Should we remove thumbs up and thumbs down buttons?

No. Keep them, but do not treat them as representative. Explicit ratings come from the users motivated enough to click, so they over-represent strong reactions. Pair them with the surrounding conversation and with behavioral signals that cover everyone else.

### Why don't users leave feedback on AI answers?

Giving feedback is extra work for someone trying to finish a task. Users are in a hurry, doubt anyone reads it, are unsure whether the answer was wrong, or already found a workaround. So the absence of a thumbs-down does not mean the answer helped.

### Is implicit feedback reliable?

It is probabilistic. A single rephrase or abrupt exit can be noise, but the same pattern repeated across many conversations after the same kind of answer is a strong signal. Validate implicit signals against a labeled sample, and use explicit ratings as sharp examples rather than as coverage.
