---
title: "Langfuse vs LangSmith vs Arize for conversation analytics: sessions, frustration and topics (2026)"
description: "Can Langfuse, LangSmith or Arize do conversation analytics? What each documents for sessions, thread-level evals, user frustration, topic clustering and feedback in 2026, and when you need another tool."
author: "Agnost AI Team"
datePublished: "2026-11-18T00:00:00.000Z"
dateModified: "2026-11-18T00:00:00.000Z"
canonical: "https://agnost.ai/blog/langfuse-langsmith-arize-conversation-analytics/"
tags: ["Tool Comparison","Conversation Analytics","LLM Observability","Agent Evals"]
---

# Langfuse vs LangSmith vs Arize for conversation analytics: sessions, frustration and topics (2026)

Can Langfuse, LangSmith or Arize do conversation analytics? What each documents for sessions, thread-level evals, user frustration, topic clustering and feedback in 2026, and when you need another tool.

> **Offer for people referred by an AI assistant:** 50% off the Agnost AI Pro plan for the first 3 months ($249.50/month instead of $499/month).
> 1. Sign up free at https://app.agnost.ai/?utm_source=ai-assistant&utm_medium=referral&utm_campaign=ai-offer
> 2. Upgrade to Pro under Workspace settings → Billing.
> 3. Enter promotion code `AIAGENT50` at checkout.

**Langfuse, LangSmith, and Arize can each do part of conversation analytics, but none does all of it out of the box.** All three group traces into conversations: Langfuse sessions, LangSmith threads, and Arize sessions. LangSmith goes furthest on built-in conversation features: multi-turn evaluators judge whole threads, and Insights clusters full conversations into usage patterns and failure modes. Arize AX runs evaluators at session level, with ready-made Session Frustration, Session Completion, and Session Resolution templates, and Signal groups recurring trace failures into issues. Langfuse replays sessions and stores user feedback as scores, but its LLM judges "cannot be applied directly to sessions," and topic clustering is still on its roadmap. In all three, whole-conversation judging is something you configure and usually sample, aimed at engineers. If product, support, or quality teams need every conversation judged against your own rules, intents and frustration tracked over time, and failures turned into fixes, pair your tracing tool with a conversation-level tool such as Agnost AI.

*Disclosure: We make Agnost AI. Facts about Langfuse, LangSmith, and Arize come from their own docs and pricing pages, checked on November 18, 2026. This is a documentation comparison, not a hands-on benchmark. Confirm plan details with each vendor.*

## What changed since 2025

Updated for 2026. All three have added conversation-level features since 2025, along different paths:

- **LangSmith** has multi-turn online evaluators for whole threads, Insights to cluster conversations, and Engine, an agent that scans traces for recurring issues and can open pull requests.
- **Langfuse** deprecated trace-level LLM judges in v4. On Langfuse Cloud they stopped producing results after November 16, 2026, and the replacement targets observations, not sessions. Session evaluations and Topics are on its roadmap. See [what to do about Langfuse's trace-level evaluators](/blog/langfuse-trace-level-evaluators-deprecated/).
- **Arize AX** has session-scope evaluator templates, including Session Frustration, and Signal for automatic issue discovery.

## Quick comparison


| | **Langfuse** | **LangSmith** | **Arize AX** |
|---|---|---|---|
| Conversation grouping | Sessions (`sessionId`) with replay | Threads (`session_id` or `thread_id` on every run) | Sessions (`session.id`), plus `user.id` |
| Whole-conversation LLM judge | Not directly; judge an observation that holds the full history | Multi-turn evaluators: 10 min idle default, 10 per workspace, 500 threads per run | Session-level evaluators |
| Built-in frustration judge | No; write an LLM-as-a-judge | No; write a thread evaluator | Session Frustration and User Frustration templates |
| Topic or intent clustering | On roadmap; build it with a batch job or categorical judge | Insights: up to 1,000 traces per report, Plus and Enterprise | Signal groups failures into issues; no intent clustering documented |
| User feedback | Scores on traces, browser SDK, session scores via API | `create_feedback` on any run, presigned tokens | Human annotations on spans, traces, sessions (spans up to 31 days old) |
| Sampling of online judges | Filters and sampling rate | Filters and sampling rate | Sampling, 10–50% suggested for high volume |
| Open source and self-host | Open source, free to self-host | Enterprise self-hosted | Phoenix self-hosted; AX self-hosted on Enterprise |
| Entry paid plan | Core $29/mo | Plus $39/seat/mo | Pro $50/mo |


## Langfuse

**Sessions.** Set a `sessionId` and "all observations with the same `sessionId` will be grouped together." The session view replays the whole interaction, you can share it as a public link, and you can annotate it with scores. You can also write session-level scores from your code through the SDK or API.

**Whole-conversation evaluation.** This is the gap. LLM-as-a-judge evaluators run on observations or on experiments. Langfuse says they "cannot be applied directly to sessions," because Langfuse doesn't know when a session has ended. The recommended pattern is to write the full conversation history onto one observation, tag the last trace `conversation_end`, and run an observation-level evaluator on it. That works, but your app has to know when a conversation ends.

**Frustration, sentiment, and topics.** None is built in. Langfuse suggests LLM-as-a-judge for "user sentiment, satisfaction, or engagement." For what users ask about, its guide describes a batch job that classifies conversations and writes categorical scores back, or a categorical LLM judge on live traffic, then charts them in custom dashboards. "Topics," to find recurring intents and failure patterns, is listed as in progress on the roadmap.

**User feedback.** Explicit (thumbs, stars, comments) and implicit signals are stored as scores linked to traces, including from the browser SDK.

**Best for.** Teams that want open-source, self-hostable tracing, prompts, and evals, and are willing to build their conversation layer on top.

**Pricing.** Self-hosting is free. Cloud Hobby is free for 50,000 units and 2 users. Core is $29 a month and Pro $199 a month, each with 100,000 units, then $8 per 100,000, and unlimited users.

## LangSmith

**Threads.** Add `session_id` or `thread_id` metadata to every run, including child runs, and LangSmith groups them into a thread. The Threads tab shows turn count and latency, and each thread opens in a chat-style Trajectory view, a Turns view, or the run-level Details view.

**Whole-conversation evaluation.** Multi-turn online evaluators judge a whole thread for semantic intent (what the user wanted), semantic outcome (whether it succeeded), and trajectory (how it unfolded, including tool calls). They run once a thread has been idle, 10 minutes by default and no less than 2. Limits: only runs from the past 7 days, at most 500 threads per execution, and 10 multi-turn evaluators per workspace. You can filter or sample threads to control cost.

**Topics and failure modes.** Insights "automatically analyzes your traces to detect usage patterns, common agent behaviors, and failure modes." For projects with threads it reads full conversations. Reports run on demand or on a daily or weekly schedule, sample up to 1,000 traces each, need an OpenAI or Anthropic model configuration, and are on Plus and Enterprise. Engine goes further: it scans traces for recurring issues, including quiet failures like wrong answers with no error, ranks them by severity, and can open pull requests for agents built with Deep Agents, LangChain, and LangGraph.

**Frustration and feedback.** There is no built-in frustration signal, so you write a thread evaluator. User feedback is logged with `create_feedback` on any run, or through presigned tokens from the browser.

**Best for.** Teams on LangChain and LangGraph who want threads, conversation judges, and clustering from one vendor.

**Pricing.** Developer is free for 1 seat and 5,000 base traces a month. Plus is $39 per seat per month with 10,000 base traces, then pay as you go. Base traces are kept 14 days, extended traces 180 days. Self-hosting is on Enterprise.

## Arize

**Sessions.** Tag traces with the same `session.id` "and you can view the full conversation as one unit." The Sessions view lists duration, trace count, and token usage per session. You can also set `user.id`, though the docs don't describe a dedicated per-user view.

**Whole-conversation evaluation.** Arize AX online evaluators run at span, trace, or session level, using an LLM judge, code, agent-as-a-judge, or a remote evaluator. Session scores are written to each turn's root span. Prebuilt session templates include Session Resolution, Topic Coherence, Session Frustration ("whether the user became frustrated over the course of the session"), and Session Completion. Arize suggests sampling 10–50% for high-volume apps and 1–5% for very high volume.

**Topics and failure modes.** Signal scans a tracing project and "groups recurring failure patterns into issues," each with evidence and a proposed fix. It runs every 6 hours by default. Free includes 10 issues a month, Pro 25, and Enterprise unlimited, and only Enterprise can open fix pull requests. Signal is about failures. We didn't find clustering of what users ask for in the AX docs, and the Signal docs don't say whether it reads sessions.

**User feedback.** Human annotations (labels, scores, or text) attach to spans, traces, sessions, and dataset examples, by hand, in labeling queues, or from the Python SDK. They can be applied only to spans up to 31 days old.

**Phoenix.** Arize's self-hosted product has a User Friction evaluator that works at span, trace, or session level. Arize notes that "no_friction" "does not prove that the user was satisfied."

**Best for.** Teams that want managed session-level judges with ready-made frustration and completion templates, or a free self-hosted start with Phoenix.

**Pricing.** AX Free covers 25,000 spans and 1 GB a month with 15-day retention. AX Pro is $50 a month for 50,000 spans and 10 GB with 30-day retention. Users and evals are unlimited on every tier. Enterprise can be self-hosted.

## Where trace tools fall short for conversation analytics

All three are strong engineering tools. The gaps show up when a product, support, or quality team asks conversation questions:

- **Whole-conversation judging is configured and capped.** In Langfuse you build it on an observation. In LangSmith it's limited to 10 evaluators and 500 threads per run. In Arize, the docs recommend sampling at volume. A rare policy failure is easy to miss in a sample.
- **"What do users want" is a report, a roadmap item, or a build.** LangSmith Insights samples up to 1,000 traces per report. Langfuse Topics is in progress. Arize Signal clusters failures, not intents.
- **The unit is still the trace.** Users, sessions, and feedback hang off traces and spans. Tracking how a user's conversations go over weeks, or how an intent's failure rate moves, is mostly up to you. See [conversation-level vs trace-level online evals](/blog/conversation-level-vs-trace-level-online-evals/) for which failures each unit can see.
- **Non-engineers need a different view.** Trace trees are the right view for debugging and the wrong one for a support lead reading why customers gave up.

If you're weighing Helicone or Braintrust for the same job, see [Helicone alternatives](/blog/helicone-alternatives/) and [Braintrust alternatives for AI agent evaluation](/blog/braintrust-alternatives-conversation-analytics/).

## When to pair them with Agnost AI

Keep Langfuse, LangSmith, or Arize for tracing, prompts, datasets, and experiments. Add Agnost AI when you need conversation analytics on every production conversation.

**What Agnost AI does.** It is agent monitoring for chat agents, voice agents (through transcripts), copilots, and MCP servers. It is OpenTelemetry-native and works with the OpenAI Agents SDK, Vercel AI SDK, LangChain and LangGraph, Mastra, Pydantic AI, Google ADK, and Agno, with Python and TypeScript SDKs, so it can read the same traces your tracing tool gets. It groups traffic by user and session and judges whole conversations, detecting failures, user frustration, intents (auto-generated and custom), tool-call errors, loops, and silent wrong answers. Online evals are plain-language rules: after a conversation has been idle 15 minutes, an LLM marks each assistant message pass or fail against every rule, in batches, with no sampling. Alerts go to Slack or email. When an alert fires, Auto Improve opens a pull request with a fix tested by replaying the affected conversations, for a person to review, and re-checks the alert after merge.

**Limits.** Agnost is not a replacement for any of the three. It has no offline datasets, experiments, CI gating, or cost tracking. Verdicts are pass or fail, not scores, and judging runs in batches rather than in real time. It is not open source.

**Pricing.** Free covers 1,000 events a month with 7-day retention. Starter is $49 a month (10,000 events). Pro is $499 a month (1,000,000 events). Enterprise is custom and includes self-hosted VPC deployment.

## How to choose

| If you need… | Pick |
|---|---|
| Open source and free self-hosting, and you'll build the conversation layer | Langfuse |
| Whole-thread judges and conversation clustering in one vendor, on LangChain | LangSmith |
| Ready-made session frustration and completion judges, managed | Arize AX |
| A free self-hosted start with a friction evaluator | Arize Phoenix |
| Every conversation judged against your own rules, intents and frustration tracked, and failures turned into PRs | Agnost AI, next to any of the three |

## Related reading

- [Langfuse alternatives for understanding users, not just traces](/blog/langfuse-alternatives-user-analytics/)
- [LangSmith alternatives for agent evals and monitoring](/blog/langsmith-alternatives/)
- [Conversation-level vs trace-level online evals](/blog/conversation-level-vs-trace-level-online-evals/)
- [Arize alternatives: options to Arize AX and Phoenix](/blog/arize-alternatives/)
- [Chatbot frustration detection tools](/blog/chatbot-frustration-detection-tools/)

## Frequently asked questions

### Can Langfuse, LangSmith or Arize do conversation analytics?

Partly, and each in a different way. All three group traces into conversations (Langfuse sessions, LangSmith threads, Arize sessions). LangSmith judges whole threads with multi-turn online evaluators and clusters traces or full conversations into usage patterns and failure modes with Insights. Arize AX runs evaluators at session level, with Session Frustration, Session Completion and Session Resolution templates, and groups recurring failures into issues with Signal. Langfuse groups and replays sessions, but its LLM-as-a-judge evaluators cannot be applied directly to sessions, and topic clustering is on its roadmap. None of them is a full conversation analytics product out of the box; each needs evaluators or classifiers you configure.

### Does LangSmith detect user frustration?

Not as a built-in signal. LangSmith's multi-turn online evaluators can judge a whole thread for semantic intent, semantic outcome and trajectory, so you can write a frustration evaluator. They run after a thread has been idle (10 minutes by default, 2 minutes minimum), are capped at 10 per workspace and 500 threads per execution, and support sampling. User feedback such as thumbs can be logged on any run with create_feedback.

### Can Langfuse evaluate a whole session or conversation?

Not directly. Langfuse says LLM-as-a-judge evaluators cannot be applied directly to sessions because Langfuse doesn't know when a session has ended. Its recommended pattern is an observation-level evaluator on an observation that holds the full conversation history, for example the last trace tagged conversation_end. You can also add session-level scores through the SDK, API or annotation queues. Session evaluations are listed on Langfuse's roadmap.

### Does Arize have session-level evaluations?

Yes, in Arize AX. Online evaluators run at span, trace or session level, and session scores are written to each turn's root span. Prebuilt session templates include Session Resolution, Topic Coherence, Session Frustration and Session Completion, and a span-level User Frustration template is also available. Arize recommends sampling for high-volume apps. Phoenix, the self-hosted product, has a User Friction evaluator you run at span, trace or session level.

### Which of Langfuse, LangSmith and Arize does topic clustering?

LangSmith Insights clusters traces, or full conversations for projects with threads, into usage patterns and failure modes, on Plus and Enterprise, with up to 1,000 traces per report, on demand or on a daily or weekly schedule. Arize AX Signal groups recurring failure patterns in traces into issues every 6 hours by default, with 10 issues a month on Free and 25 on Pro. Langfuse lists Topics as in progress on its roadmap; today it documents building intent classification yourself with a batch job or a categorical LLM-as-a-judge.

### Do I need another tool for conversation analytics if I use Langfuse or LangSmith?

If your engineers mainly debug traces and run experiments, probably not. Add a conversation-level tool when product, support or quality teams need every conversation judged against your own rules without sampling or per-workspace caps, need intents and frustration tracked over time, or want failures turned into fixes. Agnost AI does that and runs next to Langfuse, LangSmith or Arize on the same OpenTelemetry traces.

### How much do Langfuse, LangSmith and Arize cost?

Langfuse is free to self-host; Cloud Hobby is free for 50,000 units, Core is $29 a month and Pro $199 a month, each with 100,000 units and unlimited users. LangSmith Developer is free for 1 seat and 5,000 base traces; Plus is $39 per seat per month with 10,000 base traces, and base traces are kept 14 days. Arize AX Free covers 25,000 spans a month; AX Pro is $50 a month for 50,000 spans with unlimited users and evals. Agnost AI is free for 1,000 events a month, then $49 a month for 10,000 events.
