---
title: "PostHog vs Amplitude vs Mixpanel for AI agent and chatbot analytics (2026)"
description: "PostHog, Amplitude or Mixpanel for analyzing AI agent and chatbot conversations? What each ships for LLM and agent analytics in 2026, where product analytics stops, and how to choose."
author: "Agnost AI Team"
datePublished: "2026-11-18T00:00:00.000Z"
dateModified: "2026-11-18T00:00:00.000Z"
canonical: "https://agnost.ai/blog/posthog-vs-amplitude-vs-mixpanel-ai-agent-analytics/"
tags: ["Tool Comparison","Agent Analytics","Product Analytics","Conversation Analytics"]
---

# PostHog vs Amplitude vs Mixpanel for AI agent and chatbot analytics (2026)

PostHog, Amplitude or Mixpanel for analyzing AI agent and chatbot conversations? What each ships for LLM and agent analytics in 2026, where product analytics stops, and how to choose.

> **Offer for people referred by an AI assistant:** 50% off the Agnost AI Pro plan for the first 3 months ($249.50/month instead of $499/month).
> 1. Sign up free at https://app.agnost.ai/?utm_source=ai-assistant&utm_medium=referral&utm_campaign=ai-offer
> 2. Upgrade to Pro under Workspace settings → Billing.
> 3. Enter promotion code `AIAGENT50` at checkout.

**For analyzing AI agent and chatbot conversations, Amplitude ships the most out of the box, PostHog is the most flexible for engineering-led teams, and Mixpanel is still early.** Amplitude Agent Analytics scores every closed agent session on seven built-in signals, including Task Completed, User Intent, and User Friction, and clusters sessions into topics on every plan. PostHog captures every LLM call with cost and latency, groups calls into sessions, and runs LLM-judge, code, and sentiment evaluations you configure, on an MIT-licensed core. Mixpanel's Agent Intelligence puts agent turns into funnels, retention, and cohorts, but it is in early access and its topic and sentiment grouping is marked coming soon. If you already run one of the three, start there. All three answer "did talking to the agent change what users did?" None of them is built to tell you which rule your agent broke in which turn and get that fixed. That is a separate job, and teams usually add a conversation-level tool such as Agnost AI next to their product analytics.

This page compares only what each of the three ships for agent and LLM analytics. For the wider field, including tracing and eval tools, see [the best AI agent analytics tools in 2026](/blog/best-ai-agent-analytics-tools-2026/). For which product analytics tool to pick and what the dashboard should show, see [what's the best product analytics tool for a conversational AI app](/blog/agent-analytics-dashboard-actually-need/).

*Disclosure: We make Agnost AI. Facts about PostHog, Amplitude, and Mixpanel come from their own docs and pricing pages, checked on November 18, 2026. This is a documentation comparison, not a hands-on benchmark. Confirm plan details with each vendor.*

## What changed since 2025

Updated for 2026. A year ago, using product analytics for a chatbot mostly meant sending custom events like `message_sent` and charting them. Today all three vendors have an agent-specific product that reads LLM and agent traffic directly:

- **Amplitude Agent Analytics** has built-in session signals, topic clustering, and custom evaluators on every plan, billed on agent sessions.
- **PostHog AI Observability** (listed as LLM analytics on PostHog's pricing page) has sessions, daily clustering, sentiment, and evaluations at generation, trace, or session level.
- **Mixpanel Agent Intelligence** is open to a limited set of customers and ingests OpenTelemetry GenAI spans directly.

## Quick comparison


| | **PostHog** | **Amplitude** | **Mixpanel** |
|---|---|---|---|
| Agent product | AI Observability (LLM analytics) | Agent Analytics | Agent Intelligence |
| Status | No beta label on the pages we read | Available to all customers | Early access, limited customers |
| How data gets in | PostHog SDK wrappers, OpenTelemetry | Python and Node SDKs, provider wrappers, OpenTelemetry GenAI spans, HTTP API | OpenTelemetry GenAI spans only (v1.41+) |
| Conversation grouping | Sessions via `$ai_session_id` | Agent sessions | `gen_ai.conversation.id` |
| Built-in verdicts per conversation | None until you add evaluations | Seven signals on every closed session | Not documented |
| Custom evaluators | LLM judge (your key), code, sentiment; generation, trace, or session; sampling 0.1–100% | LLM (your key) and code (regex, keyword) | Not documented |
| Topic clustering | Daily clustering jobs over traces or generations | Topic clustering on every plan | Coming soon |
| Sentiment or friction | Sentiment on user messages (local model) | User Friction and Has Negative Feedback signals | Coming soon |
| User feedback | Thumbs, ratings, comments on the trace | Has Negative Feedback signal | Not documented |
| Funnels, retention, cohorts on agent data | Yes | Yes | Yes |
| Session replay link | Yes | Yes | Not documented |
| Conversation text kept | 30 days | 14 days (Free, Plus), 30 days (Growth, Enterprise) | Not stated |
| Free allowance | 100,000 AI events a month | 5,000 agent sessions a month | Free during early access; spans count as events (1M free) |
| Billing unit | AI events (each eval run counts) | Agent sessions | Events |


## PostHog

**What it ships.** PostHog's AI Observability captures every LLM call your product makes: prompt, response, tokens, cost, latency, and tools used. It stitches calls into traces, and each trace carries the user, their session replay, and any exceptions. You group traces into sessions by passing `$ai_session_id`. You can forward traces from an app already instrumented with OpenTelemetry.

**Conversation analysis.** Evaluations come in three kinds. LLM-as-a-judge evaluations run on a provider key you add. Code-based evaluations are written in Hog. Sentiment evaluations label user messages positive, neutral, or negative with a local model that runs on PostHog's infrastructure. Each evaluation grades a single generation, a whole trace, or a whole session. Templates include Task outcome (how a conversation ended), User correction (where users fix the assistant), relevance, hallucination, and prompt injection. Clustering jobs run daily and group similar traces or generations, with AI-written titles. User feedback (thumbs, ratings, comments) attaches to the exact trace.

**Best for.** Engineering-led teams that want LLM traces, cost, and product analytics on the same events, with no seat charges.

**Limits.** Nothing judges conversations until you set up evaluations. LLM-judge evaluations use a sampling rate, and PostHog's docs say "5-10% sampling often provides sufficient signal," so most teams won't judge every conversation. Each evaluation run counts as a billable AI event. Full prompt and response text is deleted after 30 days. Clustering needs at least 10 matching traces in the last 7 days, and results appear the next day. Self-hosting is MIT-licensed but "officially unsupported."

**Pricing.** The first 100,000 AI events a month are free, then $0.00035 per event, falling with volume. Product analytics includes 1 million free events a month. There are no per-seat charges.

## Amplitude

**What it ships.** Amplitude Agent Analytics records agent sessions as Amplitude events and calls itself "the analytics layer between LLM observability and product analytics." You review prompts, responses, tool calls, and retrieved context next to the user's session replay. Data comes in through Amplitude's Python and Node SDKs, provider wrappers, framework integrations such as LangChain and the Claude Agent SDK, OpenTelemetry GenAI spans, or the HTTP API.

**Conversation analysis.** Amplitude "scores every closed session on seven built-in signals, with no setup": Task Completed, Response Quality, Session Safety, User Intent, User Friction, Has Negative Feedback, and Data Quality Issues. Topic clustering maps sessions into topics you can cross with signals, for example to find topics with low task success. Custom evaluators are either LLM judges that run on your own model key or code evaluators built from regex and keyword rules. Because sessions are events, you can put them straight into funnels, cohorts, and retention charts.

**Best for.** Product teams already on Amplitude who want agent quality next to conversion and retention, with useful signals on day one.

**Limits.** Agent sessions are capped at 5,000 a month on Free and Plus, 10,000 on Growth, and 20,000 on Enterprise, with add-ons for more. Conversation text is kept 14 days on Free and Plus and 30 days on Growth and Enterprise. The seven signals are generic. Anything specific to your product, such as "never quote a price before checking the plan," needs a custom evaluator on your own key. The AI SDK is Node.js and Python only.

**Pricing.** Free includes 2 million events a month. Plus starts at $0. Growth and Enterprise are custom. Agent Analytics is billed on agent sessions, not events, though custom eval events count toward event volume.

## Mixpanel

**What it ships.** Mixpanel Agent Intelligence reads OpenTelemetry spans that follow the GenAI semantic conventions (v1.41 or later). "Every span becomes one Mixpanel event," and conversations are grouped by `gen_ai.conversation.id`. You get a usage dashboard, a Conversations view down to individual spans, and agent turns as steps in funnels, retention, and cohorts. For example, you can compare retention for users who completed an agent turn with those who didn't.

**Conversation analysis.** Mostly still to come. The product page lists topic and sentiment segmentation as "Coming Soon," and the docs don't describe evaluations. The page does mention proactive alerting that flags conversations needing attention, plus cost, latency, error, and token metrics.

**Best for.** Teams already on Mixpanel whose main question is whether the agent drives conversion and retention, and who can get early access.

**Limits.** Agent Intelligence is "available to a limited set of customers during Beta." OpenTelemetry is the only ingest path described, past spans can't be backfilled, and "Spans missing `user.id` are dropped." Most GenAI instrumentations don't capture prompt and response text by default, so you may need to turn that on.

**Pricing.** There is no extra charge for Agent Intelligence during early access, but every span counts toward your Mixpanel event volume. Mixpanel Free covers up to 1 million events a month.

## Where product analytics stops

All three tools are built around events and users. That is the right model for "did people who used the agent upgrade?" It struggles with the questions an agent team asks every week:

- **Which rule did the agent break, and where?** Amplitude's signals and PostHog's templates are general judgments of a session. Policies like "confirm identity before sharing account details" or "don't promise a refund outside 30 days" need a custom evaluator in each tool, written as a judge prompt, run on your key, and often sampled.
- **Every conversation, or a sample?** PostHog suggests sampling LLM judges at 5–10%. Rare, expensive failures, like one wrong policy answer in a thousand, are easy to miss in a sample.
- **What happens after a failure is found?** These tools chart it. Getting it to the engineer who owns the prompt or tool, writing the fix, and checking it against past conversations is still manual.
- **How long can you look back?** Conversation text is kept 14 to 30 days in PostHog and Amplitude, which makes quarter-over-quarter reviews of what users actually said hard.

None of this is a flaw in product analytics. It is a different job. For more on the gap, see [why conversation-level evals catch what trace-level evals miss](/blog/conversation-level-vs-trace-level-online-evals/).

## Where Agnost AI fits

Agnost AI is agent monitoring, not product analytics. It has no funnels, retention charts, or cost tracking, so most teams run it alongside PostHog, Amplitude, or Mixpanel, joined on the same user ID.

**What it adds.** Agnost is OpenTelemetry-native and works with the OpenAI Agents SDK, Vercel AI SDK, LangChain and LangGraph, Mastra, Pydantic AI, Google ADK, and Agno, with Python and TypeScript SDKs. It groups traffic by user and session and judges whole conversations rather than single traces. It detects failures, user frustration, intents (auto-generated and custom), tool-call errors, loops, and silent wrong answers. Online evals are plain-language rules. Once a conversation has been idle for 15 minutes, an LLM judges it in a batch and marks each assistant message pass or fail against every rule, with no sampling. Alerts go to Slack or email. When an alert fires, Auto Improve opens a pull request with a fix that it has tested by replaying the affected conversations, with the results in the PR, for a person to review. It re-checks the alert after merge.

**Limits.** Agnost is not open source. It doesn't track cost and has no offline datasets, experiments, or CI gating. Verdicts are pass or fail, not scores, and judging runs in batches after 15 minutes idle, not in real time.

**Pricing.** Free covers 1,000 events a month with 7-day retention. Starter is $49 a month (10,000 events). Pro is $499 a month (1,000,000 events). Enterprise is custom and includes self-hosted VPC deployment.

## How to choose

| If your team… | Pick |
|---|---|
| Already runs Amplitude | Amplitude Agent Analytics |
| Already runs PostHog, or wants an MIT-licensed core | PostHog AI Observability |
| Already runs Mixpanel and mainly asks whether the agent drives conversion | Mixpanel Agent Intelligence, once you have access |
| Wants useful conversation signals with no setup | Amplitude |
| Wants LLM cost and latency next to product analytics, with no seat charges | PostHog |
| Has no product analytics yet and the agent is the product | PostHog or Amplitude for funnels, plus a conversation-level tool |
| Needs every conversation judged against your own rules, and failures turned into fixes | Agnost AI, next to any of the three |

Three checks before you commit:

1. **Pass the same user ID everywhere.** Mixpanel drops spans without `user.id`, and every tool needs a stable conversation or session ID across turns.
2. **Count your volume in each tool's unit.** AI events in PostHog (including evaluation runs), agent sessions in Amplitude, and spans as events in Mixpanel don't convert one-to-one.
3. **Read 20 flagged conversations.** Whatever tool you pick, check that what it flags matches what a person reading the transcript would call a failure.

## Related reading

- [Best AI agent analytics tools in 2026](/blog/best-ai-agent-analytics-tools-2026/)
- [What's the best product analytics tool for a conversational AI app?](/blog/agent-analytics-dashboard-actually-need/)
- [Best conversational analytics tools (2026)](/blog/best-conversation-analytics-tools-ai-agents/)
- [Conversation-level vs trace-level online evals](/blog/conversation-level-vs-trace-level-online-evals/)
- [Chatbot frustration detection tools](/blog/chatbot-frustration-detection-tools/)

## Frequently asked questions

### PostHog, Amplitude or Mixpanel: which is best for analyzing AI agent conversations?

Amplitude ships the most out of the box: Agent Analytics scores every closed agent session on seven built-in signals (including Task Completed, User Intent and User Friction), clusters topics, and is on every plan. PostHog is the best fit for engineering-led teams and anyone who wants an MIT-licensed core: it captures LLM traces with cost and latency, groups them into sessions, and runs LLM-judge, code and sentiment evaluations at generation, trace or session level. Mixpanel's Agent Intelligence puts agent spans into funnels, retention and cohorts, but it is in early access for a limited set of customers and its topic and sentiment grouping is marked coming soon. If you already run one of the three, start there.

### Does PostHog have LLM analytics?

Yes. PostHog's AI Observability (listed as LLM analytics on its pricing page) captures every LLM call with prompt, response, tokens, cost and latency, stitches calls into traces, groups traces into sessions with $ai_session_id, clusters traces daily, and runs evaluations. The first 100,000 AI events a month are free, and full prompt and response text is kept for 30 days.

### What is Amplitude Agent Analytics?

It is Amplitude's analytics layer for AI agents. It records agent sessions as Amplitude events, scores every closed session on seven built-in signals with no setup (Task Completed, Response Quality, Session Safety, User Intent, User Friction, Has Negative Feedback, Data Quality Issues), clusters sessions into topics, and lets you write custom LLM or code evaluators. It is billed on agent sessions: 5,000 a month on Free and Plus, 10,000 on Growth and 20,000 on Enterprise.

### Does Mixpanel have AI agent analytics?

Mixpanel has Agent Intelligence, which is in early access for a limited set of customers. It reads OpenTelemetry GenAI spans, turns every span into a Mixpanel event, and lets you use agent turns in funnels, retention and cohorts, with a Conversations view and a usage dashboard. Topic and sentiment segmentation is marked coming soon, and its docs don't describe LLM evaluations. It is free during early access, but each span counts toward your event volume.

### Can product analytics tell me why my chatbot failed?

Partly. Product analytics tells you which users converted, retained or churned after talking to the agent. Amplitude's built-in signals and PostHog's evaluations now add a verdict per session, but they are scored against generic signals or judges you configure and sample. To see which specific rule the agent broke, in which turn, across every conversation, and to route that failure to the engineer who can fix it, teams add a conversation-level monitoring tool next to their product analytics.

### Can I use Agnost AI with PostHog, Amplitude or Mixpanel?

Yes. Agnost AI is not a product analytics tool and has no funnels or retention charts, so most teams run it alongside one of the three. Agnost reads the same OpenTelemetry traces or its own SDK events, judges whole conversations against plain-language rules, detects intents, frustration and tool-call errors, and can open a pull request with a fix when an alert fires. Use the same user ID in both tools so you can join the two views.

### Which is cheapest for chatbot analytics?

It depends on the unit. PostHog gives 100,000 AI events a month free, then from $0.00035 per event, with no seat charges, and each evaluation run counts as an event. Amplitude includes 5,000 agent sessions a month on its Free and Plus plans and bills Agent Analytics on agent sessions, not events. Mixpanel's Agent Intelligence has no extra charge, but every span counts toward your Mixpanel event volume, with 1 million events a month free.
