LaunchRoundsSubmit Software

LLM Observability and Evaluation Tools List (November 2026)

By Alston Antony, editor · Updated November 2026 · How we rank

This LLM observability and evaluation tools list for November 2026 compares 75 tools; the highest-ranked are Langfuse, Portkey and PromptLayer. Most offer a free plan, so shortlist two or three, test them on your own work, then compare entry prices before you pay.

Tools
75
37% have a known pricing model
Free plan
86%
24 of 28 where known
Median entry price
–
too few prices to report
Have an API
100%
16 of 16 where known

Top picks: LLM observability and evaluation tools

The most complete listings with clear pricing, ordered by our published score.

Top 5 side by side

The same facts for each of the leading listings. “Not published” means we found no answer, not that the answer is no.

Top LLM observability and evaluation tools compared on best use, free plan, entry price, platforms, API and main limit
ToolBest forFree planEntry priceRuns onAPIKey limit
LangfuseTeams tracing, evaluating and improving LLM apps in productionYes$29/moSelf-hostedYesFree tier caps at 50k units/month and 30 days of data
PortkeyTeams taking LLM apps to production with monitoring and controlYes$49/moNot publishedYesFree tier caps logs at 10k per month and is not for production
PromptLayerTeams versioning, evaluating and monitoring prompts and agentsYes$49/moWebYesFree tier caps requests, eval executions and dataset size
Weights & BiasesML and AI teams logging experiments and evaluating modelsYes$60/moNot publishedNot publishedFree tier caps seats, storage and ingestion; Pro limited to teams under 50
AgentaTeams building and operating AI agents they can self-hostYes$29/moSelf-hostedNot publishedModel costs excluded; Hobby caps at 2 members and 5,000 runs monthly

How to choose LLM observability and evaluation tools

Best free LLM observability and evaluation tools

24 of the 28 LLM observability and evaluation tools with known pricing offer a free plan or are free to use (86%). These are the highest-ranked ones that are free to start. Free plans usually limit usage, so check the limits on each tool page.

Compare all 75 LLM observability and evaluation tools

Showing 1 to 25 of 75, in score order. The rest are on the next 2 pages.

Filter:
Comparison of LLM observability and evaluation tools: best for, free plan, starting price and focus areas
ToolBest forFree planStarting priceFocus areas
LangfuseTeams tracing, evaluating and improving LLM apps in productionYes$29/moPrompt Management
PortkeyTeams taking LLM apps to production with monitoring and controlYes$49/moAI Gateway, Guardrails, Prompt Management
PromptLayerTeams versioning, evaluating and monitoring prompts and agentsYes$49/moPrompt Library
Weights & BiasesML and AI teams logging experiments and evaluating modelsYes$60/moExperiment Tracking
AgentaTeams building and operating AI agents they can self-hostYes$29/mo—
LangtailTeams that need shared, testable prompt management for LLM appsYes$99/moPrompt Management
OpenLITTeams needing OpenTelemetry-based tracing, evals and guardrails for agentsYes—Guardrails
VivgridTeams running agent tools in the cloud with tracing and evaluationYes$25/moAI Agents
Label StudioTeams labeling multimodal data and running AI evaluation workflowsYes——
MLflowTeams tracking, evaluating and deploying LLM and ML modelsYes—Experiment Tracking, Model Deployment
bottest.aiTeams automating regression, performance and security tests for chatbotsYes——
PromptfooDevelopers and security teams testing AI apps for vulnerabilitiesYes—Guardrails
CometML and GenAI teams tracking experiments and debugging LLM agentsYes—Experiment Tracking
Fiddler AIEnterprises governing agents across creation and production layersYes—AI Governance, Guardrails
Hamming AIQA and reliability for voice and chat agents before and after launchNo——
Maxim AITeams centralising LLM access, cost control and observabilityYes—AI Gateway
OpikDevelopers tracing, evaluating and monitoring LLM agentsYes—Prompt Management
Scale AIAI labs and enterprises needing training data and model evaluationsNo—Training Data
AsymptoteSecurity teams governing coding agentsYes——
GiskardAn open-source framework for testing all ML models, from tabular to LLMs.YesFree—
AgentopsEngineers tracing, debugging and costing AI agent runsYes——
Rhesis AITeams building AI agents with domain-expert collaborationYes——
SaaSpecPay-as-you-go LLM software billed per userNo——
FoundryTeams training and evaluating browser agents at scaleUnknown—Training Data
OrqTeams governing many AI agents, models and tool callsUnknown—AI Gateway, Guardrails

LLM observability and evaluation tools: common questions

Are there free LLM observability and evaluation tools?

Yes. 24 of the 28 LLM observability and evaluation tools with known pricing have a free plan (86%). Examples: Langfuse, Portkey and PromptLayer.

What is the cheapest of the LLM observability and evaluation tools?

Vivgrid has the lowest starting price we found, $25/mo. Across 7 tools with a readable price the median is $49/mo.

How many LLM observability and evaluation tools are there?

We list 75 LLM observability and evaluation tools as of November 2026.

How many LLM observability and evaluation tools are open source?

13 tools (17%) are open source or list a license: Weights & Biases, OpenLIT and Label Studio and more.

Who are LLM observability and evaluation tools built for?

Most listings target developers (72), enterprise (43) and researchers (3).

What do LLM observability and evaluation tools usually focus on?

The most common focus areas are AI Gateway (6), Guardrails (6) and Prompt Management (6).

LLM observability and evaluation tools: the numbers

Tools listed
75
in LLM observability and evaluation tools
Pricing models
freemium 68% · free or open source 18% · paid only 14%
based on 28 tools with known pricing
With a free plan
86%
24 of 28 where we know
Median starting price
$49/mo
7 tools with a readable price, from $25 to $99
With an API
100%
16 of 16 where we know
Platforms
self hosted 13 · cli 2 · web 1
14 tools list a platform
Added in 90 days
75
before 2026-11-18

LLM observability and evaluation tools: in-depth data

How the 75 LLM observability and evaluation tools we list break down by pricing, openness, audience and features, counted from our own data as of November 2026.

  • A free plan is 21 points more common here than across the whole directory (86% against 65%).
Show the full breakdown (7 charts)

How they are priced

Freemium is the most common model: 68% of the 28 LLM observability and evaluation tools with a known pricing model.

  • Freemium68%19
  • Free or open source18%5
  • Paid only14%4

Based on 28 of 75 tools with a known pricing model.

Access and openness

86% offer a free plan, 17% open source or licensed, 100% have an API and 88% built around AI.

  • Offer a free plan86%24
  • Open source or licensed17%13
  • Have an API100%16
  • Built around AI88%66

Based on each bar uses the tools where that fact is known; open source uses all tools in the set.

Who they are built for

Developers is the largest audience at 96% of the 75 tools that name one.

  • Developers96%72
  • Enterprise57%43
  • Researchers4%3
  • Finance teams3%2

Based on 75 of 75 tools that name an audience (a tool can name several).

What they are used for

AI Gateway comes up most, on 24% of the 25 tools with tagged use cases.

  • AI Gateway24%6
  • Guardrails24%6
  • Prompt Management24%6
  • AI Governance16%4
  • Experiment Tracking12%3
  • Training Data8%2

Based on 25 of 75 tools with at least one tagged use case (a tool can carry several).

Most common features

Guardrails appears on 21% of the 29 tools that list features.

  • Guardrails21%6
  • Cost tracking17%5
  • Prompt management17%5
  • Ai gateway14%4
  • Observability14%4

Based on 29 of 75 tools with a feature list.

Where they run

Self-hosted is the most common platform, listed by 93% of the 14 tools that name one.

  • Self-hosted93%13
  • Command line14%2
  • Web7%1

Based on 14 of 75 tools that list a platform.

How many plans they offer

The typical tool publishes 3 plans; 33% publish four or more.

  • One or two plans17%2
  • Three plans50%6
  • Four or more plans33%4

Based on 12 of 75 tools with published plans.

LLM observability and evaluation tools against the whole directory

LLM observability and evaluation tools compared with the whole directory
MeasureLLM observability and evaluation toolsAll tools
Free plan86%65%
API available100%100%
Open source or licensed17%1%
Freemium pricing68%55%

Ready to try one? See Langfuse, our top listing for LLM observability and evaluation tools, or compare all 75.

Part of AI Developer Tools. Data last updated November 2026 · Methodology · LaunchRounds