Open-source LLM engineering platform for traces, evals, and prompt management
Langfuse is an open-source product for LLM application observability: it ingests traces and spans from your stack, supports datasets and prompt/version workflows, and offers optional Langfuse Cloud or self-hosted deployment. It integrates with popular Python/JS SDKs and frameworks that emit OpenTelemetry-compatible telemetry, so teams can debug agent loops, compare prompt iterations, and monitor production quality metrics without building a custom analytics pipeline from scratch.
Use cases
- Debugging tool-heavy agent runs where failures occur deep in a call chain
- Tracking latency and token usage across routes and model versions
- Building eval sets from production traces for regression testing before rollout
- Comparing prompt edits with consistent datasets rather than anecdotal chat checks
- Giving platform teams a shared view of LLM behavior in staging and production
Key features
- Trace and session views for multi-step LLM and agent workflows
- Prompt management with versioning and side-by-side comparisons
- Datasets and scoring workflows for offline evaluation and regression checks
- SDK integrations for Python and JavaScript ecosystems
- Self-hosting option alongside Langfuse Cloud for teams with data residency requirements
Who Is It For?
- ML and platform engineers operating LLM services
- Product teams shipping agentic features who need production visibility
- Developers self-hosting models or gateways who want trace storage under their control
Frequently Asked Questions
- Is Langfuse the same as a generic APM?
- It is specialized for LLM workloads: traces include prompts, completions, tool calls, and scores rather than only HTTP timings, though it can sit alongside traditional APM.
- Can I run Langfuse on my own infrastructure?
- Yes—Langfuse documents self-hosted deployment patterns in addition to its managed cloud offering.
- Does it replace automated evaluation?
- No—it helps you collect data and run eval workflows; you still define tasks, judges, or heuristics appropriate to your product.
Related
Related
3 Indexed items
LangSmith
LangSmith is LangChain's hosted and self-hostable platform for tracing, monitoring, and improving LLM applications. Official documentation at docs.langchain.com describes instrumenting apps via environment variables, framework integrations (OpenAI, Anthropic, CrewAI, Vercel AI SDK, Pydantic AI, and others listed on the integrations page), or the LangSmith SDK so teams can inspect multi-step runs, compare prompt versions, build datasets, run offline and online evaluations, configure automations, and collect feedback queues—without assembling bespoke analytics for agent loops.
Braintrust
Braintrust documents an AI observability platform at braintrust.dev where teams instrument applications to capture traces (inputs, outputs, latency, token usage, nested tool calls), analyze logs, annotate with human feedback, run experiments and scorers, and iterate on prompts before deployment. Official docs describe a workflow spanning Instrument → Observe → Annotate → Evaluate → Deploy, with auto-instrumentation for major providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others listed in the integrations directory) and frameworks such as LangChain, LangGraph, Vercel AI SDK, and Pydantic AI. Span types documented include task, llm, function, tool, and score spans, each capturing metrics and metadata for debugging and building evaluation datasets.
LiteLLM
LiteLLM is an open-source Python library and proxy stack documented at docs.litellm.ai that exposes a single `completion()` interface across providers such as OpenAI, Anthropic, Vertex AI, Bedrock, and Ollama using OpenAI-compatible request and response shapes. The project documents a Router with retry, fallback, and load-balancing across deployments, optional observability callbacks (Langfuse, MLflow, Helicone, and others listed in observability guides), and a self-hosted LiteLLM Proxy (LLM Gateway) with virtual keys, spend tracking, guardrails, and an admin UI. Recent documentation also describes an MCP Gateway that centralizes MCP tool access with per-key, per-team, and per-organization permissions.