Free LLM Observability
As of August 2026, AIDiveForge tracks 11 free llm observability. The top three by verified-data score are burnban, J-lens Qwen, and Subtext. Curated free llm observability tracked by AIDiveForge. Each tool listed is currently free. Listings are verified against each tool's live website and re-checked regularly.
Last updated July 17, 2026 · 11 tools
Ranked by AIDiveForge's verified-data score: data completeness, verification recency, community rating, and real visitor engagement. How we rank · No tool can pay for placement.
1. burnban
Burnban reads supported agent log files already sitting on disk, prices the recorded usage against public API list rates, and lets you set daily, weekly, monthly, or per-agent spend caps enforced in the request path — all from a local dashboard at localhost:4141. The ledger is SQLite on your machine. No keys leave to a Burnban server, no prompts hit a control plane, no account is required. The sharp edge is the word 'supported': log format and provider coverage are scoped, and anything outside that scope remains invisible to the meter. Teams tracking unsupported agents or providers find Burnban shows them a partial picture.
FreeOpen SourceSelf-hostedVerified Jul 17, 2026
2. J-lens Qwen
jlens-qwen36 fits a Jacobian lens to a local Qwen3.6-27B (4-bit) model running on Apple Silicon via MLX, then renders a layer-by-layer visualization of which tokens the model is pushing toward at each position during generation. The canonical demo is a blackmail email prompt: the model outputs a calm, compliant reply, but the workspace band exposes what the intermediate layers were actually predicting — a gap that standard logit inspection never surfaces. This is a single-model, single-hardware tool. It runs on macOS with MLX; there is no cloud backend, no REST API, and no adapter for any other model family. Researchers without Apple Silicon and Qwen3.6-27B (4-bit) specifically cannot run it at all.
FreeOpen SourceSelf-hostedVerified Jul 14, 2026
3. Subtext
Subtext is an open-source, self-hosted tool that surfaces a local language model's internal representations — specifically the J-space global workspace identified in Anthropic research — as the model reads and generates, before output tokens appear. You get a browser-based live view and the ability to record, export, and replay sessions for later analysis. The stack is Python-based and runs against local models, so there is no cloud dependency and no data leaves the machine. The ceiling appears quickly: Subtext has no API, no integration hooks, and no support for models it cannot instrument directly — which means anything running behind a remote endpoint stays opaque.
FreeOpen SourceSelf-hostedVerified Jul 14, 2026
4. role-model
role-model is an open protocol and reference router runtime that routes LLM requests across endpoints using declared capability profiles, routing policy, and measured performance — then emits a structured decision artifact you can inspect after the fact. The router narrows candidates by role and task metadata, rejects endpoints that fail capability, locality, or budget checks, and scores what's left against latency, cost, and reliability evidence. The decision is deterministic enough to audit, not a hidden runtime guess. The baseline role set covers chat, code editing, code review, tool calling, and embeddings — so it fits mixed-workload deployments out of the box. Teams needing autonomous multi-step planning or tool loops will find this deliberately out of scope.
FreeOpen SourceAPISelf-hostedVerified Jun 30, 2026
5. AgentMeter
AgentMeter runs locally — no cloud sync, no account creation, no vendor dashboard to log into — and parses the tool calls, token counts, and caching splits that CLI agents like Claude Code, Gemini CLI, Codex CLI, and Copilot CLI generate. It surfaces the three-tier cost structure that prompt caching creates (input, cached-input, and output tokens each priced differently), which the raw API bill flattens into noise. The value-multiplier calculation compares API spend against estimated developer time saved, giving you a number to put in front of a manager. The wall appears when you need alerting, real-time budget enforcement, or integration with a team billing system — none of that is here.
FreeOpen SourceSelf-hostedVerified Jun 12, 2026
6. Beacon
Beacon is an open-source endpoint telemetry layer that runs locally alongside AI agents, capturing prompts, tool calls, file modifications, and approval workflows before any of that activity disappears into the void. It normalizes that telemetry and forwards it to SIEM platforms like Wazuh, Elastic, or Splunk, so security teams can apply the same detection logic they already run against the rest of the fleet. The architecture is self-hosted by design — no data leaves the endpoint unless you route it there yourself. The project is early-stage; the plugin ecosystem covers the major local agent harnesses but gaps exist for less common runtimes. Teams with agents not yet on the supported list write custom collector plugins — which means more surface area to maintain.
FreeOpen SourceSelf-hostedVerified Jun 1, 2026
7. Flightdeck
Every LLM call, MCP event, and tool invocation your agents make streams to a live dashboard — per-agent timelines and a fleet-wide feed, not batched logs you dig through after the incident. The vendor describes token budgets and MCP allow/block rules you set before problems hit, plus the ability to issue live directives to running agents without restarting them. The self-hosted, Apache-2.0 model means no telemetry leaves your infrastructure — critical for teams in regulated environments or those burned by SaaS observability vendors billing by event volume. The project is early-stage by star count, and the operational surface you take on by self-hosting is real.
FreeOpen SourceSelf-hostedVerified Jun 12, 2026
8. Orchid
Orchid sits between your agent and any API it talks to, capturing traffic into a local SQLite file — no cloud account, no SDK changes, no telemetry leaving your machine. The built-in web UI lets you step through a completed run, inspect every prompt, response, token count, and cost. The proxy also runs a built-in MCP server, so your IDE assistant in Cursor, VS Code, or Claude Code can query recorded traffic directly. Replay is deterministic and costs nothing in API fees. The ceiling appears when your team needs cross-service aggregation or production alerting — this tool is a local debugger, not an observability platform.
FreeOpen SourceSelf-hostedVerified Jun 25, 2026
9. PromptShark
PromptShark is a local MITM proxy written in Go and C++ that sits between your agent and any OpenAI-compatible API endpoint. Every request and response pair is captured, logged, and surfaced in a real-time dashboard — no changes to your agent code, just a single base_url swap. The loop detector flags infinite tool-calling cycles automatically. The time-travel replay feature lets you re-run or edit any historical API step without firing a live request, which means no extra token spend during debugging. The self-hosted, MIT-licensed architecture means your prompts and API keys never leave your own infrastructure.
FreeOpen SourceAPISelf-hostedVerified Jun 18, 2026
10. Rootsign
RootSign is an open-source Python library that attaches tamper-evident provenance logging to AI agent actions — tool calls, API hits, database writes — capturing a verifiable record of what happened, in what order, and under whose authorization. The vendor describes it as the agent capture layer of a broader Agent Accountability Platform. It installs via pip and ships a Docker Compose quickstart for self-hosting, so the audit trail stays inside your infrastructure. The library integrates with LangGraph and CrewAI by wrapping agent actions at the point of execution. At low log volume the architecture holds; teams with high-throughput agents running thousands of tool calls per hour will hit questions the current documentation does not answer about storage scaling and query performance.
FreeOpen SourceSelf-hostedVerified Jun 20, 2026
11. Selvedge
Selvedge is a local MCP server that AI coding agents (Claude Code, Cursor, Copilot) call as they work, logging the reasoning behind every change into a SQLite file that lives next to your code under .selvedge/. Queries are entity-scoped — you ask about users.email or deps/stripe, not line numbers — so the answer surfaces in the same terms you search in. The vendor describes zero telemetry, no accounts, and no external servers; everything stays on disk. The wall appears when your team needs cross-repo provenance or wants to pipe this data into an existing observability stack — Selvedge emits records but does not integrate with those systems out of the box.
FreeOpen SourceSelf-hostedVerified Jun 1, 2026
Listings on this page are sourced and verified by the AIDiveForge data pipeline. AIDiveForge is editorially independent — no money changes hands for inclusion.