Get This Tool
Kitaru
Summary
Running a model swap or testing a failure path means re-hitting production infrastructure, paying full API costs, and hoping the environment stays stable enough to mean something — Kitaru exists because that loop is broken.
Kitaru wraps your existing agent SDK — PydanticAI, OpenAI Agents, Claude Agent SDK, or raw Python — and turns every model call, tool call, and intermediate step into a durable checkpoint. When you want to ask what would have happened with a cheaper model or a failed retriever, you replay from a specific checkpoint with one override. Nothing re-executes in production. The vendor's own benchmark shows 200 replayed executions on a cheaper model matching outputs in 192 of 200 cases at 84% lower cost. The ceiling appears when your agent's behavior depends on state that Kitaru's adapter doesn't intercept — external side effects or SDK internals the wrapper never sees won't be faithfully replayed.
Bottom line: Pick Kitaru when you need to validate a model downgrade or simulate tool failures against real prior runs without burning production budget — but if your agent relies on side effects or SDK internals outside the adapter's interception surface, your replays will diverge and the what-if stops being trustworthy.
Community Performance Report Card
No community ratings yet. Be the first to rate this tool!
Community Benchmarks Community
Sign in to submit a benchmarkNo community benchmarks yet. Be the first to share a real-world data point.
Pros
Sign in to edit- Replay from any named checkpoint with a single model or override argument, so testing a cheaper model against 200 real prior executions costs a fraction of re-running them live — the vendor reports 84% cost reduction in their own benchmark.
- Adapter-based instrumentation wraps the runner objects you already call (OpenAI Agents, Claude Agent SDK, PydanticAI, raw Python), so you are not rewriting agent logic to get checkpointing — the diff between instrumented and uninstrumented code is a class swap.
- Tool failure simulation via override parameters (`lookup_mode=timeout`, `overrides={checkpoint.lookup_order: stale_order}`) means you can reproduce edge cases that only appeared in one production run without manufacturing a synthetic fixture that may not match real call structure.
- Durable checkpoints support human approval gates mid-execution, so a compliance or review step can pause the agent at a named checkpoint and wait for a sign-off before continuing — without rebuilding the agent's control flow.
- Apache 2.0 open source with a self-hosted path, so the checkpoint store and replay engine stay inside your infrastructure boundary — no production trace data leaves your environment unless you opt into ZenML Pro.
Cons
Sign in to edit- Replay fidelity breaks when agent behavior depends on state the adapter never intercepted — external database reads that changed between the original run and the replay, webhook side effects, or SDK internals the wrapper sits outside of will produce divergent results, and the what-if conclusion becomes unreliable. Teams working around this add manual checkpointing at additional call sites, which means maintaining instrumentation that grows with the agent's surface area.
- Adapter coverage is scoped to PydanticAI, OpenAI Agents SDK, Claude Agent SDK, and raw Python — teams running other frameworks or heavily customized SDK forks have no adapter and must write their own wrapper against Kitaru's primitives, which the docs describe as 'agent runtime primitives and APIs' without a pre-built path. Teams on LangChain or LlamaIndex agent stacks have no drop-in adapter and will evaluate alternatives that instrument at the framework level.
- The replay model assumes the original run was fully checkpointed — if an agent crashed before a checkpoint was written, there is nothing to replay from. Recovery from mid-run crashes depends on checkpoint granularity configured at instrumentation time, not retroactively. Teams that instrument coarsely discover this the first time they need to replay a run that ended partway through a tool chain.
Community Reviews
Sign in to write a reviewNo reviews yet. Be the first to share your experience.
About
- Platforms
- Python, self-hosted cloud
- API Available
- Yes
- Self-Hosted
- Yes
- Last Updated
- 2026-07-07T08:16:03.595Z
Best For
Who it's for
- Teams running production AI agents who need replay-based experimentation
- Developers using PydanticAI, OpenAI Agents, or Claude SDKs
- Users requiring self-hosted durable execution for agents
What it does well
- Test cheaper models on real agent executions without production impact
- Simulate tool failures or degraded retriever results via overrides
- Compare model swaps or input changes across a cohort of prior runs
- Recover from crashes or insert human approval via durable checkpoints
Integrations
Discussion Community
Sign in to commentNo discussion yet. Sign in to start the conversation.
Compare Kitaru
Spotted incorrect or missing data? Join our community of contributors.
Sign Up to ContributeCommunity Notes & Tips Community
Sign in to contributeBe the first to contribute. General notes, observations, gotchas, and tips from people who use this tool day-to-day.
Frequently Asked Questions
- Is Kitaru free?
- Yes — Kitaru is fully free to use. There is no paid tier.
- Is Kitaru open source?
- Yes. Kitaru is open source.
- Does Kitaru have an API?
- Yes. Kitaru exposes a developer API. See the official documentation at https://kitaru.ai for details.
- Can I self-host Kitaru?
- Yes. Kitaru supports self-hosting on your own infrastructure.
- What platforms does Kitaru support?
- Kitaru is available on: Python, self-hosted cloud.
Hours Saved & ROI Stories Community
Sign in to contributeBe the first to contribute. Concrete time/cost savings, with context. e.g. "Cut my code review backlog from 4h to 45m per week."
Curated lists that include this category
Most teams testing a cheaper model or a degraded retriever have two bad options: re-run in production and pay the cost and the risk, or write synthetic test cases that never quite match reality. Kitaru takes a third path — it wraps the calls and steps your agent SDK already exposes and stores them as replayable checkpoints, so a replay is reading back a faithful record of what actually happened, not reconstructing it from scratch. The core workflow is three lines: instrument your agent with a Kitaru adapter, run it in production, then call `client.executions.replay()` with an execution ID and a single override — a different model, a mocked tool return, a degraded retriever result, or a dropped reranker.
The differentiating mechanic is durable execution scoped to agent runs. Kitaru’s adapters for OpenAI Agents SDK and Claude Agent SDK are thin wrappers around the runner objects you already use — `KitaruRunner` replaces `Runner`, `KitaruClaudeRunner` replaces the direct `query` call. The checkpointing strategy is configurable at the call level. Replay reads those checkpoints back, so the what-if reflects the actual model and tool call sequence from the original run, not a re-execution under different conditions.
Kitaru fits teams running agents in production who need an experimentation layer without a separate staging environment or synthetic fixtures. It is open source under Apache 2.0 and self-hosted, with ZenML Pro available separately as a managed control plane. Where it breaks: if your agent’s correctness depends on state that the adapter never intercepts — database writes, external webhook calls, or SDK internals the wrapper sits outside of — replay diverges from reality and the experiment loses its value. Teams hitting that boundary either instrument more aggressively at the SDK level or accept that some failure modes require production reruns regardless.