Get This Tool
LangDrift
Pricing
- Model
- Free
Summary
You ship your agent in English, run your evals, everything passes — then a French-speaking user triggers a completely different tool-calling pattern and you find out in production. Langdrift is a locale-aware eval harness built to catch that class of failure before it reaches users.
Langdrift runs your agent prompts across multiple locales and compares behavior — checking whether tool calls, response structure, and decision paths stay consistent when the input language changes. The core problem it addresses is language-induced behavior drift: the same logical request, rephrased in German or Japanese, producing a different agent output than the English baseline. It fits cleanly into CI pipelines where you need deterministic, repeatable checks across locale variants. The project is built and maintained by a single developer, Rubén González, which means the feature surface reflects a focused scope — not a product roadmap backed by a team.
Bottom line: Langdrift earns its place in a CI pipeline for any team shipping agents to multilingual users; it does not replace a full eval framework when your team needs cross-model benchmarking, dataset management, or hosted result dashboards.
Community Performance Report Card
No community ratings yet. Be the first to rate this tool!
Community Benchmarks Community
Sign in to submit a benchmarkNo community benchmarks yet. Be the first to share a real-world data point.
Pros
Sign in to edit- Locale-specific behavioral diffing is built into the core eval loop, which means teams avoid writing and maintaining their own comparison layer on top of a generic test framework.
- Self-hosted and dependency-free at runtime, so it drops into a CI pipeline without requiring an external API or a hosted service that can become a availability dependency.
- Open-source under a single developer's GitHub, which means you can read the full implementation, fork it, and adapt the diffing logic to your agent's specific tool-call schema without waiting on a vendor.
- Targets tool-use preservation across locales specifically, so agents that invoke external tools based on user intent get tested for the failure mode where a translated prompt stops triggering the right tool call.
Cons
Sign in to edit- There is no hosted dashboard or result persistence layer — eval output lives wherever your CI logs live. Teams that need to track drift trends across releases, share results with non-engineering stakeholders, or compare runs over time have to build that infrastructure themselves.
- The project is a solo-maintained open-source repository with no stated team, roadmap, or issue SLA. When a breaking change in your agent framework stops the harness from running, you are either waiting on one person or patching it yourself.
- Langdrift covers locale-induced drift and stops there. Teams that also need cross-model consistency testing, latency benchmarking, or safety eval coverage will hit the boundary of what the harness does within the first sprint — at which point they are evaluating purpose-built platforms like LangSmith or a broader eval framework and treating Langdrift as a single-purpose complement rather than a primary eval system.
Community Reviews
Sign in to write a reviewNo reviews yet. Be the first to share your experience.
About
- Platforms
- Node.js CLI (npm)
- API Available
- No
- Self-Hosted
- Yes
- Last Updated
- 2026-07-09T18:37:55.138Z
Best For
Who it's for
- Teams shipping multilingual AI agents
- Developers needing locale-specific eval harnesses
- CI pipelines requiring deterministic agent tests
- Researchers studying language-induced behavior drift
What it does well
- Testing multilingual agent consistency
- CI regression checks for locale drift
- Benchmarking agent behavior across languages
- Validating tool-use preservation in localized prompts
Integrations
Discussion Community
Sign in to commentNo discussion yet. Sign in to start the conversation.
Compare LangDrift
Spotted incorrect or missing data? Join our community of contributors.
Sign Up to ContributeCommunity Notes & Tips Community
Sign in to contributeBe the first to contribute. General notes, observations, gotchas, and tips from people who use this tool day-to-day.
Frequently Asked Questions
- Is LangDrift free?
- Yes — LangDrift is fully free to use. There is no paid tier.
- Is LangDrift open source?
- Yes. LangDrift is open source.
- Can I self-host LangDrift?
- Yes. LangDrift supports self-hosting on your own infrastructure.
- What platforms does LangDrift support?
- LangDrift is available on: Node.js CLI (npm).
Hours Saved & ROI Stories Community
Sign in to contributeBe the first to contribute. Concrete time/cost savings, with context. e.g. "Cut my code review backlog from 4h to 45m per week."
Curated lists that include this category
Locale consistency is one of the least-tested dimensions of agent behavior — most eval harnesses treat language as a cosmetic variable, not a behavioral one. Langdrift treats locale as a first-class test axis: you define expected agent behaviors and tool-call patterns once, then the harness runs those expectations across a set of locale-specific prompt variants, flagging wherever the agent diverges from baseline. The workflow follows the pattern of writing locale-annotated test cases, executing them against your agent, and collecting structured diff output that identifies where language choice changed the outcome.
The differentiating design decision is that Langdrift is built specifically around locale-induced drift rather than general agent correctness. Generic eval frameworks will let you write multilingual tests manually, but they give you no tooling for the comparison logic — you end up maintaining your own diffing layer. Langdrift bakes that comparison in, so the eval output directly answers the question: did this locale produce a different behavior, and where.
For teams running CI regression checks, Langdrift’s self-hosted, open-source structure means it installs into an existing pipeline without a hosted dependency or API key requirement. The tradeoff is scope: there is no result dashboard, no dataset versioning, no model-routing layer. Teams that outgrow the harness — because they need cross-model comparisons, large-scale benchmark datasets, or collaborative result review — will find themselves building wrapper tooling or migrating to a broader eval platform. The project is maintained by one developer, so feature velocity and long-term support carry the risks that come with solo-maintained open-source software.