Self-Hosted LLM Evaluation & Benchmarks
As of August 2026, AIDiveForge tracks 5 self-hosted llm evaluation & benchmarks. The top three by verified-data score are LangDrift, Bloom, and GEDD. Curated self-hosted llm evaluation & benchmarks tracked by AIDiveForge. Listings are verified against each tool's live website and re-checked regularly.
Last updated July 29, 2026 · 5 tools
Ranked by AIDiveForge's verified-data score: data completeness, verification recency, community rating, and real visitor engagement. How we rank · No tool can pay for placement.
1. LangDrift
Langdrift runs your agent prompts across multiple locales and compares behavior — checking whether tool calls, response structure, and decision paths stay consistent when the input language changes. The core problem it addresses is language-induced behavior drift: the same logical request, rephrased in German or Japanese, producing a different agent output than the English baseline. It fits cleanly into CI pipelines where you need deterministic, repeatable checks across locale variants. The project is built and maintained by a single developer, Rubén González, which means the feature surface reflects a focused scope — not a product roadmap backed by a team.
FreeOpen SourceSelf-hostedVerified Jul 9, 2026
2. Bloom
Bloom generates targeted evaluation suites for arbitrary behavioral traits.
FreeAPISelf-hostedVerified Apr 20, 2026
3. GEDD
The vendor describes GEDD as a release-readiness tool for AI product managers and domain experts. A PM loads realistic launch-risk scenarios, the domain expert reviews the agent in the shape of the actual task, names failure modes in their own vocabulary, and the session exits with a release report plus a validated evaluation set. That loop converts qualitative judgment into regression gates usable in CI/CD. The ceiling appears when you need programmatic API access — GEDD exposes none, so teams that want to pipe evaluation results into downstream automation build that bridge themselves. Setup requires local installation via pip and depends on sagemaker-mlflow, grounded-evals, and mlflow.
FreeOpen SourceSelf-hostedVerified Jun 9, 2026
4. HermesBench
OpenResume is a browser-based resume builder and parser that keeps all data local: nothing is sent to a server, no account is required. You fill in a form, the tool renders an ATS-optimized PDF in real time, and you download it. The parser side lets you drop in an existing resume and see exactly how an automated screener will read it — which fields it finds, which it misses. The tool handles one job well. It does not support multiple resume versions with branching tailoring logic, and teams needing bulk generation or API-driven output will find no hooks to connect to.
FreeOpen SourceSelf-hostedVerified Jun 9, 2026
5. Proctor
Proctor wraps each agent execution in a Linux sandbox that cuts off access to hidden tests, fix history, and network egress, so the agent cannot read the answers before producing them. After the run, it produces a cryptographically signed verdict bundle that a third party can verify without re-running anything. The signing and forbidden-access timeline together mean cheating leaves a detectable trace. The tool targets researchers and benchmark maintainers on Linux — it is not a hosted service, carries no API surface, and requires you to operate your own infrastructure. Teams with Windows-only CI pipelines or no Linux sandbox provisioning hit an immediate wall.
FreeOpen SourceSelf-hostedVerified Jun 24, 2026
Listings on this page are sourced and verified by the AIDiveForge data pipeline. AIDiveForge is editorially independent — no money changes hands for inclusion.