Free LLM Evaluation & Benchmarks
As of August 2026, AIDiveForge tracks 7 free llm evaluation & benchmarks. The top three by verified-data score are LangDrift, SOCBench, and Agent Island. Curated free llm evaluation & benchmarks tracked by AIDiveForge. Each tool listed is currently free. Listings are verified against each tool's live website and re-checked regularly.
Last updated July 29, 2026 · 7 tools
Ranked by AIDiveForge's verified-data score: data completeness, verification recency, community rating, and real visitor engagement. How we rank · No tool can pay for placement.
1. LangDrift
Langdrift runs your agent prompts across multiple locales and compares behavior — checking whether tool calls, response structure, and decision paths stay consistent when the input language changes. The core problem it addresses is language-induced behavior drift: the same logical request, rephrased in German or Japanese, producing a different agent output than the English baseline. It fits cleanly into CI pipelines where you need deterministic, repeatable checks across locale variants. The project is built and maintained by a single developer, Rubén González, which means the feature surface reflects a focused scope — not a product roadmap backed by a team.
FreeOpen SourceSelf-hostedVerified Jul 9, 2026
2. SOCBench
The published detection module runs three frontier LLMs across four analyst personas against 1,205 labeled NetFlow units, scoring each provider-persona pair on F1 (per-flow, per-pair, per-host), verdict accuracy, cost per alert, latency, and completion rate. That scoreboard lets you stop trusting marketing and start comparing models on telemetry that resembles what a real monitoring tier sees. The ceiling is visible immediately: detection is live, but triage, investigation, threat hunting, detection engineering, and threat intelligence are all roadmap items. If your evaluation need extends beyond binary flow classification, SOCBench does not cover it yet — and the roadmap carries no committed dates.
FreeOpen SourceVerified Jul 8, 2026
3. Agent Island
Built by the Stanford Digital Economy Lab and described in arXiv paper 2605.04312, Agent Island puts language models into a shared environment and measures strategic behavior — not just task completion. The benchmark exposes gaps that standard evals miss: can a model read the room, shift alliances, and avoid being outmaneuvered by another agent? The interface exposes play and log views so researchers can inspect run-by-run behavior. Where it breaks: there is no API, no self-hosted option, and no published code repository, so teams cannot integrate Agent Island into a CI pipeline or adapt the environment to their own agent design.
FreeOpen SourceVerified Jun 20, 2026
4. Bloom
Bloom generates targeted evaluation suites for arbitrary behavioral traits.
FreeAPISelf-hostedVerified Apr 20, 2026
5. GEDD
The vendor describes GEDD as a release-readiness tool for AI product managers and domain experts. A PM loads realistic launch-risk scenarios, the domain expert reviews the agent in the shape of the actual task, names failure modes in their own vocabulary, and the session exits with a release report plus a validated evaluation set. That loop converts qualitative judgment into regression gates usable in CI/CD. The ceiling appears when you need programmatic API access — GEDD exposes none, so teams that want to pipe evaluation results into downstream automation build that bridge themselves. Setup requires local installation via pip and depends on sagemaker-mlflow, grounded-evals, and mlflow.
FreeOpen SourceSelf-hostedVerified Jun 9, 2026
6. HermesBench
OpenResume is a browser-based resume builder and parser that keeps all data local: nothing is sent to a server, no account is required. You fill in a form, the tool renders an ATS-optimized PDF in real time, and you download it. The parser side lets you drop in an existing resume and see exactly how an automated screener will read it — which fields it finds, which it misses. The tool handles one job well. It does not support multiple resume versions with branching tailoring logic, and teams needing bulk generation or API-driven output will find no hooks to connect to.
FreeOpen SourceSelf-hostedVerified Jun 9, 2026
7. Proctor
Proctor wraps each agent execution in a Linux sandbox that cuts off access to hidden tests, fix history, and network egress, so the agent cannot read the answers before producing them. After the run, it produces a cryptographically signed verdict bundle that a third party can verify without re-running anything. The signing and forbidden-access timeline together mean cheating leaves a detectable trace. The tool targets researchers and benchmark maintainers on Linux — it is not a hosted service, carries no API surface, and requires you to operate your own infrastructure. Teams with Windows-only CI pipelines or no Linux sandbox provisioning hit an immediate wall.
FreeOpen SourceSelf-hostedVerified Jun 24, 2026
Listings on this page are sourced and verified by the AIDiveForge data pipeline. AIDiveForge is editorially independent — no money changes hands for inclusion.