Cuey
Freemium
Power users comparing multiple LLMs daily
Bloom
FreemiumOpen SourceAPISelf-HostedAgentic
Anthropic's evaluation framework that auto-generates targeted test suites to measure specific AI behaviors at scale.
$17-$100+/mo · Free for everyone - Chat on web, desktop, and mobile with usage limits · Checked Oct 19, 2026
Similar tools
CueyFreemiumCuey sits inside your existing ChatGPT, Claude, or Gemini session and routes the same prompt…
AgiRankerFreeThe tool lets you rank frontier models on a 0–100 AGI Score, explore per-domain breakdowns…
oqoqoFreemiumOqoqo runs agent evaluation experiments on managed cloud infrastructure, spinning up sandboxed…
ClientCodedPaidClientCoded runs adversarial multi-turn conversations against your agent, scores every exchange…
Get This Tool
License: MIT Any use incl. commercial
Local-run terms: MIT license permits unrestricted use, modification, and distribution for any purpose, including commercial applications, with proper attribution.
Screenshots 1
Summary
Bloom tackles a real gap in AI safety work: most evaluations are either broad (and miss edge cases) or manual (and don't scale). This tool generates reproducible test scenarios for narrow, high-stakes behaviors—like refusal consistency or prompt injection resilience—then scores models across thousands of permutations. The catch is structural: it's only as good as the seed prompts and judging logic you feed it, so teams still need to validate results manually for ambiguous cases. Free to use, but assumes you have both the domain expertise to set it up and the infrastructure to run it.
Bottom line: *Use this if you're a safety researcher or red-teaming group who needs systematic, repeatable measurements of specific model risks.*
Pricing Plans
SubscriptionLast verified 2 weeks ago- Price
- $17-$100+/mo
- Free Tier
- Free for everyone - Chat on web, desktop, and mobile with usage limits
Free
Free
Free for everyone
- Try Claude Chat on web, desktop, and mobile
- Search the web, create files, and run code
- Memory across conversations
- Usage limits reset on rolling five-hour session window
Pro
$20per month
$17/yr Save 93%
For everyday work
- Claude Code included
- More usage
- Hand off and schedule tasks
- Claude Dashboards (beta)
- More Claude models
- Claude in Chrome and Microsoft 365
Max 5x
$100per month
For people who work with Claude all day - 5x more usage than Pro
- Claude Code included
- Everything in Pro, plus
- 5x more usage than Pro per 5-hour session
- Higher output limits for all tasks
- Early access to advanced Claude features
- Priority access at high traffic times
Max 20x
$200per month
For people who work with Claude all day - 20x more usage than Pro
- Claude Code included
- Everything in Pro, plus
- 20x more usage than Pro per 5-hour session
- Higher output limits for all tasks
- Early access to advanced Claude features
- Priority access at high traffic times
Enterprise
Custom
For large businesses operating at scale
- Pricing is $20 per seat per month plus usage billed at API rates
- Billed annually
- Everything in Team plus advanced security and admin controls
- SCIM, audit logs, custom data retention, role-based access
View full pricing on anthropic.com →
Pricing may have changed since last verified. Check the official site for current plans.
Find your fit
Before you choose Bloom
More options in LLM Evaluation & Benchmarks. Compare pricing, platforms, and features side by side.
AgiRanker
Free · Open source
Researchers tracking verifiable AI capability trends
Pros
- Reproducible and targeted evaluations that quantify frequency and severity across automatically generated scenarios
- Evaluations correlate strongly with hand-labelled judgments and reliably separate baseline models from intentionally misaligned ones
- Researchers can extensively configure Bloom's behavior, through choosing models for each stage, adjusting interactions' length and modality
- Using Bloom evaluations took only a few days to conceptualize, refine and generate
- Integrates with Weights & Biases for experiments at scale and exports Inspect-compatible transcripts
Cons
- Bloom is only as robust as the seeds and judging logic that power it; teams should treat seeds as living governance artifacts, and for ambiguous or highly contextual behaviors, periodic manual review is still necessary
- Bloom's evaluation suite is unlikely to match the precise distribution of scenarios found in existing benchmarks, and since model behavior can be sensitive to context and prompt variations, direct comparisons are unreliable
About
- Platforms
- Python; integrates with Anthropic and OpenAI models via LiteLLM; supports Weights & Biases
- Languages
- Python
- API Available
- Yes
- Self-Hosted
- Yes
- Last Updated
- April 20, 2026
Best For
Who it's for
- Regression testing, release gating, and tracking mitigations over time
- AI safety and alignment research teams
- Studying narrow but critical risks that may be missed by broader evaluations
- Evaluating frontier AI models for specific behavioral traits
- Automating evaluation suite generation without manual engineering
What it does well
- Measuring behaviors like delusional sycophancy, long-horizon sabotage, self-preservation and self-preferential bias
- Regression testing, release gating, and tracking mitigations over time
- Investigating jailbreak susceptibility, self-preferential bias, and long-horizon sabotage risks
- Quantifying frequency and severity of target behaviors across generated scenarios
- Baseline model comparison and intentional misalignment detection
Integrations
Frequently Asked Questions
- Is Bloom free?
- Bloom offers a free option alongside paid upgrades. Pricing: $17-$100+/mo. Check the current allowances and terms before choosing a plan.
- Is Bloom open source?
- Yes. Bloom is open source — the source repository is at https://github.com/safety-research/bloom.
- Does Bloom have an API?
- Yes. Bloom exposes a developer API. See the official documentation at https://anthropic.com for details.
- Can I self-host Bloom?
- Yes. Bloom supports self-hosting on your own infrastructure.
- When was Bloom released?
- Bloom was first released in 2025.
- What platforms does Bloom support?
- Bloom is available on: Python; integrates with Anthropic and OpenAI models via LiteLLM; supports Weights & Biases.
People also compared
Curated lists that include this category
Anthropic's evaluation framework that auto-generates targeted test suites to measure specific AI behaviors at scale.
Bloom tackles a real gap in AI safety work: most evaluations are either broad (and miss edge cases) or manual (and don't scale). This tool generates reproducible test scenarios for narrow, high-stakes behaviors—like refusal consistency or prompt injection resilience—then scores models across thousands of permutations. The catch is structural: it's only as good as the seed prompts and judging logic you feed it, so teams still need to validate results manually for ambiguous cases. Free to use, but assumes you have both the domain expertise to set it up and the infrastructure to run it.
*Use this if you're a safety researcher or red-teaming group who needs systematic, repeatable measurements of specific model risks.*
Contribute
- Work on Bloom? Claim this listing.
- Embed a badge for your README or site.
Sign in to rate this tool or suggest an edit.