Self-Hosted RAG Frameworks
As of August 2026, AIDiveForge tracks 22 self-hosted rag frameworks. The top three by verified-data score are CMEM, PixelRAG, and Engramma Memory. Curated self-hosted rag frameworks tracked by AIDiveForge. Listings are verified against each tool's live website and re-checked regularly.
Last updated July 29, 2026 · 22 tools
Ranked by AIDiveForge's verified-data score: data completeness, verification recency, community rating, and real visitor engagement. How we rank · No tool can pay for placement.
1. CMEM
The open-source claude-mem engine hooks into Claude Code, Cursor, Windsurf, and CLI agents, writing decisions and dead ends into a local SQLite observations database as your agent works. CMEM Cloud mirrors that database behind a private MCP endpoint any agent or IDE can read, so the context one agent built in one session is available to the next one without manual handoff. Vector search over the observations store means retrieval is semantic, not keyword-based — you query by meaning, not by remembering what you typed three sprints ago. The ceiling appears at the team coordination layer: role-based read/write scoping and per-project isolation are paid-only features, so solo developers get the full engine but teams hit a paywall before they get the shared-brain behavior the product is built around.
PaidAPISelf-hostedVerified Jul 26, 2026
2. PixelRAG
PixelRAG is an open-source retrieval framework that indexes document pages as images and searches over them using vision-language models, so structure that defeats text extraction — column layouts, embedded charts, dense tables — stays intact through the retrieval step. The hosted API requires no key and the pip-installable package supports self-hosted deployments, which means teams can run it locally without routing data through external services. Where it fits cleanly: Wikipedia-scale visual QA and any RAG pipeline where the page's visual structure carries meaning the text alone loses. Where it breaks: the screenshot-per-page approach trades token efficiency gains on visual content against higher compute per retrieved chunk, and the evidence base for how it performs past Wikipedia-scale collections is thin. Teams pushing beyond the documented use cases are largely on their own.
FreeOpen SourceAPISelf-hostedVerified Jun 28, 2026
3. Engramma Memory
The library combines exact kNN search, Hopfield energy networks, and multi-head attention in a single local install, so agents can retrieve, pattern-complete, and generalize across stored knowledge without stitching together separate systems. The dependency surface is intentionally minimal — NumPy and nothing else — which means local prototyping adds no infrastructure overhead. The ceiling arrives when you move beyond a laptop: local mode has no persistence layer built for concurrent production writes, and the path to production runs through Engramma Cloud, a paid-only hosted backend. Teams scaling beyond local experiments will be evaluating that cloud offering rather than a self-managed stack.
PaidOpen SourceSelf-hostedVerified Jul 9, 2026
4. Mwe-MCP
The store lives on your own server as human-readable Markdown files, which means you can open a file, spot a hallucinated fact, and correct it without touching any agent code. Per-fragment access controls let you scope which agents or users can read or write each memory entry — so a household assistant and a work agent can coexist without leaking context across boundaries. The docs describe an overnight self-organizing pass that restructures the wiki without agent intervention. Where this breaks: teams expecting a managed cloud endpoint will find none — the vendor states AGPL self-hosted only. Standing up and maintaining the server is your problem.
FreeOpen SourceAPISelf-hostedVerified Jul 25, 2026
5. WeAura AI Agent
Aura pulls your Prometheus metrics, Kubernetes state, runbooks, and Git history into a single retrieval layer, then answers incident questions with citations pointing to the exact file and line that informed the response. When a Grafana alert fires, it correlates infrastructure state, classifies severity, deduplicates events, and can open a Jira ticket with a root-cause hypothesis attached — all before you have finished reading the alert body. Every suggested write operation stops for your approval before anything touches production. The self-hosted path runs via Helm with on-prem embeddings, which matters for teams whose data cannot leave their network. Teams with sparse runbook coverage or thin observability instrumentation will get proportionally thin answers.
PaidAPISelf-hostedVerified Jul 11, 2026
6. LightRAG
The tool indexes documents into both a vector store and a graph of entities and relationships, then queries both at retrieval time — so a question about how two concepts relate pulls connected nodes, not just cosine-similar text. Self-hosting is first-class: the repo ships Dockerfiles, a docker-compose stack, and Kubernetes manifests, so you are not routing data through an external API. The graph construction step is slower than plain vector indexing, and at document-collection scale that latency becomes a real scheduling concern. Community reports on the GitHub issue tracker (195 open issues) suggest the surface area for edge cases is wide, meaning teams moving beyond the examples folder should plan for debugging time. For multimodal or highly structured corpora the graph extraction quality depends heavily on the LLM you point at it.
FreeOpen SourceAPISelf-hostedVerified Jul 2, 2026
7. AgentRecall
AgentRecall is a memory layer that gives AI agents persistent context across sessions — so a support agent recalls a customer's past issue, a sales agent remembers where a deal stalled, and a coding assistant doesn't ask you to re-explain your architecture for the third time. The vendor describes a retrieval-and-storage infrastructure that indexes memories and surfaces relevant ones at query time, rather than stuffing the full conversation history into every prompt. The cloud tier caps at 1,000 stored memories, which is adequate for prototyping but a ceiling teams hit in production. Self-hosting under the MIT license removes that ceiling and keeps data inside your own infrastructure — the tradeoff is that you own the ops. API access covers JavaScript and Python environments.
Paid$9/month for Pro (cloud); self-hosted is freeAPISelf-hostedVerified Jun 1, 2026
8. ArXiv Scholar
ArXiv Scholar is an open-source RAG infrastructure that indexes roughly 5,600 curated AI engineering papers from arXiv and exposes them through a streaming API, so agents and developers can query verified literature instead of relying on a model's training memory. The retrieval pipeline runs a 1ms ML-based router that classifies each query as Direct, Decompose, or HyDE before spinning up hybrid dense-plus-sparse search and a cross-encoder re-ranker. Every answer ships with real arXiv paper IDs attached. The hard ceiling is the corpus: 5,600 papers covering RAG, LLMs, agents, training, and inference — nothing outside that domain, and nothing beyond what was ingested through the pipeline as of June 2026. The public endpoint is rate-limited to 5 requests per minute per IP, which breaks any agent loop that needs to fire queries in bursts.
FreeOpen SourceAPISelf-hostedVerified Jun 18, 2026
9. Cognita
An open-source RAG framework for building and deploying scalable retrieval-augmented generation applications.
FreeAPISelf-hostedVerified Apr 29, 2026
10. Deep Memory
The library pairs a GraphRAG implementation with a Vocabulary system: a shared, schema-enforced dictionary of node types, relationship labels, and property constraints that every agent queries before writing. The result is consistent graph data across sessions without prompting every agent with walls of example documents — the schema replaces the examples, trimming token overhead. Backends include Neo4j, SQL Server, Azure Cosmos DB, and an in-memory option, all wired up via Docker Compose quickstarts the docs describe. Where the ceiling appears: there is no hosted service, no GUI, and no API surface — this is a library you embed and operate, which means your team owns the infra from day one.
FreeOpen SourceSelf-hostedVerified Jun 11, 2026
11. Dify
Open-source LLM app development platform combining AI workflow, RAG pipeline, agent capabilities, model management, observability features and more.
Paid$59/moAPISelf-hostedVerified May 1, 2026
12. Elysia
An open-source framework that spins up an end-to-end agentic RAG application with just two terminal commands.
FreeAPISelf-hostedVerified May 1, 2026
13. HarvestGuard
The system fuses live satellite vegetation indices, rainfall anomaly data, and WFP food security indicators, then routes that combined signal through Claude to produce country-level crop failure risk assessments. Docker handles deployment; an Anthropic API key handles the inference. For an NGO standing up a proof-of-concept or a research institution prototyping AI plus Earth observation, the architecture is legible and the cost surface is clear — you pay for API calls, not a platform license. The wall appears when you need operational guarantees: this is a single-maintainer GitHub project with one star, no issue history, and no documented accuracy benchmarks against historical famine events. Teams that need auditable model provenance or SLA-backed uptime will hit that ceiling fast.
FreeOpen SourceAPISelf-hostedVerified Jun 9, 2026
14. Honcho
Every message written to Honcho triggers automatic reasoning via the vendor's Neuromancer model, which learns user psychology and behavioral patterns rather than just indexing text. The `context()` call returns a curated summary plus conversation history shaped to a token budget you set — the vendor claims 60–90% token reduction versus naive retrieval. Multi-participant sessions model each peer separately, so a group conversation doesn't collapse everyone's state into one blob. The ceiling appears when you need reasoning beyond user memory — Honcho does not run tasks, make decisions, or coordinate agents; it only informs them. Teams building full autonomous pipelines still wire Honcho into a separate orchestration layer.
PaidOpen SourceAPISelf-hostedVerified Jun 9, 2026
15. LanceDB
Open-source embedded vector database for multimodal AI with billion-scale search on Lance columnar format.
PaidAPISelf-hostedVerified Apr 22, 2026
16. LMCache
The library plugs into vLLM or TGI backends and stores KV cache tensors so that overlapping prompt prefixes — system prompts, document chunks, conversation history — are served from cache on subsequent requests. The vendor states 8–10x latency improvements for prompt caching workloads and 4–10x for RAG queries where the same document chunks appear across requests. The compression and streaming techniques described in the backing research (CacheGen, CacheBlend) are what make cache delivery fast enough to beat recomputation. The ceiling appears when your workload has little prompt overlap — unique user queries with no shared prefix — at which point the cache layer adds infrastructure without meaningful savings.
FreeOpen SourceSelf-hostedVerified Jun 18, 2026
17. Local RAG memory system
The server stores, retrieves, and versions memories using local ChromaDB, so context survives across sessions without touching any cloud service. You run it via Docker or Python, wire it into your MCP client once, and your assistant can recall preferences, project context, or past decisions on demand. Conflict detection flags when an incoming memory update collides with something already stored, so you are not silently overwriting context. The architecture fits solo developers and privacy-focused workflows well — it was built for exactly that. Where it strains: teams expecting multi-user memory sharing or production-grade scaling will find ChromaDB's local single-process model is not the right foundation.
FreeOpen SourceAPISelf-hostedVerified Jun 18, 2026
18. local-deep-research
The tool autonomously plans and executes multi-step research tasks: it queries sources, follows citations, synthesizes findings, and returns results with full attribution — all without a cloud handoff. The vendor reports ~95% on SimpleQA benchmarks using models like Qwen3-27B on a single RTX 3090, which gives you a concrete hardware target. It pulls from 10+ search backends including arXiv, PubMed, and private document collections. Where it breaks: running capable local models demands real GPU headroom, and teams without that hardware will either throttle to weaker models or route queries to cloud LLMs — at which point the privacy guarantee depends entirely on which cloud endpoint they configure. The 109 open issues and 210 open pull requests on GitHub signal an active but fast-moving codebase; production stability requires version pinning.
FreeOpen SourceAPISelf-hostedVerified Jun 9, 2026
19. Memori
The vendor states Memori classifies each chat turn into facts, preferences, rules, and summaries, then pulls targeted snippets at recall time rather than re-injecting full history. On the LoCoMo benchmark, the docs report 81.95% accuracy while cutting token usage by 95% versus full-context retrieval — a meaningful number if your cost problem is upstream of the model choice. The memory graph shows how entities connect across sessions, and every recall result ships with lineage explaining why that snippet was included, which matters when an enterprise audit asks why the agent said what it said. The ceiling appears when your retrieval logic needs fine-grained control the SDK's zero-configuration defaults don't expose — teams at that point are writing wrapper logic to compensate. Self-hosted deployment is available, so organizations with data-residency requirements are not locked into the cloud path.
Paid$19/monthAPISelf-hostedVerified Jun 9, 2026
20. MemoryOps
The project implements a full memory lifecycle — typed capture, policy evaluation before storage or retrieval, hybrid retrieval combining vector and structured lookup, deletion guarantees, and per-tenant isolation — as a self-hosted governance layer sitting between your AI assistant and whatever storage backends you run. Policy evaluation runs before memory is written or surfaced, which means you can enforce rules about what gets stored rather than cleaning up after the fact. The audit trail is first-class, not a log file bolted on. Where it breaks: this is a governance layer, not a memory provider with managed infrastructure, so your team owns the deployment, the backend wiring, and the operational burden. Early community adoption is thin — the repo reports five stars at the time of writing — which means when you hit an undocumented edge case, you are reading source code, not finding a Stack Overflow answer.
FreeOpen SourceSelf-hostedVerified Jun 22, 2026
21. OpenRAG
OpenRAG is a modular framework for exploring Retrieval-Augmented Generation (RAG) techniques, built for transparency and rapid experimentation to develop document-grounded AI systems—fully ready for production-scale deployment. It uses Ray to parallelize chunking, embedding, and ingestion across CPUs and GPUs, enabling fast, scalable processing of large document sets, and can be deployed seamlessly on Kubernetes for distributed, production-grade workloads. Advanced loaders like Docling and Marker parse complex layouts with OCR-enhanced PDFs, and chunk contextualization significantly boosts retrieval relevance. The platform ships with fully OpenAI-compatible chat API for seamless integration with tools like LangChain, OpenWebUI, or N8N—no adapter work required. Built-in clustering auto-generates synthetic QA datasets from your indexed documents, and a local LLM scores each query-chunk pair to help you tune retrieval before production. Two friction points surface at scale: in collaborative systems where documents update hourly, embeddings are recomputed every time by vLLM, which is computationally expensive, and admin users cannot grant access to partitions they were not explicitly given access to—the admin role does not override partition-level access restrictions.
FreeAPISelf-hostedVerified May 7, 2026
22. RAGFlow
Open-source RAG engine with deep document understanding, hybrid search, and agentic workflow orchestration.
PaidOpen Source$29/moAPISelf-hostedVerified May 15, 2026
Listings on this page are sourced and verified by the AIDiveForge data pipeline. AIDiveForge is editorially independent — no money changes hands for inclusion.