Inference Engines & InfraFree
MTPLX
The vendor states a 2.24× decode speedup on Qwen3-27B running on an M5 Max MacBook Pro, achieved by using the model's own built-in MTP…
VerifiedOSS
View MTPLXLocal Inference Runtimes
Local inference runtimes (Ollama, LM Studio, llama.cpp).
Inference Engines & InfraFree
Atlas Inference Engine
The vendor page benchmarks Atlas at 3.1x the decode throughput of vLLM on Nvidia DGX Spark hardware — 111 tok/s average versus 37 tok/s on…
VerifiedOSS
View Atlas Inference Engine
Inference Engines & InfraFreemium
Cactus
Open-source inference engine for deploying AI models locally on mobile and edge devices with automatic cloud fallback.
VerifiedFreemium
View Cactus
Agent FrameworksFreemium
Thunderbolt
Open-source, self-hosted enterprise AI client emphasizing data sovereignty and model choice.
VerifiedFreemium
View Thunderbolt
Inference Engines & InfraFree
OpenVINO™ Toolkit
Open-source toolkit for optimizing and deploying AI inference on Intel and multi-platform hardware.
Verified
View OpenVINO™ Toolkit
Inference Engines & InfraFreemium
Ollama
Ollama downloads open-source models like Llama 2 and Mistral and runs them on your own hardware—no API calls, no subscriptions, no data…
OSSFreemium
View Ollama