Inference Engines & InfraFree
LM Studio
LM Studio is a free desktop app for local models with a headless server (llmster), local APIs and SDKs. We ran llmster on a CPU-only server…
Verified
View LM StudioCategory
Local Inference Runtimes
Local inference runtimes (Ollama, LM Studio, llama.cpp).
Inference Engines & InfraFree
MTPLX
The vendor states a 2.24× decode speedup on Qwen3-27B running on an M5 Max MacBook Pro, achieved by using the model's own built-in MTP…
VerifiedOSS
View MTPLX
Inference Engines & InfraFree
Atlas Inference Engine
The vendor page benchmarks Atlas at 3.1x the decode throughput of vLLM on Nvidia DGX Spark hardware — 111 tok/s average versus 37 tok/s on…
VerifiedOSS
View Atlas Inference Engine
Inference Engines & InfraFreemium
Cactus
Open-source inference engine for deploying AI models locally on mobile and edge devices with automatic cloud fallback.
VerifiedFreemium
View Cactus
Agent FrameworksFreemium
Thunderbolt
Open-source, self-hosted enterprise AI client emphasizing data sovereignty and model choice.
VerifiedOSSFreemium
View Thunderbolt
Inference Engines & InfraFree
OpenVINO™ Toolkit
Open-source toolkit for optimizing and deploying AI inference on Intel and multi-platform hardware.
VerifiedOSS
View OpenVINO™ Toolkit
Inference Engines & InfraFreemium
Ollama
Ollama downloads open models and runs them locally behind a command line and API, with optional cloud-hosted models. We ran it on a 3-core…
OSSFreemium
View Ollama