AIDiveForge field notes · Evidence recorded October 15–16, 2026
Our controlled pair showed modest generation differences and different process-memory and Docker-image footprints. It did not establish a universal runtime winner.
What we held constant
We compared Ollama and llama.cpp back to back using the same qwen2.5:3b Q4_K_M model, three CPU cores and a 4,096-token context. The host was a shared CPU-only cloud VM. The recorded versions were Ollama 0.40.5 and llama.cpp build 11681, commit 5e03a1ff4. These are the versions tested in October 2026, not a claim about the newest releases.
We explicitly matched the thread settings to the CPU allocation. Otherwise this experiment would mix a default-threading problem with a runtime comparison. We report generation speed as a ratio because a shared VM is not a dependable source of transferable tokens-per-second figures.
The measured pair
| Observation | Ollama | llama.cpp |
|---|---|---|
| First answer from a fresh server; model file not cached | 7.9 s | 8.7 s |
| Generation rate, 53-token prompt | 1.07x | 1.00x |
| Generation rate, 441-token prompt | 1.05x | 1.00x |
| Generation rate, 723-token prompt | 1.17x | 1.00x |
| Model-process memory | 2.2 GB | 1.6 GB |
| Docker image size in this setup | 9.37 GB | 1.21 GB |
How to interpret a small lead
Ollama generated faster in these three prompt-length cases, but the differences were much smaller than the large thread-setting effects in our separate diagnostics. A five-percent difference on a shared host should not decide an architecture by itself. Repeat the pair on the hardware and concurrency level that matter to you, and record CPU steal while it runs.
The startup row includes reaching a first answer from a fresh server. It is not interchangeable with a runtime's internal model-loading timer or an application's command duration. The llama.cpp record also contained a much slower first-ever cold start; its later cold starts were shorter. A single startup observation can therefore be a poor description of the experience after installation.
A decision based on operating requirements
Choose what to test next based on the interface you need. If an existing application already talks to your Ollama installation, the measured difference here does not by itself justify rebuilding that integration. If you are constructing a deliberately small server image, inspect the llama.cpp deployment and the full set of components it requires. Those are operating choices, not claims that either tool is better in every setting.
Keep image bytes, model files and running memory as separate entries in your capacity plan. A smaller image does not imply a proportionally smaller model process, and a model-process measurement does not tell you whether your whole machine has enough memory. Our companion memory guide explains what was and was not counted.
A useful reproduction checklist
The outcome to carry forward is a short, inspectable comparison under declared conditions. These measurements do not cover GPU performance, long-context workloads, multi-user serving or the quality of the model's answers.
- Use the same model file or verify that the packaged model is genuinely equivalent.
- Fix CPU limits, thread counts, context, output limit and request concurrency.
- Run alternating pairs with identical prompts and record generated-token counts.
- Keep cold-start and warm-generation experiments separate.
- Save versions, raw responses and host-load observations before writing a recommendation.
Test conditions and sources
8-vCPU AMD EPYC 9354P cloud VM, each test limited to 3 cores and 10 GB RAM, no GPU, no internet (isolated Docker network). Local-model runs blocked external access. Hosted-model reruns explicitly allowed the Anthropic API. The machine was shared; generation ratios apply to their paired sessions and do not predict GPU or other-machine performance.
These are dated observations, not a claim about the latest release or every machine. Recommendations and example procedures are our interpretation of the recorded results.