AIDiveForge field notes · Evidence recorded October 15–16, 2026
The generation results were close enough to treat as noise on our shared VM. Startup boundaries and LocalAI's separately installed backend mattered more to the operational comparison.
A second paired session, not a combined league table
This comparison used the same GGUF model file, three CPU cores and a 4,096-token context for LocalAI and llama.cpp. The tools were run back to back on October 16. LocalAI used its own llama-cpp backend. The llama.cpp measurements in this pair came from a second run that day; they are not the llama.cpp measurements in the separate Ollama comparison.
That distinction matters. Copying one number from each session into a single ranking would hide changes in host conditions. CPU steal was 12.8 percent during this llama.cpp run and 11.5 percent during LocalAI's run. We treated differences under about ten percent as within the noise for this session.
The recorded observations
| Observation | llama.cpp | LocalAI |
|---|---|---|
| First answer from a fresh server; model file not cached | 3.5 s | 6.4 s |
| Generation rate, 53-token prompt | 0.98x | 1.00x |
| Generation rate, 441-token prompt | 0.96x | 1.00x |
| Generation rate, 723-token prompt | 0.99x | 1.00x |
| Model-process memory | 1.6 GB | 1.7 GB |
| Docker image size before LocalAI's separate backend | 1.21 GB | 1.20 GB |
Why we did not award a speed winner
The generation ratios are close to one. A two- or four-percent gap does not support a broad speed recommendation under the observed host conditions. Prompt processing also varied with prompt length: one short-prompt result favored llama.cpp, while the longer cases were close. Combining those into one unlabeled speed number would answer a question we did not test.
The startup observation is more concrete, but still narrow. It measures server start to a first answer under the recorded cold condition. It does not establish which deployment will respond fastest after a restart on your own storage or when serving a GPU-backed model. The repeatability of the boundary is as important as the number.
Include the backend in an installation plan
In our setup, LocalAI's main Docker image did not contain the engine that ultimately ran the model. We installed its llama-cpp backend separately. The recorded image pull took about thirteen seconds and the backend installation about four seconds on that server's connection. Those are dated transfer observations, not download-time promises.
The log warned that the backend was installed without signature verification. We report the warning because it was part of this installation, not as a general statement about every LocalAI package. For a controlled deployment, pin the intended image and backend, record their provenance and test the exact offline startup procedure before depending on it.
The practical decision
If your application needs LocalAI's surrounding service interface, measure the complete service against that requirement. If you need only the narrower llama.cpp server, evaluate the smaller operational arrangement directly. This benchmark supports investigating those deployment differences; it does not support replacing a service because of a few percent of generation-rate variation.
Our next test would use a dedicated host and the actual concurrency, prompt mix and model you intend to serve. Until then, keeping the setup reproducible is more useful than turning noisy ratios into a ranking.
Test conditions and sources
8-vCPU AMD EPYC 9354P cloud VM, each test limited to 3 cores and 10 GB RAM, no GPU, no internet (isolated Docker network). Local-model runs blocked external access. Hosted-model reruns explicitly allowed the Anthropic API. The machine was shared; generation ratios apply to their paired sessions and do not predict GPU or other-machine performance.
These are dated observations, not a claim about the latest release or every machine. Recommendations and example procedures are our interpretation of the recorded results.