Practical notes from running local models and coding agents. Each article explains the observed result, the conditions that limit it, and a next step you can try.
Field notes
- Ollama vs llama.cpp: a paired test with the same 3B model
Our controlled pair showed modest generation differences and different process-memory and Docker-image footprints. It did not establish a universal runtime winner.
- How much RAM did our 3B local model actually use?
We observed 1.6–2.2 GB in the model processes running one 3B GGUF. Those numbers are not whole-system RAM requirements or a recommendation for a particular laptop.
- OpenCode with a 3B model: check the files, not just the exit status
OpenCode created the requested file in all three small-model attempts, but passed neither of the other tasks. Its hosted-model rerun passed all nine attempts.
- An offline local-AI checklist from the requests our tools attempted
Several prepared tools continued working while external hosts were blocked. That does not make a clean install offline-ready or prove a tool never contacts the network.
- LocalAI vs llama.cpp: the differences our paired test could support
The generation results were close enough to treat as noise on our shared VM. Startup boundaries and LocalAI's separately installed backend mattered more to the operational comparison.
- LM Studio cold starts: the load command and engine timed different things
Our lms load command took 44.6 seconds while the engine reported 8.1 seconds loading the model. The two timers measured different boundaries.
- A 1.2 GB image is not a 1.2 GB local-AI installation
Our runtime images ranged from about 1.2 to 9.4 GB, but image size excluded other components. Treat the image, model files, additional backends and running memory as separate…
- Ollama in Docker: check the thread count before changing hardware
In our three-core container, Ollama started eight threads. Explicitly matching the thread setting to the available cores changed generation time substantially.
- Requesting 64K context did not make our local model run at 64K
Ollama capped our requested 65,536-token context at 32,768 for qwen2.5:3b. The effective setting mattered when interpreting the agent results.
- Goose and Hermes: the model changed our task-test results
Goose completed five of nine attempts with our small local model and Hermes completed none. With the recorded hosted model, both completed all nine. The model and context…