03 / ENGINEERING, WITH RECEIPTS
CUDA graph capture and fused operations keep the decode path close to the GPU. Here’s what one measured setup delivered.
1.25× the recorded Ollama throughput
on NVIDIA DGX Spark GB10.
Recorded March 27, 2026 on one NVIDIA DGX Spark GB10 with 128 GB unified memory. Fixed 8-word prompt, 128 generated tokens, greedy sampling, batch size 1, three-run median, fp32 compute and KV cache. Ollama version 0.17.7. The advantage narrows with model size; these results show parity around 3B–7B.
Zerfoo’s timer includes prefill; Ollama reports decode only. Zerfoo re-quantizes Q4_K weights to Q4_0, while Ollama uses Q4_K_M arithmetic. No quality-normalized comparison was performed, and Ollama’s model tags were unpinned. These are single-machine results, not independently reproduced or a claim about concurrent serving or other hardware.
Full methodology and limitations ↗04 / OPEN WEIGHTS. CLEAR EXPECTATIONS.
GGUF is the common format. Start with a model that has published runtime evidence, then explore the wider architecture registry.
Explore the verified-model matrix ↗Architecture registration is not model verification. Check the matrix for quantization, hardware, parity status, and known limitations.
05 / BUILT IN THE OPEN
Use the whole framework or the pieces you need. The ecosystem covers tensors, tokenization, numeric types, and model conversion.
MEET THE BUILDER
I’m David Ndungu. I build Zerfoo, and I help engineering teams run Claude Code and other coding agents safely in production. This is the engineering underneath that work, open for you to inspect.
Have a workflow in mind? Let’s talk ↗MEET THE BUILDER
I’m David Ndungu. I build Zerfoo, and I help engineering teams run Claude Code and other coding agents safely in production. This is the engineering underneath that work, open for you to inspect. It’s one piece of a broader practice — building and hardening the infrastructure agentic work runs on, including Ajent, where coding agents share what they find.
Have a workflow in mind? Let’s talk ↗A FEW GOOD QUESTIONS
Go developers who want to embed language-model inference in an application or run an OpenAI-compatible server on their own infrastructure. Start with a benchmarked model and validate your workload. The repository also contains training, tabular ML, and time-series packages with varying maturity.
No. CPU inference is available. CUDA acceleration requires compatible NVIDIA hardware and runtime libraries. Default Go builds do not need a C toolchain; optional GPU build tags can. See the GPU setup guide for requirements.
Zerfoo is released under the Apache 2.0 license. Review the license and its terms for your use. Model weights have their own licenses, separate from the framework.
No. GGUF is a file format, and each architecture needs a compatible implementation. The registry is broader than the verified-model set. Check the model matrix for evidence and known gaps before choosing a model.
Try the quickstart, reproduce a benchmark, or bring a real workload. Report an issue with your model, hardware, version, and reproduction steps. Contributions follow the repository’s contribution guide.
YOUR NEXT FEATURE STARTS HERE
Open source. Apache 2.0. Yours to build on.