LLM CPU Benchmarks

Products/LLM CPU BenchmarksPublic · OpenOpen the benchmarks

Tool-calling accuracy, throughput, and memory for small (~4B) open-weight LLMs running on a commodity x86 CPU box. Qwen3.5-4B · Gemma-4-E4B-it · Phi-4-mini, each measured with standard and TurboQuant KV-cache compression.

Live site: deemwar-products.github.io/llama-cpu-benchmarks·Findings article·Results table·API JSON

What's measured

Apples-to-apples

Three modern ~4B tool-calling models on the same CPU box, with the same harness, the same scenarios, the same warm-up budget. The numbers are comparable.

Standard vs TurboQuant

Every model is measured with default KV cache and again with TurboQuant compression — so you can see what compression actually costs in tool-calling accuracy.

CPU-only target

No GPU. Commodity x86. The benchmark exists for teams who need on-prem, air-gapped, or budget-constrained deployments — the same teams Deemwar builds for.

Methodology

BFCL test harness adapted for tool-calling accuracy on a fixed scenario set
Per-cell driver scripts so every run is reproducible from a single command
Standard vs TurboQuant KV-cache compression measured side by side
Throughput, memory footprint, and tool-call success rate captured per run
Aggregate summary published as static JSON — wire it into your own dashboards

Need a custom CPU-LLM benchmark?

Open the live results, or talk to us about running the same harness against your target hardware, quantisation, or tool surface.