Ollama Local LLM Benchmark

Serial execution benchmark suite for local LLM evaluation with real-time system monitoring via Netdata.

Overview

This benchmark evaluates 6 large language models running locally via Ollama, measuring:

• Cold load performance
• Time to first token (TTFT)
• Generation throughput (tokens/sec)
• System resource utilization (CPU, GPU, VRAM, Temperature)
• Response quality metrics

Tested Models

Model	Family	Parameters	Quantization	Tokens/sec	Duration
gemma4:26b-a4b-it-q4_K_M	Gemma4	26B	Q4_K_M	70.44	109s
gemma4:31b-it-q4_K_M	Gemma4	31B	Q4_K_M	17.61	536s
qwen3.5:35b-a3b-q4_K_M	Qwen3.5	35B	Q4_K_M	63.90	242s
qwen3.6:35b	Qwen3.6	35B	Full	64.97	174s
qwen3.6:27b-q4_K_M	Qwen3.6	27B	Q4_K_M	19.25	588s
qwen3.6:27b	Qwen3.6	27B	Full	19.40	372s

System Specifications

• OS: Unraid OS 7.3.0
• Kernel: Linux 6.18.29-Unraid
• CPU: 32 cores
• RAM: 128 GB
• GPU 0: NVIDIA (Active)
• GPU 1: NVIDIA (Secondary)
• Ollama: Running on 192.168.1.100:11434
• Netdata: v2.10.0-255-nightly for monitoring

Key Findings

Performance Winners

1. qwen3.6:35b (full precision) - 64.97 tokens/sec
2. qwen3.5:35b-a3b-q4_K_M (quantized) - 63.90 tokens/sec
3. gemma4:26b-a4b-it-q4_K_M (quantized) - 70.44 tokens/sec (best TPS, fewer tokens)

Observations

• Qwen models show better quantization efficiency than Gemma models
• Full precision models can outperform quantized variants in some cases
• GPU temperature peaked at 85°C during intensive loading
• gemma4:31b was surprisingly slow at only 17.61 tokens/sec despite Q4 quantization

Resource Utilization Summary

Metric	Min	Max	Average
GPU0 Utilization	1%	100%	46.7%
GPU0 Temperature	51°C	85°C	71°C
GPU0 Power	23W	189W	95W
GPU0 VRAM	1.1GB	14.1GB	2.3GB
System Load	9.37	26.29	14.31
RAM Used	832MB	7.9GB	2.5GB

Files

• index.html - Interactive benchmark results dashboard
• data/ - Raw benchmark data and metrics
• results/ - Processed results and summaries

Usage

View the benchmark results:

# Open in browser
open index.html

Data Collection

Metrics were collected using Netdata API at 2-second intervals during the entire benchmark run:

• GPU utilization, temperature, power draw
• VRAM usage for both GPUs
• CPU utilization and system load
• RAM usage

License

MIT License - Feel free to use this benchmark suite for your own testing.

Author

Created by Shlomi G as part of the GoldyAI project.

---

🔗 Live Demo

View the interactive benchmark results: 👉 Live Demo

(Deployed on Vercel)