Ollama on a dedicated NVMe with performance tuning

This guide assumes you've already finished NVIDIA GPU passthrough to an Ubuntu VM on Proxmox. That guide covers the underlying setup this one builds on. If you're picking up mid-stack, scan its 'Overview' step to confirm your environment matches.

Dedicate a fast NVMe to Ollama's model store. The original build used a second 1.8 TB NVMe at /dev/nvme1n1 — substitute your own device path throughout.

If the drive has any existing partitions or filesystems, deal with them first using your preferred tools (lvremove / vgremove / pvremove for LVM, mdadm --zero-superblock for old mdraid, etc.) before wiping.

Add a 500 GB virtual disk backed by the model-storage NVMe. Directory-backed storage doesn't support snapshots — exclude this disk from snapshots and backups explicitly so neither operation fails on the VM.

Inside the VM, format the new virtual disk ext4 and mount it permanently at /mnt/models. This is where all Ollama model weights will live.

Run the official Ollama install script. It detects the NVIDIA GPU automatically and installs a systemd service that starts on boot.

By default, Ollama stores models in ~/.ollama. Override this to use the dedicated 500GB NVMe:

Three systemd drop-in files configure Ollama performance. All are created at /etc/systemd/system/ollama.service.d/ (system service, not user service): override.conf — OLLAMA_MODELS path (created during initial setup) Effect of each setting: kv-cache.conf: OLLAMA_KV_CACHE_TYPE=q8_0 stores the KV cache in 8-bit instead of fp16, roughly halving KV VRAM usage with negligible quality loss. Critical for fitting 32K+ context on 24GB. performance.conf: OLLAMA_FLASH_ATTENTION=1 enables FlashAttention for 10–30% throughput improvement. OLLAMA_KEEP_ALIVE=20m keeps models loaded for 20 minutes after the last request. The shorter timeout helps when multiple runtimes share the GPU — VRAM is freed quickly so a different agent can load its model without waiting. Note: deepseek-r1:32b is capped at 12K context. At 32K the model weights (19 GB) plus KV cache exceed 24 GB VRAM and OOM. Set num_ctx 12288 in its Modelfile.

All models use Q4_K_M quantization unless noted and are configured with the context windows below via Modelfile. The RTX 3090 has 24GB VRAM — leave at least 2–3GB headroom and do not load multiple large models simultaneously. Model Use Context VRAM Notes Sanity check / GPU test 32K 1.3 GB Q8_0 Default / coding / tool calls 32K 18 GB MoE, ~147 tok/s General reasoning 64K 17 GB Q4_K_M, thinking adaptive Deep reasoning 12K 19 GB Q4_K_M, avoid tool-use, ~38 tok/s Uncensored / fast 32K 5 GB Q4_K_M, ~149 tok/s Uncensored reasoning 64K 17 GB Q4_K_M

When routing tasks from Claude Code to local models via the Ollama MCP server, use this as a guide:

• Default local route: qwen3-coder:30b — best tool calling reliability, RL-trained on SWE-bench
• General reasoning: qwen3.5:27b — use for non-coding tasks; supports adaptive thinking
• Complex reasoning: deepseek-r1:32b — capped at 12K context (OOMs at 32K); avoid for tool-use tasks
• Uncensored general: dolphin3
• Uncensored reasoning: huihui_ai/qwen3.5-abliterated:27b
• Quick tests: llama3.2:1b Note: Route to uncensored models only when content filters on standard models are blocking a legitimate task, for creative writing with mature themes, or security research/red-teaming. Never use as default.

Each model requires a Modelfile to set 32K context. The pattern for each:

Verify Model Storage Location.

With this layer in place, the next guide in the series is Self-host OpenClaw with HTTPS, Brave search, and GitHub access. Run the OpenClaw agent gateway on your Ollama VM behind an nginx TLS reverse proxy. Wire up the Brave search skill, give it authenticated GitHub access for code tasks, and configure a CLAUDE.md so remote Claude Code sessions have persistent context.