TechSetupGuides
Intermediateproxmoxdebianvlanunifiwireguardfail2banssh

Proxmox VE 9 host setup for AI workloads

Install Proxmox VE 9, fix the default APT repos, harden network and SSH access, set up a dedicated backup disk, and create a scoped user for Claude Code. The foundation for everything else in the multi-agent AI stack.

  1. Step 1

    Overview and Goals

    This guide documents the end-to-end setup of a self-hosted AI workstation built on Proxmox VE, with an NVIDIA RTX 3090 passed through to a dedicated Ubuntu VM running Ollama for local LLM inference. Remote access is provided via WireGuard VPN and Claude Code over SSH. Goals of this stack:

    • Privacy Sensitive code and data never leaves the local network when routed to local models. Claude handles planning and hard reasoning via the API; repetitive or private tasks route to local models via an MCP tool.
    • Cost control Offload bulk and repetitive inference to free local compute. Use Claude API credits only for tasks that genuinely need frontier reasoning.
    • Full environment access Claude Code connects via SSH and has direct access to files, Docker containers, VMs, and the Proxmox host itself.
    • Work from anywhere The WireGuard VPN means the full homelab is accessible from any device with no local install beyond Claude Code.
    • Isolated environments Proxmox lets you run multiple sandboxed VMs with snapshots and easy rollbacks. The AI inference workload is fully isolated from the hypervisor. The flow when in use:
    • Laptop: Claude Code desktop app
    • Transport: WireGuard VPN → Tailscale tunnel
    • Proxmox host: Homelab box at 192.168.20.6
    • Inference VM: Ubuntu VM, RTX 3090 passthrough, Ollama
  2. Step 2

    Host Machine

    • CPU: Intel i7-8700K • 6 cores / 12 threads
    • RAM: 32GB DDR4 3200 MT/s
    • Motherboard: ASRock Z370 Extreme4
    • OS: Proxmox VE 9.1 (Debian Trixie)
  3. Step 3

    GPU

    • Card: NVIDIA GeForce RTX 3090
    • VRAM: 24GB GDDR6X
    • Bandwidth: 936 GB/s
    • Best for: Models up to ~32B parameters
    • Driver: 595.58.03
    • CUDA: 13.2
  4. Step 4

    Storage Layout

    Device Purpose Proxmox Storage ID Proxmox OS + VM pool (active) local / local-lvm AI model weights + scratch (500GB allocated to VM) model-storage ISO images + daily VM backups backup-hdd SATA SSDs — spare, not in use —

    nvme0n1
nvme1n1
sda (HDD)
sdb, sdc, sdd
  5. Step 5

    Overview

    • Router: Unifi Dream Router 7
    • Default LAN: 192.168.10.0/24
    • Homelab VLAN: 192.168.20.0/24 (VLAN ID 2)
    • Proxmox host IP: 192.168.20.6
    • Ollama VM IP: 192.168.20.110 (DHCP)
    • WireGuard server: HAOS on Pi 4 • 192.168.10.18
    • VPN subnet: 10.99.99.0/24
  6. Step 6

    VLAN Isolation

    The Proxmox host sits on a dedicated homelab VLAN (192.168.20.0/24), isolated from the main LAN. This means homelab VMs can’t reach household devices by default, but the main LAN can reach Proxmox for management.

  7. Step 7

    Unifi Firewall Rules (zone-based)

    Rule Direction Detail Homelab → Default LAN Connection state: Established, Related Default LAN → Homelab TCP ports 22, 8006 • New, Est., Related Homelab → Default LAN All traffic — Action: Block

    1. Allow established/related
2. Allow SSH + WebUI
3. Block all
  8. Step 8

    WireGuard VPN

    WireGuard runs as an official addon on Home Assistant OS (HAOS) on a Raspberry Pi 4 (8GB) at 192.168.10.18. This provides secure remote access to the full homelab network from anywhere.

    • WireGuard addon version: v0.13.0
    • VPN subnet: 10.99.99.0/24
    • The homelab VLAN (192.168.20.x) is reachable through the tunnel via a static route Static route added on the Proxmox host so return traffic from the homelab VLAN knows how to reach VPN clients: Note: This route is added permanently via /etc/network/interfaces or a post-up hook so it survives reboots.
    ip route add 10.99.99.0/24 via 192.168.20.1
  9. Step 9

    Why This Matters

    GPU passthrough requires the CPU to support hardware virtualisation and IOMMU (input–output memory management unit). IOMMU allows the hypervisor to assign a physical PCI device exclusively to a VM. Without it, the GPU cannot be isolated from the host and passed through.

  10. Step 10

    Required BIOS Settings

    • Intel Virtualisation Technology (VT-x) — enables KVM hypervisor
    • Intel VT-d (Virtualisation Technology for Directed I/O) — enables IOMMU
    • Disable CSM (Compatibility Support Module) — ensures pure UEFI boot
    • Disable Secure Boot — required for unsigned kernel modules (vfio-pci)
    • Disable Resizable BAR (ReBAR) / Above 4G Decoding if causing instability Note: On the ASRock Z370 Extreme4, VT-d is found under Advanced → CPU Configuration. CSM is under Boot → CSM.
  11. Step 11

    Verification After Boot

    After enabling VT-d and booting Proxmox, verify IOMMU is active: Note: Proxmox 9 / newer kernels enable IOMMU automatically when VT-d is on in BIOS. Adding intel_iommu=on iommu=pt explicitly to GRUB is still recommended for passthrough performance (iommu=pt = passthrough mode, reduces overhead for devices not being passed through).

    dmesg | grep -e IOMMU -e iommu | head -20
# Expected: devices being assigned to iommu groups
cat /proc/cmdline
# Should contain: intel_iommu=on iommu=pt
  12. Step 12

    Adding Kernel Parameters

    Enable IOMMU and passthrough mode at the kernel level by editing GRUB. After updating, run update-grub and reboot for the new boot parameters to take effect.

    nano /etc/default/grub
# Change:
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
# To:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
update-grub
reboot
  13. Step 13

    Install Details

    • Version: Proxmox VE 9.1
    • Kernel: 7.0.2-2-pve
    • Base OS: Debian Trixie
    • Hostname: pve.homelab.local
    • IP: 192.168.20.6/24
    • Gateway: 192.168.20.1
    • DNS: 192.168.20.1
    • Install disk: nvme0n1 (Samsung 980 PRO 2TB)
  14. Step 14

    Fixing APT Repositories (PVE 9 Specific)

    Proxmox VE 9 ships with only enterprise repositories enabled by default. These require a paid subscription and return HTTP 401 errors, blocking all package installation including sudo. PVE 9 uses the newer .sources format (deb822) rather than the old .list format.

  15. Step 15

    Fixing APT Repositories (PVE 9 Specific) — Step 1: Disable enterprise repos

    Step 1: Disable enterprise repos.

    # Disable enterprise PVE repo
nano /etc/apt/sources.list.d/pve-enterprise.sources
# Add at top: Enabled: no
# Disable enterprise Ceph repo
nano /etc/apt/sources.list.d/ceph.sources
# Add at top: Enabled: no
  16. Step 16

    Fixing APT Repositories (PVE 9 Specific) — Step 2: Add no-subscription repos

    Step 2: Add no-subscription repos.

    nano /etc/apt/sources.list.d/pve-no-subscription.sources
# Paste:
Types: deb
URIs: http://download.proxmox.com/debian/pve
Suites: trixie
Components: pve-no-subscription
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
nano /etc/apt/sources.list.d/ceph-no-subscription.sources
# Paste:
Types: deb
URIs: http://download.proxmox.com/debian/ceph-squid
Suites: trixie
Components: no-subscription
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
  17. Step 17

    Fixing APT Repositories (PVE 9 Specific) — Step 3: Update

    Note: The Ceph no-subscription repo points to ceph-squid specifically. If Proxmox ships a newer Ceph version in future, this suite name will need updating.

    apt update
# The keyring file (proxmox-archive-keyring.gpg) is pre-installed on PVE.
# No signing errors expected.
  18. Step 18

    Why Backups Before Anything Else

    VM backups to a separate physical disk provide disaster recovery that snapshots cannot — snapshots live on the same disk as the VM and are lost if that disk fails. The HDD is configured as a Proxmox backup target before any VMs are created.

  19. Step 19

    Identify the HDD

    Find the disk that will hold the backup target. The lsblk output shows every block device with its size and model — pick the spinning disk that's not in use elsewhere.

    lsblk -o NAME,SIZE,TYPE,MOUNTPOINT,MODEL
# Look for the HDD by size and model
# In this build: sda (Toshiba 1.8T)
  20. Step 20

    Partition, Format, and Mount

    Wipe the HDD, create a single GPT partition, format it ext4, and mount it permanently at /mnt/backups via /etc/fstab. Use the partition UUID rather than the device name so the mount survives if the kernel renumbers devices on reboot.

    # Install parted (not present by default on PVE 9)
apt-get install -y parted
# Wipe and partition
wipefs -a /dev/sda
sgdisk --zap-all /dev/sda
parted /dev/sda mklabel gpt
parted /dev/sda mkpart primary ext4 0% 100%
mkfs.ext4 /dev/sda1
# Mount permanently
mkdir -p /mnt/backups
blkid /dev/sda1   # note the UUID
echo 'UUID=<your-uuid>  /mnt/backups  ext4  defaults  0  2' >> /etc/fstab
mount -a
systemctl daemon-reload
df -h /mnt/backups  # verify
  21. Step 21

    Register with Proxmox

    Tell Proxmox about the new directory so it appears as a storage option for VM backups in the web UI.

    pvesm add dir backup-hdd --path /mnt/backups --content backup
pvesm status  # backup-hdd should show active
  22. Step 22

    Schedule Daily Backups

    In the Proxmox web UI: Datacenter → Backup → Add

    • Storage: backup-hdd
    • Schedule: 02:00
    • Mode: Snapshot
    • Selection: All VMs
    • Retention: Keep Last 5
    • Compression: ZSTD
  23. Step 23

    Why a Dedicated User

    Claude Code connects to the Proxmox host via SSH. Rather than giving it root or your personal account, a dedicated claude-code user with scoped sudo permissions limits blast radius, provides an audit trail, and makes access easy to revoke.

  24. Step 24

    Install sudo

    sudo is not installed on Proxmox VE by default. Install it after fixing the APT repos:

    apt-get install -y sudo
  25. Step 25

    Create the User

    Create the dedicated claude-code system user and lock its password — login will be SSH-key only.

    useradd -m -s /bin/bash claude-code
passwd -l claude-code  # disable password login (SSH key only)
  26. Step 26

    Sudoers Rule

    Grant passwordless sudo access only to the specific commands Claude Code needs. Never grant ALL=(ALL) NOPASSWD: ALL.

    visudo -f /etc/sudoers.d/claude-code
# Paste the following:
# Proxmox VM management
claude-code ALL=(ALL) NOPASSWD: /usr/sbin/qm, /usr/sbin/pvesh, /usr/sbin/pvesm
# Package management
claude-code ALL=(ALL) NOPASSWD: /usr/bin/apt-get, /usr/bin/apt
# Network and kernel module config
claude-code ALL=(ALL) NOPASSWD: /usr/sbin/modprobe, /usr/bin/tee /etc/modprobe.d/*
# Systemd service management
claude-code ALL=(ALL) NOPASSWD: /usr/bin/systemctl
  27. Step 27

    SSH Overview — Three Hops

    There are three SSH relationships to configure:

    • Laptop → Proxmox host (claude-code user) — used by Claude Code desktop to connect to the homelab
    • Proxmox host → Ollama VM (claude-code user → ollama user) — used by Claude Code when it needs to run commands inside the VM
    • Direct access → Ollama VM (for manual administration)
  28. Step 28

    Hop 1: Laptop → Proxmox Host

    Generate a dedicated key pair on your laptop: Set correct permissions on the Proxmox host: Add SSH config entry on your laptop (~/.ssh/config):

    ssh-keygen -t ed25519 -C 'claude-code-homelab' -f ~/.ssh/claude_homelab
# Copy public key to Proxmox host
ssh-copy-id -i ~/.ssh/claude_homelab.pub claude-code@192.168.20.6
# Windows: manually append instead
# cat ~/.ssh/claude_homelab.pub
# Then on Proxmox host as root:
# mkdir -p /home/claude-code/.ssh
# nano /home/claude-code/.ssh/authorized_keys  (paste key)
chmod 700 /home/claude-code/.ssh
chmod 600 /home/claude-code/.ssh/authorized_keys
chown -R claude-code:claude-code /home/claude-code/.ssh
Host homelab
    HostName 192.168.20.6
    User claude-code
    IdentityFile ~/.ssh/claude_homelab
    IdentitiesOnly yes
  29. Step 29

    Hop 2: Proxmox Host → Ollama VM

    Claude Code runs as the claude-code user on the Proxmox host. When it needs to execute commands inside the Ollama VM, it SSHes from there to the VM. This requires a second key pair generated on the Proxmox host itself. Contents of /home/claude-code/.ssh/config:

    # On the Proxmox host, switch to the claude-code user
su - claude-code
# Generate a key pair for the VM hop
ssh-keygen -t ed25519 -C 'claude-code-to-ollama' -f ~/.ssh/ollama_vm
# Copy the public key to the Ollama VM
ssh-copy-id -i ~/.ssh/ollama_vm.pub ollama@192.168.20.110
# Add SSH config so the alias works
nano ~/.ssh/config
Host ollama-vm
    HostName 192.168.20.110
    User ollama
    IdentityFile ~/.ssh/ollama_vm
    IdentitiesOnly yes
  30. Step 30

    Verify Both Hops

    Test the SSH key chain end-to-end: laptop → Proxmox (claude-code) → Ollama VM (ollama). The Proxmox command should run without a password prompt; the VM hop should show working GPU and Ollama.

    # From your laptop — test Proxmox access
ssh homelab
sudo qm status 100   # should work without password prompt
sudo pvesm status
# From Proxmox host as claude-code — test VM access
su - claude-code
ssh ollama-vm
nvidia-smi          # confirms GPU is up inside VM
ollama list         # confirms Ollama is running
  31. Step 31

    Claude Code Desktop SSH Connection

    In Claude Code Desktop, Code tab → Select folder → SSH:

    • Host: 192.168.20.6 (or the alias: homelab)
    • User: claude-code
    • Identity file: ~/.ssh/claude_homelab
    • Folder: ~ (home directory, where CLAUDE.md lives) Note: Claude Code installs itself on the remote machine automatically on first connection. No manual install needed on the Proxmox host.
  32. Step 32

    Purpose

    Fail2ban monitors log files and automatically bans IP addresses that show signs of brute-force attacks. On this homelab, it protects both SSH access and the Proxmox web UI (port 8006).

  33. Step 33

    Install

    Install fail2ban from the standard Ubuntu/Debian repos.

    apt-get install -y fail2ban
  34. Step 34

    Configure

    Never edit jail.conf directly — it is overwritten on updates. Use a drop-in file for the Proxmox jail: Note: Proxmox VE uses systemd journald and has no /var/log/syslog or /var/log/daemon.log by default. The backend=systemd setting is required. The sshd jail is automatically configured by fail2ban via the default jail.conf — no additional config needed.

    # Create Proxmox-specific jail (backend=systemd since PVE uses journald, not syslog)
cat > /etc/fail2ban/jail.d/proxmox.conf << 'EOF'
[proxmox]
enabled  = true
port     = https,http,8006
filter   = proxmox
backend  = systemd
maxretry = 5
bantime  = 1h
EOF
# Create the Proxmox filter
nano /etc/fail2ban/filter.d/proxmox.conf
# Paste:
[Definition]
failregex = pvedaemon\[.*\]: authentication failure; rhost=<HOST> user=.* msg=.*
journalmatch = _SYSTEMD_UNIT=pvedaemon.service
ignoreregex =
systemctl enable fail2ban
systemctl restart fail2ban
# Verify
fail2ban-client status
# Expected: Jail list: proxmox, sshd
fail2ban-client status proxmox
fail2ban-client status sshd
  35. Step 35

    Next: continue building the stack

    With this layer in place, the next guide in the series is NVIDIA GPU passthrough to an Ubuntu VM on Proxmox. Pass an RTX 3090 through to a dedicated Ubuntu 24.04 VM using VFIO. Covers IOMMU verification, driver blacklisting, VM creation strategy (install OS first, attach GPU after), and a snapshot discipline tailored to PCI-passthrough VMs.

Feature requests

Sign in to suggest features or vote on existing ones.

No feature requests yet.

Discussion

0 people marked this as worked·Sign in to mark your own.

Sign in to join the discussion.

No comments yet.