Proxmox Network Adapter Pass Through

Long time ago, I konw virtualization is go very good solution to server. But how about router build in virtualization? After tried to do it, I was failed. because virtual switch made networking very slow. So i just use dedicate server as a router.

Recently days, I just learn proxmox and find pci-e pass through function. I think maybe it is way to make routing virtualization.

Here are what I do:

Enable VT-d and X2APic Mode

Configure BIOS to enable VT-d and X2APic

Chang Grub boot settings

Inel CPU

Edit grub configure.
Inter CPU
Edit grub
Edit /etc/default/grub, then change GRUB_CMDLINE_LINUX_DEFAULT="quiet" to GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
Update grub

update-grub

AMD CPU

Edit grub file.
Edit /etc/default/grub
Change GRUB_CMDLINE_LINUX_DEFAULT="quiet" to GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
Update grub
update-grub

##Change Required modules

Edit /etc/modules

Add

1
2
3
4
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Check IOMMU enable or not

1
dmesg | grep ecap

script to make sure IOMMU is enabled

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
#!/bin/sh
if [ $(dmesg | grep ecap | wc -l) -eq 0 ]; then
    echo "No interrupt remapping support found"
    exit 1
fi

for i in $(dmesg | grep ecap | awk '{print $NF}'); do
    if [ $(( (0x$i & 0xf) >> 3 )) -ne 1 ]; then
        echo "Interrupt remapping not supported"
        exit 1
fi
done

Verify IOMMU isolation

1
find /sys/kernel/iommu_groups/ -type l

Add PCI-E Device to VM

Run lspci to check address of network pci-e adapter device. like this

1
lspci

1
hostpci0: 01:00.0

Add configure of yoru VM /etc/pve/qemu-server/<vmid>.conf
Add configure like this to set pci-expres passthrough.

1
2
  machine: q35
  hostpci0: 00:00.0,pcie=1

Reference

Pci passthrough