How do I configure GPU passthrough for virtual machines?

Configuring GPU passthrough for virtual machines (VMs) allows you to dedicate a physical GPU to a VM, enabling high-performance workloads like AI, machine learning, video rendering, or gaming. Here’s a step-by-step guide to configure GPU passthrough, applicable to popular hypervisors such as VMware ESXi, Proxmox VE, or KVM/QEMU.

Prerequisites:

Hardware Requirements:
A CPU and motherboard that support Input-Output Memory Management Unit (IOMMU). For Intel, this is called Intel VT-d, and for AMD, it’s AMD-Vi.
A compatible GPU that supports virtualization (e.g., NVIDIA Quadro, AMD Radeon Pro, or consumer GPUs with proper driver support).
Software Requirements:
A hypervisor that supports GPU passthrough (e.g., VMware ESXi, Proxmox VE, KVM/QEMU).
Appropriate drivers for the GPU installed on the guest VM.
Access:
Admin/root access to the hypervisor host.
Remote management tools like SSH or a web interface.
Backup:
Backup the configuration of the hypervisor and VMs before proceeding.

Steps to Configure GPU Passthrough:

1. Enable IOMMU in BIOS:

Reboot the server and enter BIOS/UEFI settings.
Look for options like VT-d (for Intel) or AMD-Vi (for AMD) and enable them.
Save and exit the BIOS configuration.

2. Configure Hypervisor Settings:

Depending on the hypervisor, enable the required features for GPU passthrough.

For VMware ESXi:
– Log in to the ESXi web interface.
– Go to Host > Manage > Hardware > PCI Devices.
– Select the GPU and mark it for passthrough.
– Reboot the ESXi host to apply changes.

For Proxmox VE:
– Edit the kernel boot parameters in /etc/default/grub to enable IOMMU:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on" # For Intel GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on" # For AMD
– Update GRUB and reboot:
update-grub reboot
– Verify IOMMU groups:
find /sys/kernel/iommu_groups/ -type l

For KVM/QEMU:
– Edit the kernel parameters in /etc/default/grub to enable IOMMU as above.
– Install and configure vfio-pci kernel module to isolate the GPU for passthrough.

3. Isolate the GPU:

Ensure the GPU is not being used by the hypervisor host itself.
For Proxmox or KVM, you may need to blacklist GPU drivers like nouveau (for NVIDIA) or radeon (for AMD):
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf update-initramfs -u reboot

4. Assign GPU to the VM:

Edit the VM configuration to assign the GPU hardware.

For VMware ESXi:
– Edit the VM settings.
– Add a new PCI device and select the GPU from the list of passthrough devices.

For Proxmox VE:
– Edit the VM configuration file /etc/pve/qemu-server/<VMID>.conf and add the following lines:
hostpci0: 01:00,pcie=1
Replace 01:00 with the actual PCI address of the GPU.

For KVM/QEMU:
– Use virt-manager or edit the VM XML configuration file to add the GPU as a PCI device:
xml <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0'/> </source> </hostdev>

5. Install Drivers on the Guest VM:

Boot the VM and install the appropriate GPU drivers within the guest operating system.
For NVIDIA GPUs, you may need to address limitations in consumer GPUs by patching the driver or using NVIDIA vGPU software (if licensed).

6. Test the GPU:

Run a GPU-intensive application or benchmark tool within the VM to ensure the GPU passthrough is working correctly.

Troubleshooting Tips:

Ensure IOMMU Groups:
Verify that the GPU and its associated audio device are in the same IOMMU group.
Fix Code 43 Error (NVIDIA GPUs):
NVIDIA consumer GPUs may block virtualization. Add the following parameter in the VM configuration:
<kvm> <hidden state='on'/> </kvm>
Check Logs:
Check hypervisor logs (/var/log/syslog, dmesg, or hypervisor-specific logs) for errors.
BIOS Updates:
Ensure your motherboard BIOS is up to date for better IOMMU support.

By following these steps, you should be able to configure GPU passthrough successfully. If you encounter any issues, consult your hypervisor documentation or vendor support for additional guidance.