Setting Up the Server for SR-IOV - Setting Up the Server for SR-IOV - UG1739

Configuring the BIOS

To use SR-IOV modes, SR-IOV must be enabled in the server BIOS. The actual BIOS setting can differ between servers, but might be identified as SR-IOV or IOMMU, or as VT-d and VT-x on an Intel server.

There might be other BIOS options which should be enabled to support SR-IOV. For example on many DELL servers the following BIOS option must also be enabled:

Integrated Devices, SR-IOV Global Enable

Configuring the Kernel

You must explicitly enable the IOMMU by appending the relevant option to the kernel line in the /boot/grub/grub.conf file:

• On an AMD server the option is amd_iommu=on.
• On an Intel server the option is intel_iommu=on.

AMD recommends that you also enable the pci=realloc kernel parameter in the /boot/grub/grub.conf file. This allows the kernel to reassign addresses to PCIe apertures (such as bridges or ports) in the system when the BIOS does not allow enough PCI apertures for the maximum number of supported VFs.

Re-Mapping Interrupts for VF Passthrough

To use PCIe VF passthrough, the server must support interrupt re-mapping. If the target server does not support interrupt re-mapping you must set the following option in a user-created file (for example kvm_iommu_map_guest.conf) in the /etc/modprobe.d directory:

options vfio_iommu_type1 allow_unsafe_assigned_interrupts=1

Using Alternative Routing-ID Interpretation (ARI)

The ARI extension to the PCI Express Base Specification extends the capacity of a PCIe endpoint by increasing the number of accessible functions (PF and VF) from 8, up to 256. This is a feature of the server hardware and BIOS:

• Without ARI support , a server hosting a virtualized environment is limited to 8 functions.
• With ARI support, X4 series adapters can expose up to 240 VFs per adapter.

You should consult the documentation from your server vendor to ensure that the host server supports ARI, and to enable it.