Receive Side Scaling (RSS)

X4 series adapters support Receive Side Scaling (RSS). RSS enables packet receive-processing to scale with the number of available CPU cores. RSS requires a platform that supports MSI-X interrupts. RSS is enabled by default.

When RSS is enabled the controller uses multiple receive queues to deliver incoming packets. The receive queue selected for an incoming packet is chosen to ensure that packets within a TCP stream are all sent to the same receive queue. This ensures that packet-ordering within each stream is maintained. Each receive queue has its own dedicated MSI-X interrupt which is ideally tied to a dedicated CPU core. This allows the receive side TCP processing to be distributed amongst the available CPU cores, providing a considerable performance advantage over a conventional adapter architecture in which all received packets for a given interface are processed by only one CPU core. RSS can be restricted to process receive queues only on the NUMA node local to the X4 series adapter. To configure this the driver module option rss_numa_local must be set to 1.

By default the driver enables RSS and configures one RSS Receive queue per CPU core. The number of RSS Receive queues can be controlled via the driver module parameter rss_cpus. The following table identifies rss_cpus options.

Table 1. rss_cpus Options
Option	Description	Interrupt Affinity (MSI-X)
<num_cpus>	Indicates the number of RSS queues to create.	A separate MSI-X interrupt for a receive queue is affinitized to each CPU.
packages	An RSS queue is created for each multi-core CPU package. The first CPU in the package is chosen.	A separate MSI-X interrupt for a receive queue is affinitized to each of the designated package CPUs.
cores	An RSS queue is created for each CPU. The first hyperthread instance (If CPU has hyperthreading) is chosen. This is the default option.	A separate MSI-X interrupt for a receive queue is affinitized to each of the CPUs.
hyperthreads	An RSS queue is created for each CPU hyperthread (hyperthreading must be enabled).	A separate MSI-X interrupt for a receive queue is affinitized to each of the hyperthreads.
numa_local_cores	An RSS queue is created for each CPU that is NUMA local. The first hyperthread instance (If CPU has hyperthreading) is chosen.	A separate MSI-X interrupt for a receive queue is affinitized to each of the CPUs.
numa_local_hyperthreads	An RSS queue is created for each CPU hyperthread that is NUMA local (hyperthreading must be enabled).	A separate MSI-X interrupt for a receive queue is affinitized to each of the hyperthreads.

Add the following line either to the /etc/modprobe.conf file, or to a file with a .conf extension under the /etc/modprobe.d directory:

options sfc rss_cpus=<option>

To set rss_cpus equal to the number of CPU cores:

options sfc rss_cpus=cores

Note: The association of RSS receive queues to a CPU is governed by the MSI-X interrupt affinity of the receive queue. See Interrupt Affinity for more details.

Sometimes when running single stream applications you might want to disable RSS, because all interface processing might benefit from taking place on a single CPU:

options sfc rss_cpus=1

You must reload the driver to enable the option change:

rmmod sfc
modprobe sfc

You must then rebuild the initramfs to ensure the option change persists after a reboot:

dracut -f

Note: RSS also works for UDP packets. For UDP traffic the X4 series adapter selects the Receive CPU based on IP source and destination addresses. X4 series adapters support IPv4 and IPv6 RSS.

Receive Side Scaling (RSS) - UG1739

AMD Solarflare X4 Series Ethernet Adapter User Guide (UG1739)