Occasionally Onload needs to send a packet, which would normally be accelerated, via the kernel. This occurs when there is no destination address entry in the ARP table or to prevent an ARP table entry from becoming stale.
By default, the Linux sysctl, unres_qlen
, will
enqueue three packets per unresolved address when waiting for an ARP reply, and on a
server subject to a very high UDP or TCP traffic load this can result in packet loss on
the transmit path and packets being discarded.
The unres_qlen
value can be identified using the following command:
sysctl -a | grep unres_qlen
net.ipv4.neigh.eth2.unres_qlen = 3
net.ipv4.neigh.eth0.unres_qlen = 3
net.ipv4.neigh.lo.unres_qlen = 3
net.ipv4.neigh.default.unres_qlen = 3
Changes to the queue lengths can be made permanent in the /etc/sysctl.conf file. It is recommended to set the
unres_qlen
value to at least 50.
If packet discards are suspected, this extremely rare condition can be indicated by the cp_defer
counter produced by the onload_stackdump lots
command on UDP sockets or from the unresolved_discards
counter in the Linux /proc/net/stat arp_cache file.