Zero-Copy UDP Receive

Onload User Guide (UG1586)

Document ID
Release Date
1.2 English

The onload_zc_recv() function specifies a callback to invoke for each received UDP datagram. The callback is invoked in the context of the call to onload_zc_recv() (it blocks/spins waiting for data).

Before calling, the application must set the following in the struct onload_zc_recv_args:

Table 1. onload_zc_recv_args Fields
Field Value
Set to the callback function pointer.
Set to point to application state, this is not touched by Onload
The user application should set these to appropriate buffers and lengths (if required) as you would for recvmsg (or NULL and 0 if not used).
Set to indicate behavior (for example ONLOAD_MSG_DONTWAIT).
Figure 1. Zero-Copy recv_args
typedef enum onload_zc_callback_rc
(*onload_zc_recv_callback)(struct onload_zc_recv_args *args, int flags);
struct onload_zc_recv_args
  struct onload_zc_msg msg;
  onload_zc_recv_callback cb;
  void* user_ptr;
  int flags;
int onload_zc_recv(int fd, struct onload_zc_recv_args *args);

The callback gets to examine the data, and can control what happens next:

  • whether or not the buffer(s) are kept by the callback or are immediately freed by Onload
  • whether or not onload_zc_recv() will internally loop and invoke the callback with the next datagram, or immediately return to the application.

The next action is determined by setting flags in the return code as follows:

Table 2. Flags for Setting in the Return Code
Flag Description
The callback function can elect to retain ownership of received buffer(s) by returning ONLOAD_ZC_KEEP. Following this, the correct way to release retained buffers is to call onload_zc_release_buffers() to explicitly release the first buffer from each received datagram. Subsequent buffers pertaining to the same datagram will then be automatically released.
To suggest that Onload should loop and process more datagrams
To insist that Onload immediately return from the onload_zc_recv()

Flags can also be set by Onload:

Table 3. Flags That Can Be Set By Onload
Flag Description
Onload sets this flag to indicate that this is the last packet
Packet buffers are read only

If there is unaccelerated data on the socket from the kernel’s receive path this cannot be handled without copying. The application has two choices as follows:

Table 4. Flags for Handling Unaccelerated Data
Flag Description
Set this flag when calling onload_zc_recv(). Onload will deal with the kernel data internally and pass it to the callback
check return code Check the return code from onload_zc_recv(). If it returns ENOTEMPTY then the application must call onload_recvmsg_kernel() to retrieve the kernel data.