sfnt-stream

Onload User Guide (UG1586)
Document ID
UG1586
Release Date
2023-07-31
Revision
1.2 English

The sfnt-stream application measures RTT latency (not ½ RTT) for a fixed size message at increasing message rates. Latency is calculated from a sample of all messages sent. Message rates can be set with the rates option and the number of messages to sample using the sample option.

sfnt-stream only functions on UDP sockets. This limitation will be removed to support other protocols in the future.

Refer to the README.sfnt-stream file which is part of the Onload distribution for further information.

Usage

sfnt-stream [options] [tcp|udp|pipe|unix_stream|unix_datagram [host[:port]]]

Options

The following table lists the sfnt-stream options:

Table 1. sfnt-stream Options
Option Description
--msgsize message size (bytes)
--rates msg rates <min>-<max>[+<step>]
--millisec time per test (milliseconds)
--samples number of samples per test
--stop stop when TX rate achieved is below give percentage of target rate
--maxburst maximum burst length
--port server port number
--connect connect() UDP socket
--spin spin on non-blocking recv()
--muxer select, poll, epoll or none
--rtt report round-trip-time
--raw dump raw results to file
--percentile percentile
--mcast set the multicast address
--mcastintf

set multicast interface. The client sends this parameter to the server.

--mcastintf=eth2 both client and server use eth2

--mcastintf='eth2;eth3' client uses eth2 and server uses eth3 (quotes are required for this format)
--mcastloop IP_MULTICAST_LOOP
--ttl IP_TTL and IP_MULTICAST_TTL
--bindtodevice SO_BINDTODEVICE
--n-pipe include pipes in file descriptor set
--n-unix-d include Unix datagram in file descriptor set
--n-unix-s include Unix stream in file descriptor set
--n-udp include UDP sockets in file descriptor set
--n-tcpc include TCP sockets in file descriptor set
--n-tcpl include TCP listening sockets in file descriptor set
--tcpc-serv host:port for TCP connections
--nodelay enable TCP_NODELAY
--affinity

"<client-tx>,<client-rx>;<server-core>" enclose the values in double quotes, for example "4,5;3". This option should be set on the client side only. The client sends the <server_core> value to the server. The user must ensure that the identified server core is available on the server machine.

Important: This option will override any value set by taskset on the same command line.
--rtt-iter iterations for RTT measurement

The following table lists the standard options:

Table 2. Standard Options
Option Description
-? --help this message
-q --quiet quiet
-v --verbose display more information
--version display version information

Example Command Lines for Client/Server

[server]# ./sfnt-stream
[client]# ./sfnt-stream --affinity 1,1 udp <server-ip>
[client]# ./taskset -c 1 ./sfnt-stream --affinity="3,5;3" --mcastintf=eth4 udp \
                               <remote-ip>

Example of Bonded Interfaces

The following example configures a single bond, having two slaves interfaces, on each machine. Both server and client machines use eth4 and eth5.

[root@server src]# ifconfig eth4 0.0.0.0 down
[root@server src]# ifconfig eth5 0.0.0.0 down
[root@server src]# modprobe bonding miimon=100 mode=1 xmit_hash_policy=layer2 primary=eth5
[root@server src]# ifconfig bond0 up
[root@server src]# echo +eth4 > /sys/class/net/bond0/bonding/slaves
[root@server src]# echo +eth5 > /sys/class/net/bond0/bonding/slaves
[root@server src]# ifconfig bond0 172.16.136.28/21
NOTE: server sends to IP address of client bond
[root@server src]# onload --profile=latency taskset -c 1 ./sfnt-stream --mcastintf=bond0 --affinity "1,1;3" udp 172.16.136.27
[root@client src]# ifconfig eth4 0.0.0.0 down
[root@client src]# ifconfig eth5 0.0.0.0 down
[root@client src]# modprobe bonding miimon=100 mode=1 xmit_hash_policy=layer2 primary=eth5
[root@client src]# ifconfig bond0 up
[root@client src]# echo +eth4 > /sys/class/net/bond0/bonding/slaves
[root@client src]# echo +eth5 > /sys/class/net/bond0/bonding/slaves
[root@client src]# ifconfig bond0 172.16.136.27/21
[root@client src]# onload --profile=latency taskset -c 3 ./sfnt-stream
sfnt-stream: server: waiting for client to connect...
sfnt-stream: server: client connected
sfnt-stream: server: client 0 at 172.16.136.28:45037

Output Fields

All time measurements are nanoseconds unless otherwise stated.

Table 3. sfnt-stream Output Fields
Field Description
mps target Msg per sec target rate
mps send Msg per sec actual rate
mps recv Msg receive rate
latency mean RTT mean latency
latency min RTT minimum latency
latency median RTT median latency
latency max RTT maximum latency
latency %ile RTT 99%ile
latency stddev Standard deviation of sample
latency samples Number of messages used to calculate latency measurement
sendjit mean Mean variance when sending messages
sendjit min Minimum variance when sending messages
sendjit max Maximum variance when sending messages
sendjit behind Number of times the sender falls behind and is unable to keep up with the transmit rate
gaps n_gaps Count the number of gaps appearing in the stream
gaps n_drops Count the number of drops from stream
gaps n_ooo Count the number of sequence numbers received out of order