Connections with Onload - UG1586

HTTP Connections with nginx

Run the nginx application on system-2, increasing the number of file descriptors available, and accelerating it using the nginx_reverse_proxy Onload profile:

[system-2]# ulimit -n 1000000 && onload --profile=nginx_reverse_proxy nginx

Run multiple instances of the wrk application on system-1, so there is the capacity to generate more connection requests than can be handled. A zero-byte file is requested, and the “Connection: close” header is passed to close the connection immediately:

[system-1]# for i in {1..3}; do taskset -c $i wrk -t 1 -c 50 -d 10s
-H 'Connection: close' http://system-2/0kb.bin & done

Each instance outputs its own results. Aggregating the results gives the following totals:

Requests/sec: 111061.19
Transfer/sec:        25.81MB

The server handles 111061 connections per second (Requests/sec). This is over 3 times the rate achieved without Onload (see Connections without Onload below).

Note the following:

• the total throughput is much less than line rate, so any limit on capacity is in nginx rather than in the network.
• some instances have connect or timeout errors, or have reduced throughput, indicating that there are enough instances to reach full nginx capacity:

  Socket errors: connect 0, read 0, write 0, timeout 38