Throughput Measurement Using Timers - 2023.2 English

AI Engine Kernel and Graph Programming Guide (UG1079)

Document ID
UG1079
Release Date
2023-12-04
Version
2023.2 English

Throughput measurement using timers is one of the easiest way to measure the throughput at the AI Engine Array interface coarsely. The goal is to measure the time interval in between the input data transfer start and end.

A host code is generally decomposed in different parts:
  • Open the device.
  • Load the xclbin (bitstream for the PL and executables for the AIE array).
  • Declare the source and destination buffers.
  • Run the graph.
  • Launch the graph.
  • Launch input transfers.
  • Launch output transfers.
  • Wait for the end of input transfers.
  • Wait for the end of output transfers.
  • Wait for the end of the graph.
  • Free all buffers.

When using timers to evaluate interface throughput, the reset of the timer should be as close as possible to the beginning of the transfer. Also, the timer stop should be as close as the effective end of the transfer. The best ordering of these actions depends on the throughput that you want to estimate.

For input transfers, timer fences should be just before input transfer launch and just after the wait of the end of the input transfers. If graph processing is really fast, you can launch output transfers before input ones.

Declare the source and destination buffers as follows:
  • Launch the graph.
  • Launch output transfers.
  • Reset timer.
  • Launch input transfers.
  • Wait for the end of input transfers.
  • Stop timer.
  • Wait for the end of output transfers.
If interested in the overall system throughput, timer fences should be set right before the input transfers and right after waiting for the end of the output transfers:
  • Launch the graph.
  • Launch output transfers.
  • Reset timer.
  • Launch input transfers.
  • Wait for the end of output transfers.
  • Stop timer.
  • Wait for the end of output transfers (should be safe to skip this stage).

Once you have the time interval, knowing the amount of data allows you to compute directly the throughput of your system.