Latency and Throughput Estimates - 2024.1 English

Average Throughput is automatically computed over the duration of the simulation and displayed at the end of the AI Engine simulation. Generating a VCD file during AI Engine simulation generates Latency, Continuous Latency and Continuous Throughput estimates.

Continuous Latency is the dynamic evolution of the Latency between specified input and output ports. Continuous Throughput is the dynamic evolution of the Throughput at a specified input or output port.

aiesimulator --pkg-dir=./Work --dump-vcd foo --options-file=aiesim-options.txt

To keep the size of the VCD file to a minimum, generate SHIM and STREAM_SWITCH trace data only. In the aiesim-options.txt file, set AIE_DUMP_VCD_SHIM and AIE_DUMP_VCD_STREAM_SWITCH to true as follows:

AIE_DUMP_VCD_IO=false
AIE_DUMP_VCD_CORE=false
AIE_DUMP_VCD_SHIM=true
AIE_DUMP_VCD_MEM=false
AIE_DUMP_VCD_STREAM_SWITCH=true
AIE_DUMP_VCD_CLK=false

The Vitis IDE enables you perform latency and throughput analysis. Opening the default.run_summary file using vitis_analyzer will show the Latency measurements for the AI Engine Array in the Trace section within the Latency tab

vitis_analyzer aie/aiesimulator_output/default.aierun_summary

The Latency tab includes three columns with latency data:

First Latency

Time latency between first input data to first output data.

Last Latency

Time latency between last input data to last output data.

Average Latency

The difference between the average output sample time and average input sample time.

For all output, all corresponding input are also shown.

For additional information about the latency and throughput of the simulation:

1. Right-click an input port line, and select Export Continuous Latency, Export Continuous Throughput, or Export Table.
   Note: Only the Export Continuous Throughput and Export Table options are available for an output row.
   Figure 2. Export Option in the Context Menu for Input Ports
2. When you select to export continuous latency or throughput, the Export Continuous Latency dialog box requires the following information:
   • A filename, and
   • The width of the window. The continuous latency and throughput are computed over consecutive windows. You can set one of:
     • Number of intervals: the simulation duration is divided into N intervals on which the average latency (or throughput) is computed.
     • Active Cycles: the interval length is here directly specified in terms of number of cycles.
   Figure 3. Export Continuous Latency Dialog Box

A CSV file is then generated.

Start Time (ps), End Time (ps), Latency (ps) between 'Input_3' and 'Output_3'
2119200, 2866399, 1010400
2866400, 3613599, 1520000
3613600, 4360799, 1956000
4360800, 5107999, 1956800
5108000, 5855199, 1956800
...
12580000, 13327199, 1956000
13327200, 13327999, 1925600

• The first column is the timestamp of the first sample of the interval.
• The second column is the timestamp of the last sample of the interval.
• the third column is the computed latency (or throughput) over the interval.

The CSV file can be opened in a text editor, or in any spreadsheet software. From a spreadsheet, select the first and third columns to draw the continuous latency (or continuous throughput) against time. Examples of the analysis graph is shown in the following figures.