Latency and Throughput Estimates - 2025.2 English - UG1076

AI Engine Tools and Flows User Guide (UG1076)
Document ID
UG1076
Release Date
2025-11-20
Version
2025.2 English

AI Engine simulation automatically calculates and displays average throughput at the end of the simulation. If you generate a VCD file during the simulation, it can also provide estimates for continuous latency and throughput.

Continuous latency refers to how the latency between specified input and output ports changes over time. Continuous throughput refers to how the throughput at a specified input or output port changes.

Input and output ports can be either PLIO or GMIO. You can generate a VCD file with the command:

aiesimulator --pkg-dir=./Work --dump-vcd foo --options-file=aiesim-options.txt

To minimize the size of the VCD file, only generate SHIM and STREAM_SWITCH trace data. In the aiesim-options.txt file, set AIE_DUMP_VCD_SHIM and AIE_DUMP_VCD_STREAM_SWITCH to true while setting other options to false as follows:

AIE_DUMP_VCD_IO=false
AIE_DUMP_VCD_CORE=false
AIE_DUMP_VCD_SHIM=true
AIE_DUMP_VCD_MEM=false
AIE_DUMP_VCD_STREAM_SWITCH=true
AIE_DUMP_VCD_CLK=false
Figure 1. Continuous Throughput

Latency Estimates in the Vitis IDE

After running AI Engine Simulation with some specific options you have access to the following latency estimates:

I/O Latency
The latency between a sample entering and leaving the AI Engine array through interface tile channels.
Kernel Latency
The latency between the input and the output of a kernel.
Figure 2. Latency Tab after AI Engine Simulation Generated by Your Tool

I/O Latency and Continuous Latency

The Vitis IDE has a feature that allows you to analyze latency and throughput of the graph. Open the default.run_summary file using Vitis Analyzer to view latency measurements for the AI Engine Array in the Trace section of the Latency tab. You can access this feature by running:

vitis_analyzer aie/aiesimulator_output/default.aierun_summary

The Latency tab displays three columns that show latency data:

First Latency
The time it takes for the first input data to reach the first output data.
Last Latency
The time it takes for the last input data to reach the last output data.
Average Latency
The difference between the average output sample time and average input sample time.

In addition, the tab shows all corresponding inputs for each output.

Figure 3. Latency Table

To find more information about the latency of a simulation, use these steps:

  1. Right-click on an input port line, and select one of three options Export Continuous Latency, Plot Continuous Latency, or Export Table.
    Note: The Export Table option is only available for an output row.
    Figure 4. Options in the Context Menu for Input Ports
  2. Enter the file name and window width in the dialog box. Continuous latency and throughput are calculated over consecutive windows. You can select either the number of intervals or active cycles:
    Number of intervals
    Divides the simulation duration into N intervals to compute the average latency (or throughput).
    Active Cycles
    The interval length specified in terms of number of cycles.
    Figure 5. Export Continuous Latency Dialog Box

The tool generates a CSV file.

Start Time (ps), End Time (ps), Latency (ps) between 'Inputp_0' and 'Outputp_0' for '64' intervals
3362400, 4181599, 1876000
4252800, 5071999, 1944800
5143200, 5962399, 1964000
6033600, 6852799, 1964800
...
58593600, 59412799, 1991200
59484000, 60303199, 1991200

After selecting, a CSV file generates. This contains the start and end times of each interval, and the computed latency between input and output ports. The CSV file can be used by post processing scripts for further analysis. To view the latency graph, select the Plot Continuous Latency option. The same dialog box appears, but the latency plot is displayed in Vitis Analyzer. You can display several plots can be displayed on the same graph, as shown in the following figure.

Figure 6. Continuous Latency

Kernel Latency

In addition to data on IO latency, Vitis Analyzer provides the equivalent data on kernel latency.

Figure 7. Kernel Latency Table Generated by Your Tool

The kernel latency table displays first, last, and average latencies. You can use the right-click menu to save or plot the continuous kernel latency throughout the simulation.

This example shows values in both the in and out rows. The in row value indicates the latency between the input and output ports. If there are multiple input ports, there are multiple in rows. The values in the out row mirrors the smallest values from each column.

Throughput and Continuous Throughput

Throughput estimates are also available in the Trace section in the I/O tab. The THROUGHPUT (MBYTES/S) column gives you the average throughput on each port, computed on the total duration of the simulation.

Figure 8. Throughput Table

For more information about the throughput of a simulation, follow these steps:

  1. Right-click an input port line, and select export continuous throughput, plot continuous throughput, or export table.
    Figure 9. Options in the Context Menu for all Ports
  2. When you choose to export or plot continuous throughput, provide the file name and window width in the dialog box. Specify window width as follows:
    Graph Iterations
    Calculates throughput based on equally sized sections of input or output data. The iteration number used during the simulation is used for this calculation by default. This provides the most comprehensible results regarding the evolution of throughput over time.
    Number of intervals
    Divides simulation duration into N number of intervals to compute the average throughput for each interval.
    Active cycles per interval
    Specifies the length of the interval in terms of the number of active cycles per interval.
Figure 10. Export of Plot Continuous Throughput Dialog Box
If you select Export Continuous Throughput, the system generates a CSV file.
Start Time (ps), End Time (ps), Throughput (Mbytes/s) for 'Inputp_1 for '64' graph iterations
1486400, 2305600, 5000.0
2305600, 3128000, 4980.545
3128000, 4000000, 4697.2476
4000000, 4890400, 4600.1797
4890400, 5781600, 4596.0503
...
55673600, 56564800, 4596.0503
56564800, 57456000, 4596.0503
57456000, 58347200, 4596.0503

The CSV file contains the start and end times of each interval, as well as the computed throughput for the ports. To view the throughput graph, select the Plot Continuous Throughput option.