Troubleshooting Event Trace in Hardware - 2025.2 English - UG1076

AI Engine Tools and Flows User Guide (UG1076)
Document ID
UG1076
Release Date
2025-11-20
Version
2025.2 English
Table 1. Troubleshooting Event Trace in Hardware
Issue Resolution
Trace packets drop on certain streams. (For this issue, you can observe black bars on event trace streams in the Vitis IDE.) Option 1: Increase the number of trace streams coming out of AI Engine using the option below in build time.
v++ -c --mode aie --aie.num-trace-streams=<N>

Option 2: If using PLIO event trace streams, use the --aie.trace-plio-width=128 option for a 128-bit PLIO stream width. This ensures the PLIO stream width matches the AI Engine data rate.

Option 3: Reduce the number of tiles connected per trace stream. You can configure the AI Engine event trace metric to be applied on specific graphs or tiles. You can find more details on these options in Table 1 and Table 1.

Option 4: aiecompiler allocates trace streams for every tile in a horizontal fashion. To ensure optimal trace nets routing, AMD recommends using location constraints to ensure AI Engines are placed vertically wherever possible. This means all active AI Engines are equally distributed to trace streams.
Kernel function names are not visible from trace. This can be because the kernel is inlined either by the compiler or via an attribute. Apply --xlopt=0 to compile the design or specify kernel functions with __attribute__((noinline)) attribute to disable kernel in-lining.
Kernels start time are off by more than one hundred cycles. Apply the --broadcast-enable-core=true option to the compiler to ensure the design starts all kernels in a few clock cycles range.
Tool issues a warning message indicating trace buffer is full. Increase event trace buffer size.
For the XSDB flow, specify larger value with the -depth option in the AI Engine trace start command.
%xsdb aietrace start -graph-based-aie-tile-metrics dut:all:all_stalls -work-dir ./Work -link-summary ./bf_hw.xsa.xclbin.link_summary -base-address 0x900000000 -depth 0x8000000
For the XRT flow, update xrt.ini file and aie_trace_buffer_size line.
[Debug]
aie_trace=true

[AIE_trace_settings]
buffer_size=100M
graph_based_aie_tile_metrics = dut:all:all_stalls
Note: The following warning message is issued by both XSDB and XRT when the trace buffer is full.

Warning message:

AI Engine Trace Buffer size is full, Device trace could be incomplete.
Alternatively, you can also delay the event trace start based on time, iteration, or user-defined event. To do this use the trace option -start-type under Table 1 and Table 1 topics.
Need to know if the PL is sending or receiving data to/from the AI Engine as expected. Add monitors on PL kernels and their memory-mapped AXI4 masters.

For more information on adding Acceleration Monitors and AXI Performance Monitors, see Enabling Trace in Your Application in the Embedded Design Development Using Vitis (UG1701).