In the case of FIR filters, you can use the cascade stream within the AI Engine tile. This increases the overall throughput by splitting the compute across multiple AI Engine tiles. However, this can increase the overall latency of the FIR. For more information, refer to the Simple Filter Chain Example.
The data movement into, out of, and around the AI Engine array directly contributes to the latency of your system. For example, using window or streaming interfaces for your kernel is a form of data movement. Depending on the window size, this can introduce an additional delay when loading the input. This occurs because the AI Engine waits for the full window to be available before starting the compute.
With a streaming solution, compute starts while the system delivers the data. Streaming interfaces are common for high data rate designs. However, this can lead to additional PL functionality being required to sort and align the data for streaming. In the case of the windowing solution, ping-pong buffering can help mitigate against the increased latency.
When communicating between the PL and AI Engine array, be aware of additional latency in your system. This occurs if the PL runs more slowly than the AI Engine. In this case, you must use wider buses, (32, 64, or 128-bit) to bring the data into the array. The AXI4-Stream interconnect to the AI Engine memory and core is 32-bits. This adds latency to the overall system and it takes four cycles to consume a 128-bit bus.
For data processing in the PL, it is important to consider the DSP Engines and PL for small latency critical functions. This eliminates the need to bring the data in and out of the AI Engine array. This transfer can be costly if the latency requirements are small.
Another consideration is how you intend to control your function or application within the AI Engine array. Do you use the PS or PL to control the kernel functionality at run time? Can the PS meet the latency requirement, or do you need a PL controller? These are important factors which need to be evaluated for your specific application.
For more information, refer to the following documents:
- Versal Adaptive SoC AI Engine Architecture Manual (AM009)
- Versal Adaptive SoC AIE-ML Architecture Manual (AM020)
- Versal Adaptive SoC AIE-ML v2 Architecture Manual (AM027)
- AI Engine Tools and Flows User Guide (UG1076)
- AI Engine Kernel and Graph Programming Guide (UG1079)
- AI Engine-ML Kernel and Graph Programming Guide (UG1603)
- Vitis Tutorials: AI Engine Development