Memory and Data Movement - Memory and Data Movement - 2026.1 English - UG1504

Understanding the Versal adaptive SoC memory hierarchy helps you determine:

• The scope of the problem you need to resolve.
• How the different engines communicate data and what bandwidth is available.
• How to take advantage of the raw compute available in each AI Engine to deliver the best performance per watt for your application, if applicable.

Applications have different memory hierarchies based on their deployment environments. For example, some applications have external DDR memory requirements. Others use interfaces like JESD to bring in data from discrete analog-to-digital converters (ADCs). Regardless of where the data originates, it is important to build the correct memory hierarchy to meet the needs of your system.

For example, in systems that require external DDR memory, it is only possible to get data from DDR memory at specified rates. Therefore, your system has a fixed maximum bandwidth. For example, in Versal adaptive SoC, the maximum LPDDR memory bandwidth is approximately 34 GB/s per memory controller to the NoC.

The NoC moves the data from DDR memory throughout the adaptive SoC. In the case of the AI Engines, there is direct access to the AI Engine array. This access is via the NoC interface tiles in the AI Engine array interface. However, the bandwidth is relatively low, and this is not the best way to get the data to the AI Engine tiles.

In most cases, Versal devices have a large amount of on-chip memory. It is expected that the data is brought via the NoC. The data is brought into a staging memory in the PL, such as UltraRAM and/or block RAM. The PL interface array tiles are used for data transfer to and from the AI Engine. This is because there is much higher bandwidth from the PL into and out of the AI Engine array.

Some applications can require direct DDR memory-NoC-AI Engine communication. This communication is possible but with lower overall bandwidth available. Therefore, for most applications, use the NoC for debug, trace, and control communication via the PS or any master.

Devices that contain the AI Engine array also have local data memory in each AI Engine tile. Each tile contains eight data memory banks of 4 KB each, totaling 32 KB per tile. Each AI Engine core has direct local access to the data memory on the same AI Engine tile. It also has access to three of its neighboring data memories from tiles. These neighbors are North, South, and either East or West. This gives 128 KB local shared memory for each tile.

Devices with an AI Engine-ML array include additional rows of 512 KB memory tiles, which provide low-latency local memory storage.

The following figure shows total VC1902 available memory. For Versal AI Core devices, refer to the AMD Versal AI Core Series Product Selection Guide (XMP452). For Versal AI Edge Series Gen 2 devices, refer to the AMD Versal AI Edge Portfolio Product Selection Guide (XMP464).

Figure 1. On-Chip Memory Hierarchy in Versal Adaptive SoC AI Core Series Families

Data communication is key to enabling an efficient design, in particular when using the AI Engine array. To efficiently partition the design, you must understand the data bandwidth to and from the AI Engine and internally between the AI Engines.

• For AI Engine array interface, refer to this link in the Versal Adaptive SoC AI Engine Architecture Manual (AM009).
• For AIE-ML array interface, refer to this link in the Versal Adaptive SoC AIE-ML Architecture Manual (AM020)
• For AIE-ML v2 array interface, refer to this link in the Versal Adaptive SoC AIE-ML v2 Architecture Manual (AM027).

Figure 2. Communication Bandwidth through the Versal Adaptive SoC

For some functions, like symmetric FIRs, convolutional neural network (CNN), or beamforming, there is some data reuse (for example, coefficient and weights sharing). You can reduce memory bandwidth by using the streaming broadcast functionality to send the same weights or coefficients to multiple AI Engine tiles. Applications with a large amount of data reuse are well suited to implementation in the AI Engines. Window interfaces are better for data reuse in large filter implementation on a single tile.

The following figure shows an example dataflow through the Versal adaptive SoC into the AI Engine array. It includes important bandwidth numbers to consider when mapping your data through the device.

Figure 3. Example Dataflow to the AI Engines

Using this example dataflow, there are considerations you must make based on the specifics of your application. Getting data from the DDR memory into the staging memories in the PL (for example, UltraRAMs) depends on several factors:

• Amount of data to be transferred
• Memory controller throughput
• NoC bandwidth
• Memory size

After the data loads into the UltraRAMs, the PL can require a data sort or pre-processing stage. Assuming this is not the case, the data needs to get to the AI Engines for processing. Key care-abouts at this phase of the dataflow are the bandwidth through the AI Engine array interface. The bandwidth within the AI Engine array is also important for transferring data to the necessary tiles.

Both the AXI4-Stream bandwidth and the AI Engine tile and array data memory size determine the scope of the problem the AI Engines can solve.