Input and Output Buffers - 2024.1 English

AI Engine-ML Kernel and Graph Programming Guide (UG1603)

Document ID
UG1603
Release Date
2024-06-06
Version
2024.1 English

Input and output buffers represent a block of data that is stored contiguously on a tile’s physical memory, and that can be used by kernels in a graph. The origin of this data can be either other kernels that produce them (when they are operating on the same tile or another tile), or they can come from the PL or the MEM tile through AI Engine array interface. The buffer port can be allocated in the physical memory of the tile where the kernel executes or in the physical memory of accessible adjacent tiles.

When a kernel has a buffer port on its input side, it waits for the buffer to be fully available before it starts execution. The kernel can access the contents of the buffer port either randomly or in a linear fashion. Conversely the kernel can write a block (frame) of data to the local memory that can be used by other kernels after it has finished execution.

When the source of a buffer is a stream, this stream is sliced into contiguous blocks which are stored one by one into buffers, as illustrated below.

Figure 1. Data Stream Slicing Into Buffers

An example of kernel buffer port that resides at local tile memory is shown in the following figure. The kernel k2 is in tile (11,0) and input buffer port is allocated in the local memory of the same tile (11,0).

Figure 2. Kernel Buffer Port in Local Tile Memory

An example of a kernel buffer port that is allocated in a neighboring tile memory is shown in the following figure. The kernel k2 is in tile (11,0) and output buffer port at neighbor tile (10,0). This is the same for kernel k1 located in tile (10,0) which has both input and output buffers in the same tile (10,0).

Note: Buffers can be placed automatically by the tools as directed with user-specified design constraints. A single buffer port (including both Ping and Pong buffers) must be placed in a single tile. For AI Engine devices, a single buffer port is no larger than 32 KB. For AI Engine-ML devices, a single buffer port is no larger than 64 KB.
Figure 3. Kernel Buffer Port in Neighboring Tile Memory