Single kernel programming focuses on vectorization of an algorithm in a single AI Engine-ML. However, multiple kernel programming considers several AI Engine kernels with data flowing between them.
The ADF graph can contain a single kernel or multiple kernels interacting with PS, PL, and global memory. Each AI Engine kernel has a runtime ratio. This is a ratio of the number of cycles taken by one invocation of a kernel (processing one block of data) to the cycle budget. The cycle budget for an application is typically fixed according to the expected data throughput and the block size being processed. You specify the runtime ratio as a constraint for every AI Engine kernel in the ADF graph.
The AI Engine compiler allocates multiple kernels into a single AI Engine-ML if the following applies:
- Their combined total runtime ratio is less than one and multiple kernels fit in the AI Engine-ML program memory.
- The total resource usage, such as stream interface number, does not exceed the AI Engine-ML tile limit.
Alternatively, the compiler can allocate them into multiple AI Engine-MLs.
When programming for the AI Engine-ML, it is important to note that each AI Engine-ML has the capability to access the following:
- One 32-bit AXI4-Stream input.
- One 32-bit AXI4-Stream output.
- One 512-bit cascade stream input (coming from north or west).
- One 512-bit cascade stream output (going to south or east).
- Two 256-bit data loads.
- One 256-bit data store.
However, due to the length of the instruction, not all of these operations can be performed during the same cycle.
When programming for the AI Engine-ML v2, it is important to note that each AI Engine-ML has the capability to access the following:
- One 32-bit AXI4-Stream input.
- One 32-bit AXI4-Stream output.
- One 512-bit cascade stream input (coming from north or west).
- One 512-bit cascade stream output (going to south or east).
- At most 1024-bit data loads.
- One 512-bit data store.
However, due to the length of the instruction, not all of these operations can be performed during the same cycle.
To optimally use hardware resources, it is critical to understand the different methods available for the following operations:
- Transfer data between the ADF graph and PS, PL, and global memory.
- Transfer data between kernels.
- Balance the data movement.
- Minimize memory or stream stalls as much as possible.
The following sections discuss these methods.