Free-Running AI Engine Kernel - 2024.1 English

AI Engine-ML Kernel and Graph Programming Guide (UG1603)

Document ID
Release Date
2024.1 English

The AI Engine kernel can be made to run continuously by using graph::run(-1). This way the kernel will restart automatically after the last iteration is complete.

Note: graph::run() without an argument runs the AI Engine kernels for a previously specified number of iterations (which is infinity by default if the graph is run without any arguments). If the graph is run with a finite number of iterations, for example,;; the second run call will also run for three iterations.

However, it requires input buffers and output buffers to be ready before it can start. Thus, it has a small overhead between kernel execution iterations. This section describes a method to construct a type of kernel that has zero overhead and runs forever. It is called the free-running AI Engine kernel.

The free-running kernels can only have streaming interfaces. Loops with infinite iterations can be inside the kernel. For example:

void free_running_aie(input_stream<int32> *in, output_stream<int32> *out) {
	while(true){//for(;;) is acceptable for C++
		int32 tmp=readincr(in);
		chess_separator_scheduler();//make sure stream is flushed
		chess_separator_scheduler();//make sure stream is flushed

The free-running kernel must have its own graph defined. This graph must not have any other non-free-running kernels, because the graph never stops and non-free-running kernels will lose control after being started. The graph containing the free-running kernel must be a top-level graph that can be connected to other graphs, or it can be connected to PLIO or GMIO. A sample connection between the free-running graph and other graphs is shown as follows.

passingGraph mygraph;
freeGraph mygraph_free;
connect<> net0(mygraph.out1,;
connect<> net1(mygraph_free.out,mygraph.in2);
Figure 1. Free-Running Graph Connection

The free-running graph can be started using or automatically started after loading.