Free Running AI Engine Kernel - 2023.1 English

AI Engine Kernel and Graph Programming Guide (UG1079)

Document ID
UG1079
Release Date
2023-06-23
Version
2023.1 English

The AI Engine kernel can be made to run continuously by using graph::run(-1). This way the kernel will restart automatically after the last iteration is complete.

Note: graph::run() without an argument runs the AI Engine kernels for a previously specified number of iterations (which is infinity by default if the graph is run without any arguments). If the graph is run with a finite number of iterations, for example, mygraph.run(3); mygraph.run(); the second run call will also run for three iterations.

However, it requires input buffers and output buffers to be ready before it can start. Thus, it has a small overhead between kernel execution iterations. This section describes a method to construct a type of kernel that has zero overhead and runs forever. It is called the free running AI Engine kernel.

The free running kernels can only have streaming interfaces. Loops with infinite iterations can be inside the kernel. For example:

void free_running_aie(input_stream<int32> *in, output_stream<int32> *out) {
	while(true){//for(;;) is acceptable for C++
		int32 tmp=readincr(in);
		chess_separator_scheduler();//make sure stream is flushed
		writeincr(out,tmp+1);
		chess_separator_scheduler();//make sure stream is flushed
	}
}

The free running kernel must have its own graph defined. This graph must not have any other non-free running kernels, because the graph never stops and non-free running kernels will lose control after being started. The graph containing the free running kernel must be a top-level graph that can be connected to other graphs, or it can be connected to PLIO or GMIO. A sample connection between the free running graph and other graphs is shown as follows.

passingGraph mygraph;
freeGraph mygraph_free;
connect<> net0(mygraph.out1,mygraph_free.in);
connect<> net1(mygraph_free.out,mygraph.in2);
Figure 1. Free Running Graph Connection

The free running graph can be started using mygraph_free.run(-1) or automatically started after loading.