AI Engine GMIO Performance Profile - 2024.1 English

Profiling using C++ Class API

The code to use C++ class API is common for Linux system for various platforms. The Timer is defined as follows:

class Timer {
    std::chrono::high_resolution_clock::time_point mTimeStart;
        public:
           Timer() { reset(); }
           long long stop() {
           std::chrono::high_resolution_clock::time_point timeEnd = std::chrono::high_resolution_clock::now();
           return std::chrono::duration_cast<std::chrono::microseconds>(timeEnd - mTimeStart).count();
          }
          void reset() { mTimeStart = std::chrono::high_resolution_clock::now(); }
        };

The code to start profiling is as follows:

Timer timer;

The code to end profiling and calculate performance is as follows:

double timer_stop=timer.stop();
double throughput=(BLOCK_SIZE_in_Bytes+BLOCK_SIZE_out_Bytes)*NUM/timer_stop;
std::cout<<"Throughput (by timer GMIO in num="<<num<<",out num="<<num<<"):\t"<<throughput<<"M Bytes/s"<<std::endl;

The code is guarded by macro __TIMER__. To use this method of profiling, define __TIMER__ for g++ cross compiler in sw/Makefile:

CXXFLAGS += -std=c++17 -D__TIMER__ ......

To run it in hardware, use the following make command to build the hardware image:

make package TARGET=hw

After the package is done, run the following commands in the Linux prompt after booting Linux from an SD card (use petalinux/petalinux to login):

cd /run/media/mmcblk0p1
./host.exe a.xclbin

The output in hardware is similar as follows:

Throughput (by timer GMIO in num=4,out num=4):9882.79M Bytes/s

Profiling using AI Engine Cycles Received from AI Engine Kernels

In this design, AI Engine cycles output at the end of each iteration. Each iteration produces 256 int32 data and a very long AI Engine cycle counter number. The first and the last cycle of all AI Engine kernels to be profiled are recorded because multiple AI Engine kernels can start at different cycles though they are enabled by the same graph::run. Thus, the system throughput for all the kernels can be calculated.

Note: There is some gap between the actual performance and the calculated number because there are some data transfers before and after the recorded cycles.

The code to get AI Engine cycles and calculate the system throughput is as follows:

   unsigned long long start[NUM];
   unsigned long long end[NUM];
   unsigned long long very_beginning=0xFFFFFFFFFFFFFFFF;
   unsigned long long the_last=0;
    for(int i=0;i<NUM;i++){
        start[i]=*(unsigned long long*)&doutArray[i][256];
        end[i]=*(unsigned long long*)&doutArray[i][BLOCK_SIZE_out_Bytes/sizeof(int)-2];
            if(start[i]<very_beginning){
               very_beginning=start[i];
            }
            if(end[i]>the_last){
                the_last=end[i];
            }
        }
        std::cout<<"Throughput (by AIE kernel cycles in="<<NUM<<",out="<<NUM<<") ="<<(double)(BLOCK_SIZE_in_Bytes+BLOCK_SIZE_out_Bytes)*NUM/((double)(the_last-very_beginning)*0.8)*1000<<"M Bytes/s"<<std::endl;

The code is guarded by macro __AIE_CYCLES__. To use this method of profiling, define __AIE_CYCLES__ for g++ cross compiler in sw/Makefile:

CXXFLAGS += -std=c++17 -D__AIE_CYCLES__ ......

The commands to build and run in hardware are the same as previously shown. The output in hardware is similar as follows:

Throughput (by AIE kernel cycles in=4,out=4) =10561.8M Bytes/s

Profiling using the Event API

The AI Engine has hardware performance counters that can be configured to count hardware events for measuring performance metrics. The API used in this example can be used to profile graph throughput for specific GMIO ports. There may be conflict when multiple GMIO ports are used for the event API because of the restriction that performance counter is shared between GMIO ports that access the same AI Engine-PL interface column. Thus, all GMIO ports are constrained to different columns to avoid such conflicts.

The code to start profiling is as follows:

	xrt::aie::profiling *handle[NUM];
	for(int i=0;i<NUM;i++){
		handle[i]=new xrt::aie::profiling(device);
		handle[i]->start(xrt::aie::profiling::profiling_option::io_stream_start_to_bytes_transferred_cycles, "gr.gmioOut["+std::to_string(i)+"]", "", BLOCK_SIZE_out_Bytes);
	}

The code to end profiling and calculate performance is as follows:

	long long cycle_count[NUM];
	long long total_cycle_count=0;
	for(int i=0;i<NUM;i++){
		cycle_count[i] = handle[i]->read();
		handle[i]->stop();
		if(cycle_count[i]>total_cycle_count){
			total_cycle_count=cycle_count[i];
		}
	}
	double bandwidth = (double)(BLOCK_SIZE_in_Bytes+BLOCK_SIZE_out_Bytes)*NUM / ((double)total_cycle_count*0.8) *1000; //byte per second
	std::cout<<"Throughput (by event API) bandwidth="<<bandwidth<<"M Bytes/s"<<std::endl;

In this example, start of xrt::aie::profiling handler is called to configure the AI Engine to count the clock cycles from the stream start event to the event that indicates BLOCK_SIZE_out_Bytes bytes have been transferred, assuming that the stream stops right after the specified number of bytes are transferred.

For detailed usage about event API, refer to the Versal Adaptive SoC AI Engine Programming Environment User Guide (UG1076).

The code is guarded by macro __USE_EVENT_PROFILE__. To use this method of profiling, define __USE_EVENT_PROFILE__ for g++ cross compiler in sw/Makefile:

CXXFLAGS += -std=c++17 -D__USE_EVENT_PROFILE__ ......

The commands to build and run in hardware are the same as previously shown. The output in hardware is similar as follows:

Throughput (by event API) bandwidth=10571.6M Bytes/s