Host Code Reference with ADF API and XRT API - 2022.1 English

Versal ACAP AI Engine Programming Environment User Guide (UG1076)

Document ID
UG1076
Release Date
2022-05-25
Version
2022.1 English

This section provides a summary of the XRT APIs that control the PL kernels and graph as well as a mapping relationship between the ADF API and XRT API. Complete host code using the ADF API or the XRT API to control the graph is also provided for reference.

Note: This section only lists part of APIs. See https://github.com/xilinx/xrt for the latest and more detailed information about the XRT API.
Table 1. XRT APIs
XRT API Description
Category: Device handle (experimental/xrt_device.h)
xrtDeviceHandle xrtDeviceOpen(unsigned int index); Open a device and obtain its handle.
xrtDeviceHandle xrtDeviceOpenFromXcl(xclDeviceHandle xhdl); Get a device handle from xclDeviceHandle.
int xrtDeviceClose(xrtDeviceHandle dhdl); Close an opened device.
int xrtDeviceLoadXclbinFile(xrtDeviceHandle dhdl, const char* xclbin_fnm); Read and load an XCLBIN file.
void xrtDeviceGetXclbinUUID(xrtDeviceHandle dhdl, xuid_t out); Get UUID of XCLBIN image loaded on device.
Category: PL kernel handle (experimental/xrt_kernel.h)
xrtKernelHandle xrtPLKernelOpen(xrtDeviceHandle deviceHandle, const xuid_t xclbinId, const char *name); Open a PL kernel and obtain its handle.
int xrtKernelClose(xrtKernelHandle kernelHandle); Close an opened kernel.
xrtRunHandle xrtKernelRun(xrtKernelHandle kernelHandle, ...); Start a kernel execution.
xrtRunHandle xrtRunOpen(xrtKernelHandle kernelHandle); Open a new run handle for a kernel without starting kernel.
int xrtRunSetArg(xrtRunHandle rhdl, int index, ...); Set a specific kernel argument for this run.
int xrtRunUpdateArg(xrtRunHandle rhdl, int index, ...); Asynchronous update of kernel argument.
int xrtRunStart(xrtRunHandle rhdl); Start existing run handle.
enum ert_cmd_state xrtRunWait(xrtRunHandle rhdl); Wait for a run to complete.
int xrtRunClose(xrtRunHandle rhdl); Close a run handle.
Category: Graph handle (experimental/xrt_graph.h)
xrtGraphHandle xrtGraphOpen(xrtDeviceHandle handle, const uuid_t xclbinUUID, const char *graphName); Open a graph and obtain its handle.
void xrtGraphClose(xrtGraphHandle gh); Close an open graph.
int xrtGraphRun(xrtGraphHandle gh, int iterations); Start a graph execution.
int xrtGraphWait(xrtGraphHandle gh, uint64_t cycle); Wait a set number of AI Engine cycles since the last xrtGraphRun and then stop the graph. If cycle is 0, wait until the graph is finished. If the graph has already run more than the set number of cycles, stop the graph immediately.
int xrtGraphResume(xrtGraphHandle gh); Resume a suspended graph.
int xrtGraphEnd(xrtGraphHandle gh, uint64_t cycle); Wait a set number of AI Engine cycles since the last xrtGraphRun and then end the graph. If cycle is 0, wait until the graph is finished before ending the graph. If the graph has already run more than the set number of cycles, stop the graph immediately and end it.
int xrtGraphUpdateRTP(xrtGraphHandle gh, const char *hierPathPort, const char *buffer, size_t size); Update RTP value of port with hierarchical name.
int xrtGraphReadRTP(xrtGraphHandle gh, const char *hierPathPort, char *buffer, size_t size); Read RTP value of port with hierarchical name.
Category: AIE handle (experimental/xrt_aie.h)
int xrtAIESyncBO(xrtDeviceHandle handle, xrtBufferHandle bohdl, const char *gmioName, enum xclBOSyncDirection dir, size_t size, size_t offset); Transfer data between DDR memory and interface tile DMA channel.
Category: Buffer object handle (experimental/xrt_bo.h)
xrtBufferHandle xrtBOAlloc(xrtDeviceHandle dhdl, size_t size, xrtBufferFlags flags, xrtMemoryGroup grp); Allocate a BO of requested size with appropriate flags.
int xrtBOFree(xrtBufferHandle bhdl); Free/Deallocate the allocated BO.
int xrtBOSync(xrtBufferHandle bhdl, enum xclBOSyncDirection dir, size_t size, size_t offset); Synchronize buffer contents in requested direction.
void* xrtBOMap(xrtBufferHandle bhdl); Memory map BO into user address space.
Category: Error reporting (experimental/xrt_error.h)
int xrtErrorGetLast(xrtDeviceHandle handle, xrtErrorClass ecl, xrtErrorCode* error, uint64_t* timestamp); Get the last error code and its timestamp of a given error class.
int xrtErrorGetString(xrtDeviceHandle, xrtErrorCode error, char* out, size_t len, size_t* out_len); Get the description string of a given error code.

The following table lists the mapping between the ADF API and XRT API. The xrtGraphOpen(), xrtPLKernelOpen(), xrtRunOpen(), xrtKernelClose() XRT APIs are called inside the ADF APIs when required and there is no corresponding mapping listed.

Table 2. ADF API and XRT API Mapping
Graph API XRT API
graph::run() xrtGraphRun(xrtGraphHandle, 0) for AI Engine.
graph::run(iterations) xrtGraphRun(xrtGraphHandle, iterations) for AI Engine.
graph::wait() xrtGraphWait(xrtGraphHandle, 0) for AI Engine.
graph::wait(aie_cycles) xrtGraphWait(xrtGraphHandle aie_cycles), for AI Engine.
graph::resume() xrtGraphResume(xrtGraphHandle)
graph::end() xrtGraphEnd(xrtGraphHandle, 0) and then xrtGraphClose(xrtGraphHandle) for AI Engine.
graph::end(aie_cycles) xrtGraphEnd(xrtGraphHandle, aie_cycles) and then xrtGraphClose(xrtGraphHandle) for AI Engine.
graph::update() xrtGraphUpdateRTP() for AI Engine;
graph::read() xrtGraphReadRTP() for AI Engine;
GMIO::malloc() xrtBOAlloc()xrtBOMap()
GMIO::free() xrtBOFree()
GMIO::gm2aie_nb() N/A
GMIO::aie2gm_nb() N/A
GMIO::wait() N/A
GMIO::gm2aie() xrtSyncBOAIE(...,XCL_BO_SYNC_BO_GMIO_TO_AIE,...)
GMIO::aie2gm() xrtSyncBOAIE(...,XCL_BO_SYNC_BO_AIE_TO_GMIO,...)
adf::event APIs for profiling and event trace N/A

The following is host code using the ADF API and XRT API for reference. The __USE_ADF_API__ is a user-defined macro in the code that can be used to switch between the ADF API and XRT API to control the AI Engine graph.

#include <stdlib.h>
#include <fstream>
#include <iostream>
#include "host.h"
#include <unistd.h>
#include <complex>
#include "adf/adf_api/XRTConfig.h"
#include "experimental/xrt_kernel.h"

#include "graph.cpp"

#define OUTPUT_SIZE 2048

using namespace adf;

int main(int argc, char* argv[]) {

    size_t output_size_in_bytes = OUTPUT_SIZE * sizeof(int);

    //TARGET_DEVICE macro needs to be passed from gcc command line
    if(argc != 2) {
	printf("Usage: %d <xclbin>\r\n",argv[0]);
	return EXIT_FAILURE;
    }
    char* xclbinFilename = argv[1];
	
    int ret;
    // Open xclbin
    auto dhdl = xrtDeviceOpen(0);//device index=0
    if(!dhdl){
	printf("Device open error\n");
    }
    ret=xrtDeviceLoadXclbinFile(dhdl,xclbinFilename);
    if(ret){
	printf("Xclbin Load fail\n");
    }
    xuid_t uuid;
    xrtDeviceGetXclbinUUID(dhdl, uuid);
    
    // output memory
    xrtBufferHandle out_bohdl = xrtBOAlloc(dhdl, output_size_in_bytes, 0, /*BANK=*/0);
    std::complex<short> *host_out = (std::complex<short>*)xrtBOMap(out_bohdl);

    // s2mm ip
    xrtKernelHandle s2mm_khdl = xrtPLKernelOpen(dhdl, uuid, "s2mm");
    xrtRunHandle s2mm_rhdl = xrtRunOpen(s2mm_khdl);
    xrtRunSetArg(s2mm_rhdl, 0, out_bohdl);
    xrtRunSetArg(s2mm_rhdl, 2, OUTPUT_SIZE);
    xrtRunStart(s2mm_rhdl);
    printf("run s2mm\n");

#if __USE_ADF_API__
    // update graph parameters (RTP) & run
    adf::registerXRT(dhdl, uuid);
    printf("Register XRT\r\n");
    int narrow_filter[12] = {180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504};
    int wide_filter[12] = {-21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539};
    gr.run(16);//start AIE kernel   
    gr.update(gr.fir24.in[1], narrow_filter, 12);//update AIE kernel RTP
    printf("Update fir24 done\r\n");
    printf("Graph run done\r\n");
    gr.wait(); // wait for AIE kernel to complete
    printf("Graph wait done\r\n");    
    gr.update(gr.fir24.in[1], wide_filter, 12);//Update AIE kernel RTP
    printf("Update fir24 done\r\n");
    gr.run(16);//start AIE kernel
    printf("Graph run done\r\n");
#else
    int narrow_filter[12] = {180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504};
    int wide_filter[12] = {-21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539};
    auto ghdl=xrtGraphOpen(dhdl,uuid,"gr");
    if(!ghdl){
	printf("Graph Open error\r\n);;
    }else{
	printf("Graph Open ok\r\n");;
    }
    int size=1024;
    xrtKernelHandle noisegen_khdl = xrtPLKernelOpen(dhdl, uuid, "random_noise");
    xrtRunHandle noisegen_rhdl = xrtRunOpen(noisegen_khdl);
    xrtRunSetArg(noisegen_rhdl, 1, size);
    xrtRunStart(noisegen_rhdl);
    printf("run noisegen\n");
    ret=xrtGraphUpdateRTP(ghdl,"gr.fir24.in[1]",(char*)narrow_filter,12*sizeof(int));
    if(ret!=0){
	printf("Graph RTP update fail\n");
	return 1;
    }
    ret=xrtGraphRun(ghdl,16);
    if(ret){
	printf("Graph run error\r\n");
    }else{
	printf("Graph run ok\r\n");
    }
    ret=xrtGraphWait(ghdl,0);
    if(ret){
	printf("Graph wait error\r\n");
    }else{
	printf("Graph wait ok\r\n");
    }
    xrtRunWait(noisegen_rhdl);
    xrtRunSetArg(noisegen_rhdl, 1, size);
    xrtRunStart(noisegen_rhdl);
    printf("run noisegen\n");
    ret=xrtGraphUpdateRTP(ghdl,"gr.fir24.in[1]",(char*)wide_filter,12*sizeof(int));
    if(ret!=0){
	printf("Graph RTP update fail\n");
	return 1;
    }
    ret=xrtGraphRun(ghdl,16);
    if(ret){
	printf("Graph run error\r\n");
    }else{
	printf("Graph run ok\r\n");
    }
#endif
    // wait for s2mm done
    auto state = xrtRunWait(s2mm_rhdl);
    printf("s2mm completed with status %d\r\n",state);
	
    xrtBOSync(out_bohdl, XCL_BO_SYNC_BO_FROM_DEVICE , output_size_in_bytes,/*OFFSET=*/ 0);

    std::ofstream out("out.txt",std::ofstream::out);
    std::ifstream golden("data/filtered.txt",std::ifstream::in);
    short g_real=0,g_imag=0;
    int match = 0;
    for (int i = 0; i < OUTPUT_SIZE; i++) {
	golden >> std::dec >> g_real;
	golden >> std::dec >> g_imag;
	if(g_real!=host_out[i].real() || g_imag!=host_out[i].imag()){
		
          printf("ERROR: i=%d gold.real=%d gold.imag=%d out.real=%d out.imag=%d\r\n",i,g_real,g_imag,host_out[i].real(),host_out[i].imag());
		match=1;
	}
	out<<host_out[i].real()<<" "<<host_out[i].imag()<<" "<<std::endl;
    }
    out.close();
    golden.close();

#if __USE_ADF_API__
    gr.end();
#else
    ret=xrtGraphEnd(ghdl,0);
    if(ret){
	printf("Graph end error"\r\n);;
    }
    xrtRunClose(noisegen_rhdl);
    xrtKernelClose(noisegen_khdl);
    xrtGraphClose(ghdl);
#endif
    xrtRunClose(s2mm_rhdl);
    xrtKernelClose(s2mm_khdl);
    xrtBOFree(out_bohdl);
    xrtDeviceClose(dhdl);


    char pPass[]= "PASSED";
    char pFail[]= "FAILED";
    char* presult;
    presult = (match ? pFail : pPass); 
    printf("TEST %s\r\n",presult);

    return (match ? EXIT_FAILURE :  EXIT_SUCCESS);
}

The XRT API has C and C++ versions for controlling the PL kernels. For more information about the C++ version of the XRT API, see XRT Native APIs.