input_gmio/output_gmio and external_buffer - 2024.2 English - UG1603

AI Engine-ML Kernel and Graph Programming Guide (UG1603)
Document ID
UG1603
Release Date
2024-11-28
Version
2024.2 English

This class represents the global memory (DDR) resource management and data transfer between AI Engine-ML and global memory (DDR). The input_gmio object manages data transfer from global memory (DDR) to AI Engine-ML or read from global memory (DDR) operation. The output_gmio object manages data transfer from AI Engine-ML to global memory or write to global memory (DDR) operation. All data transfers are standard 1D linear transfers.

The object external_buffer represents the interface tile DMA with its connections to the DDR but also to the AI Engine-ML array in both directions. Read and write access scheme to the DDR can be specified using tiling parameter. For details, see AI Engine-ML External Memory Access and Tiling Parameters and Buffer Descriptors.

Base Class Member Functions

static void* malloc(size_t size);
The malloc method allocates contiguous physical memory space and returns the corresponding virtual address. It accepts a parameter, size, to specify how many bytes to be allocated. If successful, the pointer to the allocated memory space is returned. nullptr is returned in the event of a failure.
static void free(void* address);
The free method frees memory space allocated by GMIO::malloc.
return_code wait();
The wait method blocks until all the previously issued transactions are complete. This method is only applicable for GMIO objects connected to AI Engine-ML.
Note: The input_gmio and output_gmio classes are derived from GMIO base class. Functions malloc(), free(), and wait() are declared in the GMIO base class but it applies also to external_buffer.

input_gmio Member Functions

create(std::string logical_name, size_t burst_length, size_t bandwidth);

The above port specification is used to connect DDR memory to AI Engine kernels. logical_name is the name of the port. The burst_length is the length of DDR memory burst transaction (can be 64, 128, or 256 bytes), and the bandwidth is the average expected throughput in MB/s.

create(size_t burst_length, size_t bandwidth);
The above port specification is used to connect DDR memory to AI Engine kernels. The burst_length is the length of DDR memory burst transaction (can be 64, 128, or 256 bytes), and the bandwidth is the average expected throughput in MB/s.
return_code gm2aie_nb(const void* address, size_t transaction_size);
The gm2aie_nb method initiates a DDR to AI Engine-ML transfer. The memory space for the transaction is specified by the address pointer and the transaction_size parameter (in bytes). The transaction memory space needs to be within the total memory space allocated by the GMIO::malloc method. It is a non-blocking function in that it does not wait for the read transaction to complete.
return_code gm2aie(void* address, size_t transaction_size);

The gm2aie method is a blocking version of gm2aie_nb. It blocks until the AI Engine-ML to DDR read transaction completes.

output_gmio Member Functions

create(std::string logical_name, size_t burst_length, size_t bandwidth);
The above port specification is used to connect AI Engine kernels to DDR memory. logical_name is the name of the port. The burst_length is the length of DDR memory burst transaction (can be 64, 128, or 256 bytes), and the bandwidth is the average expected throughput in MB/s.
create(size_t burst_length, size_t bandwidth);
The above port specification is used to connect AI Engine kernels to DDR memory. The burst_length is the length of DDR memory burst transaction (can be 64, 128, or 256 bytes), and the bandwidth is the average expected throughput in MB/s.
return_code aie2gm_nb(void* address, size_t transaction_size);
The aie2gm_nb method initiates an AI Engine-ML to DDR transfer. The memory space for the transaction is specified by the address pointer and the transaction_size parameter (in bytes). The transaction memory space needs to be within the total memory space allocated by the GMIO::malloc method. It is a non-blocking function in that it does not wait for the write transaction to complete.
return_code aie2gm(void* address, size_t transaction_size);

The aie2gm method is a blocking version of aie2gm_nb. It blocks until the AI Engine-ML to DDR write transaction completes.

external_buffer Member Functions

create(const std::vector<uint32_t>& dimension, size_t numInputs, size_t numOutputs)
The above buffer specification is used to declare a point of connection located in an interface tile which enables communication between the DDR memory and the AI Engine-ML array. dimension is a vector defining the size of the buffer along a maximum of three dimensions. numInputs is the number of input connections and numOutputs is the number of output connections.
void setAddress(void* address);
The setAddress method sets the base address of the external buffer.
return_code aie2gm_nb(adf::port<adf::input>& in);
The aie2gm_nb method initiates an AI Engine-ML to DDR transfer. The size and the read access scheme is determined by the tiling parameters that are set in the graph. It is a non-blocking function in that it does not wait for the read transaction to complete.
return_code wait(adf::port<adf::input>& in);
Wait for the transactions on the external buffer input port to complete. This is a blocking API.
return_code gm2aie_nb(adf::port<adf::output>& out);
The gm2aie_nb method initiates a DDR to AI Engine-ML transfer. The size and write access scheme is determined by the tiling parameters that are set in the graph. It is a non-blocking function in that it does not wait for the read transaction to complete.
return_code gm2aie_nb(adf::port<adf::output>& out);

Wait for the transactions on the external buffer output port to complete. This is a blocking API.

Note: It is recommended using GMIO::malloc() and GMIO::free() to manage DDR memory resources.