Load and Store with Virtual Resource Annotations - Load and Store with Virtual Resource Annotations - 2025.1 English - UG1079

The AI Engine is able to perform several vector load or store operations per cycle. However, for the load or store operations to be executed in parallel, they must target different memory banks. In general, the compiler tries to schedule many memory accesses in the same cycle when possible but there are some exceptions. Memory accesses coming from the same pointer are scheduled on different cycles. If the compiler schedules the operations on multiple variables or pointers in the same cycle, memory bank conflicts can occur.

To avoid concurrent access to a memory with multiple variables or pointers, most memory access functions in the AI Engine API accept an enum value from aie_dm_resource defined as follows:

enum class aie_dm_resource {
  none,
  a,
  b,
  c,
  d,
  stack
};

Also, the compiler provides the following aie_dm_resource annotations to annotate different virtual resources. Accesses using types that are associated with the same virtual resource are not scheduled to access the resource at the same cycle.

__aie_dm_resource_a
__aie_dm_resource_b
__aie_dm_resource_c
__aie_dm_resource_d
__aie_dm_resource_stack

The following example shows how to annotate memory access to bind individual access to virtual resources and allow or avoid accessing memories at the same cycle.

int __aie_dm_resource_a *A;
int *B;

aie::vector<int,8> v1 = aie::load_v<8>(A);

/* Following access can be scheduled on the same cycle 
 * as the access to A since B is not annotated. 
 */
aie::vector<int,8> v2 = aie::load_v<8>(B);
 
/* Following specific access to B 
 * is annotated with the same virtual resource as A 
 * so they cannot be scheduled on the same cycle. 
 */
aie::vector<int,8> v3 = aie::load_v<8, aie_dm_resource::a>(B); 

/* vector iterator of B
 * annotated with the same virtual resource as A
 * so they cannot be scheduled on the same cycle. 
 */
auto it = aie::begin_vector<8, aie_dm_resource::a>(B); 
aie::vector<int,8> v4 = *(++it);

For example, the following code is to annotate two arrays to the same __aie_dm_resource_a that guides the compiler to not access them in the same instruction. It shows two ways to load vectors, using aie::load_v and iterators.

aie::vector<int32,8> va[32];
aie::vector<int32,8> vb[32];
int32 __aie_dm_resource_a* __restrict p_va = (int32 __aie_dm_resource_a*)va;
int32 __aie_dm_resource_a* __restrict p_vb = (int32 __aie_dm_resource_a*)vb;
auto it_b=aie::begin_vector<8>(p_vb);

// access va, vb by p_va, it_b
aie::vector<int32,8> vc;
vc=aie::load_v<8>(p_va)+*it_b;
p_va+=8;
++it_b;

void kernel_top(input_buffer<int32> & __restrict data1, 
                input_buffer<int32>& __restrict data2, ...) {
  auto w_data1 = (int32 __aie_dm_resource_a* __restrict)data1.data();
  auto w_data2 = (int32 __aie_dm_resource_b* __restrict)data2.data();
  auto pv = aie::begin_vector<8>(w_data1);
  auto pv2 = aie::begin_vector<8>(w_data2);
  auto va = *pv++;
  auto vb = *pv2++;
  ...
}

alignas(aie::vector_decl_align) static int32 coeff[256]={...};
void func(input_buffer<int32> & __restrict wa, ...... )
{
aie::vector<int32,8> v_coeff=aie::load_v<8>((int32 __aie_dm_resource_a*)coeff);
int32 __aie_dm_resource_a* __restrict p_wa = (int32 __aie_dm_resource_a*)wa.data();
auto waIter = aie::begin_vector<8>(p_wa);
aie::vector<int32,8> va;
va = *waIter;
...
}