Introduction to Scalar and Vector Programming - Introduction to Scalar and Vector Programming - 2026.1 English - UG1603

This section provides an overview of the key elements of kernel programming for scalar and vector processing elements. The following sections cover details of each element and optimization requirements.

The following example uses only the scalar engine. It demonstrates a for loop iterating through buffers of 512 int16 elements. Each loop iteration performs a single multiply of an int16 element a with an int16 element b storing the result in c and writing it to an output buffer.

Iterators read from the input buffers and write to the output buffer. For details on the iterators, refer to Iterators.

void scalar_mul(input_buffer<int16>& __restrict data1, 
    input_buffer<int16>& __restrict data2, 
    output_buffer<int32>& __restrict out){ 
  // iterator to access input "data1"  
  auto inIter1=aie::begin(data1); 
  auto inIter2=aie::begin(data2);
  // iterator to access output "out"
  auto outIter=aie::begin(out); 

  for(int i=0;i<512;i++) { 
    // read data from buffer and point to next entry
    int16 a=*inIter1++; 
    int16 b=*inIter2++; 
    int32 c=a*b; 
    // write result to buffer and point to next entry
    *outIter++=c;    
    }
}

The following example is a vectorized version for the same kernel that is executed on the vector processor.

void vect_mul(  input_buffer<int16>& __restrict data1, 
     input_buffer<int16>& __restrict data2,
     output_buffer<int32>& __restrict out){ 
  // iterator to access a vector (a collection of elements) 
  // in the buffer "data1"
  auto inIter1=aie::begin_vector<16>(data1);
  // iterator to access a vector (a collection of elements) 
  // in the buffer "data2"
  auto inIter2=aie::begin_vector<16>(data2);
  // iterator to access a vector (a collection of elements) 
  // in the buffer "out"
  auto outIter=aie::begin_vector<16>(out);

  for(int i=0;i<512/16;i++)
  chess_prepare_for_pipelining { 
     //read 8 elements from the buffer and point to the next entry
     auto va=*inIter1++;
     auto vb=*inIter2++;

     //element-by-element multiplication, with results 
     // in an accumulator register
     auto vt=aie::mul(va,vb); 

     // move data from accumulator register to vector register 
     // with a shift of zero, and transfer to output buffer; 
     // increment the iterator to point to the next entry
     *outIter++=vt.to_vector<int32>(0);
    }
}

The input buffer is read using special vector iterators that produce aie::vector<int16,16> values. A for loop iterates through buffers of 512 int16 elements, sixteen samples at a time. va and vb are two vectors of sixteen int16 elements, loaded using the vector iterators. Each loop iteration performs the multiplication of va by vb element by element, storing the result in vc and writing it to an output buffer. The output of aie::mul is an accumulator vector that is reduced by the to_vector function which returns a value of type aie::vector<int32,16> that is written to the output buffer. The following sections cover details on the data types supported by the AIE-ML and AIE-ML v2 architectures.

The __restrict keyword used on the input and output parameters of the functions allows for more aggressive compiler optimization by explicitly stating independence between data.

chess_prepare_for_pipelining is a compiler pragma that explicitly directs the kernel compiler to do pipelining for the loop. It can introduce code inlining with the directive, and thus affects program memory size.

The scalar version of this example function requires approximately 1040 cycles of execution time, while the vectorized and optimized version requires around 84 cycles. There is more than a 10x speedup in terms of execution time for the vectorized version of the kernel.

The following sections describe these elements in detail:

• Supported data types
• Available registers
• Optimizations that can be achieved on the AI Engine using concepts such as software pipelining in loops, and keywords such as __restrict.