Load and Store Using Buffer Streams - 2024.2 English - UG1603

AI Engine-ML Kernel and Graph Programming Guide (UG1603)

Document ID
UG1603
Release Date
2024-11-28
Version
2024.2 English

If iterators are not appropriate due to semantic or performance concerns, buffer streams can be used instead. Unlike iterators, Buffer streams follow a stream interface, which implies that a read or write operation will advance the stream, altering its current state.

These buffer streams are available for various datatypes:
  • Standard types: (u)int8, (u)int16, (u)int32, bfloat16, ...
  • Sparse vectors of standard type.
  • Block-floating-point data.
  • Tensor of native types to handle multidimensional addressing.

Input and Output Buffer Stream

In the function signature you have a standard buffer defined using input_buffer and output_buffer. To access this buffer, you will need to use a specific construct called a buffer_stream.. For example:
void TestFunction(adf::input_buffer<int32,adf::extents<1024>> & bin,
adf::output_buffer<int32,adf::extents<1024>> & bout)
{
  aie:input_buffer_stream<int32,32> in_bufs((int32 *)bin.data());
  aie:output_buffer_stream<int32,32> out_bufs((int32 *)bout.data());
  aie::vector<int32,32> v;

  for(int i=0;i<1024/32;i++)
  {
    in_bufs >> v; // Get the next vector from the buffer bin
    out_bufs << v; // Stores variable v into the next available address in buffer bout
  }
}

Tensor Buffer Streams

An abstraction technique is applied to address a single dimention stream in a multidimensional address map . The multidimensional access scheme is described using aie::make_tensor_descriptor<DataType,VectorLength. For example:
auto desc = aie::make_tensor_descriptor<int16, 32>(
                     aie::tensor_dim(2u, 4),
                     aie::tensor_dim(2u, 2),
                     aie::tensor_dim(2u, 1));

The three parameters aie::tensor_dim(NbElem, Step) describe the multidimensional volume on dimension 2, 1 and 0.

NbElem
The number of elements to consider sequentially on the corresponding dimension.
Step
The address incremental steps to use on the corresponding dimension relative to the first element.

Sparse-Vector Buffer Stream

Sparse vectors don't have a fixed length in memory, which is why they cannot be accessed through standard iterators. Instead, a variant of the buffer stream class can be used to load them:
aie::sparse_vector_input_buffer_stream<Type,Size> StreamName(Type * DataPointer);