Load and Store Using Buffer Streams - 2025.2 English - UG1603

If iterators are not appropriate due to semantic or performance concerns, buffer streams can be used instead. Unlike iterators, Buffer streams follow a stream interface, which implies that a read or write operation advances the stream, altering its current state.

These buffer streams are available for various datatypes:

• Standard types: (u)int8, (u)int16, (u)int32, bfloat16...
• Sparse vectors of standard type.
• Block-floating-point data.
• Tensor of native types to handle multidimensional addressing.

Input and Output Buffer Stream

In the function signature there is a standard buffer defined using input_buffer and output_buffer. To access this buffer, you need to use a specific construct called a buffer_stream. For example:

void TestFunction(adf::input_buffer<int32,adf::extents<1024>> & bin,
adf::output_buffer<int32,adf::extents<1024>> & bout)
{
  aie:input_buffer_stream<int32,32> in_bufs((int32 *)bin.data());
  aie:output_buffer_stream<int32,32> out_bufs((int32 *)bout.data());
  aie::vector<int32,32> v;

  for(int i=0;i<1024/32;i++)
  {
    in_bufs >> v; // Get the next vector from the buffer bin
    out_bufs << v; // Stores variable v into the next available address in buffer bout
  }
}

Block Floating-Point Buffer Stream

When working with block-floating-point data, it is necessary to meet specific data alignment requirements. The only way to access a buffer and load or store this data is by using a buffer stream.

void MxPassthrough(adf::input_buffer<mx9,adf::extents<32>> & bin,
adf::output_buffer<mx9,adf::extents<32>> & bout)
{
  aie:block_vector_input_buffer_stream<mx9,64> in_bufs((mx9 *)bin.data());
  aie:block_vector_output_buffer_stream<mx9,64> out_bufs((mx9 *)bout.data());
  aie::block_vector<mx9,64> v;

  for(int i=0;i<(32*16)/64;i++)
  {
    in_bufs >> v; // Get the next vector from the buffer bin
    out_bufs << v; // Stores variable v into the next available address in buffer bout
  }
}

Load from a Buffer Stream

To load a variable from the buffer, you can use the >>operator (similar to stream access in C++).

Store to a Buffer Stream

To store a variable to the buffer, you can use the << operator (similar to stream access in C++).

Both the operator >>and class function pop() retrieve the value from the stream and advance the stream's state.

Vectors are not converted to sparse vectors internally, therefore there is no sparse_vector_output_buffer class.

Tensor Buffer Streams

An abstraction technique is applied to address a single dimention stream in a multidimensional address map. The multidimensional access scheme is described using aie::make_tensor_descriptor<DataType,VectorLength>. For example:

auto desc = aie::make_tensor_descriptor<int16, 32>(
                     aie::tensor_dim(2u, 4),
                     aie::tensor_dim(2u, 2),
                     aie::tensor_dim(2u, 1));

The three parameters aie::tensor_dim(NbElem, Step) describe the multidimensional volume on dimension 2, 1 and 0.

NbElem

The number of elements to consider sequentially on the corresponding dimension.

Step

The address incremental steps to use on the corresponding dimension relative to the first element.

More detailed information is available in Multi-dimensional Addressing in AI Engine Kernels.

Sparse-Vector Buffer Stream

Sparse vectors do not have a fixed length in memory, therefore they cannot be accessed through standard iterators. Instead, a variant of the buffer stream class can be used to load them:

aie::sparse_vector_input_buffer_stream<Type,Size> StreamName(Type * DataPointer);