The AI Engine-ML scalar unit supports signed and unsigned integers in 8, 16, and 32-bit widths, along with some single-precision floating-point for specific operations.
The two main vector data types offered by the AI Engine API are vectors (aie::vector) and accumulators (aie::accum).
aie::broadcast((bfloat16){1}) is
equivalent to aie::broadcast<bfloat16,32>((bfloat16){1}) where specifying the
size as <bfloat16,32> is optional because it
is the default vector size for the bfloat16 data
type.| Vector Data Type | Supported Sizes (Default Highlighted ) |
|---|---|
| int8 | 16/32/64/128 |
| uint8 | 16/32/64/128 |
| int16 | 8/16/32/64 |
| uint16 | 8/16/32/64 |
| int32 | 4/8/16/32 |
| uint32 | 4/8/16/32 |
| float | 4/8/16/32 |
| cint16 | 4/8/16/32 |
| cint32 | 2/4/8/16 |
| bfloat16 | 8/16/32/64 |
| cfloat | 4/8/16 |
For example, aie::vector<int32,16> is a 16-element vector of integers with 32
bits. Each element of the vector is referred to as a lane. Using the smallest bit width necessary can improve performance by
making good use of registers.
An accumulator represents a collection of elements of the same class, typically obtained as a result of a multiplication operation, which is transparently mapped to the corresponding accumulator registers supported on each architecture. Accumulators commonly provide a larger number of bits, allowing you to perform long chains of operations whose intermediate results might exceed the range of regular vector types. Accumulators are parameterized by the element type, and the number of elements.
| acc32 | accfloat | caccfloat | acc64 | cacc64 | |
|---|---|---|---|---|---|
| Native accumulation bits | 32 | 64 | |||
The native accumulation bits define the minimum number of bits. The AI Engine API maps different accumulator types to the nearest
native accumulator type that supports the requirement. For example, acc48 maps to acc64 for
the AI Engine-ML architecture. The accumulator data
types listed can be used in the kernel code and is supported by AI Engine APIs.
There is a limited subset of the accumulator datatypes support for interfaces in the graph code. For details, see Stream and Cascade Data Types.
aie::vector and aie::accum have member functions to do type casting, data
extraction and insertion, and indexing. These operations are covered in the following
sections.
For more information on streams, see Streaming Data API.