The AIE-ML v2 is a highly-optimized processor featuring single-instruction multiple-data (SIMD) and very-long instruction word (VLIW) processor that supports both fixed-point and floating-point precision. As shown in the following figure, the AIE-ML v2 has a memory interface, a scalar unit, a vector unit, two load units, one store unit, and an instruction fetch and decode unit.
Figure 1.
AIE-ML v2
The features of the AIE-ML v2 include:
- Instruction-based VLIW SIMD processor
- 32-bit scalar RISC processor
- Scalar register files and special registers
- 32 x 32-bit multiplier
- 32-bit add/subtract
- ALU operations like shifts, compares, and logical operations.
- Hardware acceleration for inverse, square root, and inverse square root.
- Vector Multiplication Unit
- Vector unit supporting MAC operations for multiple precisions (for example, 512x 8-bit × 8-bit and 512x 4-bit × 4-bit)
- Sparsity is supported for all integer and floating point modes except for MX block floating point (50% sparsity)
- Support for floating point multiplication (float8, bfloat16, and float16) accumulating in single precision floating point (fp32). For fp8, several formats are supported, such as E4M3 and E5M2, and with different representation capabilities for infinity, NaNs and zeros.
- Support for multiplying MX block floating point types and accumulating in floating point. The MX9 type has eight bits per block element (sign and mantissa in twos complement), eight bits for the shared exponent, and eight bits for shared sub-tile shifts. The MX6 type has five bits per block element (sign and mantissa in twos complement), eight bits of shared exponent, and eight bits of shared sub-tile shifts. The MX4 type has three bits per block element (sign and mantissa in twos complement), eight bits of shared exponent, and eight bits of shared sub-tile shifts.
- The multiplier|multiplicand can be signed or unsigned. The accumulator is always signed.
- The accumulation can be performed in several operation modes: 64 lanes of 32 bits, or 32 lanes of 64 bits.
- The total number of multipliers and the number of accumulation lanes determine the depth of the post-adding.
-
Table 1. Supported Precision Width of the Vector Data Path Precision 1 Precision 2 Number of Accumulator Lanes Bits per Accumulator Lane Number of MACs int8 int8 64 32 512 int16 int16 64 32 128 int16 int16 32 64 128 int32 int16 32 64 64 bfloat16 bfloat16 32 SPFP 256 float16 float16 32 SPFP 256 float8 float8 64 SPFP 512 MX6 MX6 64 SPFP 1024 MX9 MX9 64 SPFP 512 - Multiplication of 4-bit by 4- bit can be emulated.
- Multiplication of 32-bit by 32-bit numbers can be emulated by decomposition into multiplications of 32x16-bit.
- Single precision floating point (SPFP) per the IEEE standard.
- MX4 multiplication is emulated using MX9.
- Vector Addition Unit
- Vector unit supporting 8, 16 or 32-bit addition, subtraction, comparison and min/max computation
- Support for non-linear functions: tanh, exp2 (bfloat16 with tanh and exp2, float16 with exp2)
- Support processing of two 512-bit wide vectors
- Includes comparisons and min/max computation for bfloat16/float16 vector
-
Load/Store Units
- For loading/storing data and weights
- AGU handles optimized address generation for ML functionality
- Two 512-bit load and one 512-bit store units with aligned addresses
- Supports 2D/3D addressing modes for ML functionality
- Ports to Streaming interconnect switch
- 2×32-bit subordinate port
- 1×32-bit manager port
- Processor bus interface: The processor bus allows the AIE-ML v2 to perform direct read/write access to local tile memory mapped registers.