The Versal AI Core Series, AI Edge Series, and AI Edge Series Gen 2 deliver breakthrough AI inference acceleration with AI Engines. This series supports a breadth of applications, including cloud for dynamic workloads and network for massive bandwidth. It also provides advanced safety and security features. AI/data scientists and software/hardware developers use high compute density to accelerate performance across applications. The AI Engine has advanced signal processing compute capability. It is well-suited for highly optimized wireless applications such as radio, 5G, backhaul, and other high-performance DSP applications.
AI Engines use an array of very-long instruction word (VLIW) processors with single instruction multiple data (SIMD) vector units. The design optimizes them for compute-intensive applications. This includes digital signal processing (DSP), 5G wireless applications, and artificial intelligence (AI) technology like machine learning (ML).
AI Engines are hardened blocks that deliver multiple levels of parallelism, including instruction-level and data-level parallelism. Instruction-level parallelism includes a scalar operation, up to two moves, two vector reads (loads), one vector write (store), and one vector instruction that can be executed—in total, a 7-way VLIW instruction per clock cycle. Vector-level operations achieve data-level parallelism, allowing multiple data sets to be processed on a per-clock-cycle basis. Each AI Engine contains the following:
- It has a vector and scalar processor.
- It has a dedicated program memory.
- It has local data memory.
- It can access adjacent local memory in any of three neighboring directions.
- It has access to DMA engines and AXI4 interconnect switches to communicate via streams to other AI Engines or to the programmable logic (PL) or the DMA.
Refer to the Versal Adaptive SoC AI Engine Architecture Manual (AM009) for specific details on the AI Engine array and interfaces.
The AI Engine-ML (AIE-ML) block is capable of delivering 2x compute throughput compared to its predecessor AI Engine blocks. The AIE-ML block targets machine learning inference applications. It delivers one of the industry's best performance per Watt for a wide range of inference applications. Refer to the Versal Adaptive SoC AIE-ML Architecture Manual (AM020) for specific details on the AIE-ML features and architecture.
As an application developer, you can choose white or black box flows. Use them to run an ML inference application on AIE-ML variants. With the white box flow you can integrate custom kernels and dataflow graphs in the AIE-ML variants programming environment. A black box flow uses performance optimized Neural Processing Unit (NPU) IP from AMD to accelerate ML workloads in the AIE-ML variants.
AMD Vitis™ AI is used as a front-end tool that parses the network graph, performs optimization, quantization of the graph, and generates compiled code that can be run on the AIE-ML variants hardware. The AIE-ML variants core tile architecture provides support for a variety of precision fixed and floating-point datatypes. The architecture allows for pipe-lined vector processing and incorporates high-density, high-speed on-chip memory that can effectively store on-chip tensors. Additionally, it features versatile datamovers that are adept at handling multi-dimensional tensors in memory. Selecting the right overlay processor architecture and properly distributing input/output tensors across on/off chip memory enables high computational efficiency in AIE-ML processing core variants.
AMD provides device drivers and libraries for user applications to access the AMD Versal™ AI Engines. See AI Engine Tools and Flows User Guide (UG1076) to learn how to use the AI Engine program environment. The Vitis IDE lets you compile, simulate, and debug the different elements of a Versal device AI Engine application. For detailed information on the Vitis IDE tool flow, refer to Versal Adaptive SoC Design Guide (UG1273). For detailed information on the AI Engine tools and flows, refer to AI Engine Tools and Flows User Guide (UG1076).
The following sections describe the software stack of the AI Engine.