AI Engine-ML Kernel and Graph Programming Guide (UG1603) - 2024.1 English

Document ID: UG1603
Release Date: 2024-06-06
Version: 2024.1 English

Overview
Navigating Content by Design Process
AI Engine-ML Architecture Overview
AI Engine-ML Memory
Tools
Single Kernel Programming
AI Engine API
Kernel Pragmas
Kernel Compiler
Kernel Simulation
Kernel Inputs and Outputs
Introduction to Scalar and Vector Programming
AI Engine API Overview
AI Engine-ML Data Types
Vector Registers
Accumulator Registers
Rounding and Saturation Modes
Casting and Datatype Conversion
Vector Initialization, Load, and Store
Alignment
Initialization
Load and Store
Load and Store From Buffer Interface
Load and Store Using Pointers
Load and Store Using Streams
Load and Store with Virtual Resource Annotations
Multi-dimensional Addressing in AI Engine Kernels
Vector Arithmetic Operations
Pre-Multiplication Operations
Vector Reduction
Bitwise Operations
Data Comparison
Data Reshaping
Iterators
Operator Overloading
Floating-Point Operations
Floating-Point Accuracy
Multiple Lanes Multiplication
Sliding Multiplication
Multi-Channel Sliding Multiplication
accumulate
Matrix Multiplications - mmul
Sparse Matrix Multiplication
Lookup Tables and Linear Approximation
Parallel Lookup Tables
Linear Approximation
API Operation Examples
Kernel Optimization
Loops
Loop Flattening and Unrolling
Software Pipelining of Loops
Scheduling Separator
Restrict Keyword
Inline Keywords
Profiling Kernel Code
Using Vitis Unified IDE and Reports
Example Design Using the AI Engine API
Matrix Multiplication
Vectorized Matrix Multiplication
Re-arranging Data in AI Engine-ML Memory Tile
Introduction to Graph Programming
Prepare the Kernels
Creating a Data Flow Graph (Including Kernels)
Runtime Ratio
Recommended Project Directory Structure
Compiling and Running the Graph from the Command Line
Memory and DMA Programming
AI Engine-ML Local Memory Access
AI Engine-ML Memory Tile Access
AI Engine-ML External Memory Access
Tiling Parameters and Buffer Descriptors
Tiling Parameters Specification
Tiling Parameters Examples
2D Matrix Transpose Using Tiling Parameters
Viewing Tiling Parameters in the Vitis IDE
Multi-Level AI Engine-ML Data Access
Examples on Memory and DMA Programming
Use Shared Buffer Up to 1 MB
Memory Tile Packet Split to Multiple Destinations
Deterministic Merge to Memory Tile
S2MM DMA Finish on TLAST
Input and Output Buffers
Buffer Port-Based Access
Buffer Port Attributes and API
Data Access Mechanisms
Synchronous Buffer Port Access
Synchronous Buffer Port With Margin Configuration
Buffer Port Multicasting
Buffer Port Connection for Multirate Processing
Asynchronous Buffer Port Access
Buffer Port Data Types
Access Data From a Buffer Port
Buffer Port Iterators
Scalar Iterators
Vector Iterators
Examples
Linear and Circular Addressing of Buffer Ports
Circular Input Buffer
Circular Output Buffer
Asynchronous Circular Buffer
Buffer Port Operations for Kernels
Moving the Current Read/Write Position Forward
Moving the Current Read/Write Position Backward
Streaming Data API
Data Access Mechanisms
Stream-Based Access
Stream Operations for Kernels
Stream and Cascade Data Types
Reading and Advancing an Input Stream
Writing and Advancing an Output Stream
Packet Stream Operations
Packet Stream Reading and Writing
Packet Processing
Interface Considerations
Data Movement Between AI Engines
Data Communication via AI Engine Data Memory
Data Communication via Memory and DMA
Data Communication via Memory Tile
Data Communication via AXI4-Stream Interconnect
Buffer vs. Stream in Data Communication
Free-Running AI Engine Kernel
Runtime Parameter Specification
AI Engine and PL Kernels Data Communication
DDR Memory Access through GMIO
Runtime Graph Control API
Graph Execution Control
Basic Iterative Graph Execution
Finite Execution of Graph
Infinite Graph Execution
Parallel Graph Execution
Timed Execution
Runtime Parameter Specification
Specifying Runtime Data Parameters
Runtime Parameter Update/Read Mechanisms
Parameter Update/Read Using Graph APIs
Runtime Graph Reconfiguration Using Control Parameters
Runtime Parameter Support Summary
Specialized Graph Constructs
Lookup Tables
Static File-Scoped Tables
Global Graph-Scoped Tables
Shared Graph-Scoped Tables
FIFO Depth
Stream Switch FIFO
Kernel Bypass
Explicit Packet Switching
Packet Switching Graph Constructs
Packet Switching and the aiesimulator
Location Constraints
Kernel Location Constraints
Buffer Location Constraints
Shared Buffer Location Constraint
FIFO Location Constraints
Area Location Constraints
Hierarchical Constraints
Relative Constraints
Buffer Allocation Control
C++ Kernel Class Support
C++ Template Support
Multicast Support
AI Engine-ML/Programmable Logic Integration
Design Flow Using RTL Programmable Logic
Design Considerations for Graphs Interacting with Programmable Logic
PL Interface Tile Capabilities
AI Engine-to-PL Rate Matching
AI Engine-ML to PL Interface Performance
Graph Programming Model
Graph Topologies
Overview
Single Kernel Graph
Linking Multiple Sub-Graphs
Hierarchical Graphs
Recursive Graphs
Graphs with Libraries Components
Programming Model Features
Configuring input_plio/output_plio
Configuring input_gmio/output_gmio
Programming Model for AI Engine-ML to DDR Memory Connection
Hardware Emulation and Hardware Flows
Performance Comparison Between AI Engine/PL and AI Engine/NoC Interfaces
Graph Programming Comparison Between AI Engine and AI Engine-ML
Tools and Flows
Compiling an AI Engine Graph Application
Simulating an AI Engine Graph Application
Performance Analysis of AI Engine Graph Application during Simulation
Performance Analysis of AI Engine Graph Application on Hardware
Programming the PS Host Application
Integrating the Application Using the Vitis Tools Flow
Using the Vitis IDE
Debugging the AI Engine Application
Mapper/Router Methodology
Adaptive Data Flow Graph Specification Reference
Return Code
Graph Objects
graph
kernel
port<T>
parameter
bypass
Graph Objects for Packet Processing
Input/Output Objects
input_gmio/output_gmio and external_buffer
input_plio/output_plio
Connections
Connection Constructor Templates
Port Combinations
Constraints
constraint<T>
constraint< std::vector<T>>
Mapping Constraints
Other Constraints
JSON Constraints
Node Constraints
Tile Constraint
Shim Constraint
Reserved Memory Constraint
Colocated Nodes Constraint
Non-colocated Nodes Constraint
Colocated Reserved Memories Constraint
Non-colocated Reserved Memories Constraint
Relative Constraints
Port Constraints
Buffers Constraint
Shared Buffers Constraint
Colocated Nodes Constraint
Not Colocated Nodes Constraint
Colocated Ports Constraint
Not Colocated Ports Constraint
Exclusive Colocated Ports Constraint
Colocated Reserved Memories Constraint
Not Colocated Reserved Memories Constraint
FIFO Constraint
Global Constraints
Area Group Constraint
IsomorphicGraphGroup Constraint
Using the Restrict Keyword in AI Engine Kernels
Pointer Aliasing
Strict Aliasing Rule
Restrict Keyword
Restrict Qualification
Undefined Behavior
Scope of Restrict Keyword in Inline Function
Benefits of Using the Restrict Keyword for Read/Modify/Write Loops
Derived Pointers
Summary
Additional Resources and Legal Notices
Finding Additional Documentation
Support Resources
References
Revision History
Please Read: Important Legal Notices

Back to home page