Kernel Optimization - 2025.2 English - UG1079

AI Engine Kernel and Graph Programming Guide (UG1079)

Document ID: UG1079
Release Date: 2025-11-26
Version: 2025.2 English

Overview
Navigating Content by Design Process
AI Engine Architecture Overview
AI Engine Memory
Tools
Single Kernel Programming
AI Engine API
Kernel Pragmas
Kernel Compiler
Kernel Simulation
Kernel Inputs and Outputs
Introduction to Scalar and Vector Programming
AI Engine API Overview
Scalar Programming
Vector Data Types
Vector Registers
Accumulator Registers
Rounding and Saturation Modes
Casting and Datatype Conversion
Vector Initialization, Load, and Store
Alignment
Initialization
Load and Store
Load and Store From Buffer Interface
Load and Store Using Pointers
Load and Store Using Streams
Load and Store with Virtual Resource Annotations
Vector Arithmetic Operations
Pre-Multiplication Operations
Vector Reduction
Bitwise Operations
Data Comparison
Data Reshaping
Iterators
Operator Overloading
Floating-Point Operations
Multiple Lanes Multiplication - sliding_mul
Matrix Multiplication - mmul
API Operation Examples
Kernel Optimization
Loops
Loop Flattening and Unrolling
Software Pipelining of Loops
Viewing Loop II in the Vitis IDE
Chess Directives and C++ Attributes
Scheduling Separator
Parallel Streams Access
Restrict Keyword
Inline Keywords
Profiling Kernel Code
Using Vitis Unified IDE and Reports
Interface Considerations
Data Movement Between AI Engines
Data Communication via AI Engine Data Memory
Data Communication via Memory and DMA
Data Communication via AXI4-Stream Interconnect
Buffer vs. Stream in Data Communication
Free-Running AI Engine Kernel
Runtime Parameter Specification
AI Engine and PL Kernels Data Communication
DDR Memory Access through GMIO
Example Designs Using the AI Engine API
FIR Filter
Design Analysis
Scalar Golden Reference
Vectorized Version using a Single Kernel
Vectorized Version using Multiple Kernels
Matrix Multiplication
Vectorized Matrix Multiplication
Data Shuffling Kernel
Introduction to Graph Programming
Prepare the Kernels
Creating a Data Flow Graph (Including Kernels)
Runtime Ratio
Recommended Project Directory Structure
Compiling and Running the Graph from the Command Line
Memory and DMA Programming
AI Engine Local Memory Access
Tiling Parameters and Buffer Descriptors
Tiling Parameters Specification
Tiling Parameters Examples
2D Matrix Transpose Using Tiling Parameters
Viewing Tiling Parameters in the Vitis IDE
Input and Output Buffers
Buffer Port-Based Access
Buffer Port Attributes and API
Data Access Mechanisms
Synchronous Buffer Port Access
Synchronous Buffer Port With Margin Configuration
Buffer Port Multicasting
Buffer Port Connection for Multirate Processing
Asynchronous Buffer Port Access
Buffer Port Data Types
Access Data From a Buffer Port
Buffer Port Iterators
Scalar Iterators
Vector Iterators
Examples
Linear and Circular Addressing of Buffer Ports
Circular Input Buffer
Circular Output Buffer
Asynchronous Circular Buffer
Buffer Port Operations for Kernels
Moving the Current Read/Write Position Forward
Moving the Current Read/Write Position Backward
Comparison between Buffer Ports and Windows
Graph Conversion
Kernel Conversion
Examples of Kernel Conversion Function Prototypes
Kernel Code Conversion
Examples of Reading From Input
Streaming Data API
Data Access Mechanisms
Stream-Based Access
Stream Operations for Kernels
Stream Data Types
Reading and Advancing an Input Stream
Writing and Advancing an Output Stream
Packet Stream Operations
Packet Stream Reading and Writing
Packet Processing
Runtime Graph Control API
Graph Execution Control
Basic Iterative Graph Execution
Finite Execution of Graph
Infinite Graph Execution
Parallel Graph Execution
Timed Execution
Runtime Parameter Specification
Specifying Runtime Data Parameters
Runtime Parameter Update/Read Mechanisms
Parameter Update/Read Using Graph APIs
Runtime Parameter Support Summary
Specialized Graph Constructs
Lookup Tables
Static File-Scoped Tables
Global Graph-Scoped Tables
Shared Graph-Scoped Tables
FIFO Depth
Stream Switch FIFO
DMA FIFO
AI Engine Tile DMA Performance
Kernel Bypass
Explicit Packet Switching
Packet Switching Graph Constructs
Example of Packet Switching between PL and AI Engine
Example of Packet Switching from an AI Engine to Multiple AI Engines
Example on Packet Switching from Multiple AI Engines to an AI Engine
Packet Split and Merge Connections
Packet Switching and the aiesimulator
Location Constraints
Kernel Location Constraints
Buffer Location Constraints
FIFO Location Constraints
PLIO and GMIO Location Constraints
Area Location Constraints
Hierarchical Constraints
Relative Constraints
Buffer Allocation Control
C++ Kernel Class Support
C++ Template Support
Multicast Support
Logical I/O Ports
Conditional Ports
Array of Graph Objects
Sharing LUTs and RTPs Parameters Between Neighboring Cores
Implementation Details for Sharing LUTs
Code Example for Sharing LUTs
Implementation Details for Sharing RTPs
Code Example for Sharing RTPs
AI Engine/Programmable Logic Integration
Design Flow Using RTL Programmable Logic
Design Considerations for Graphs Interacting with Programmable Logic
PL Interface Tile Capabilities
AI Engine-to-PL Rate Matching
AI Engine to PL Interface Performance
AI Engine to PL Interface AXI Protocol
AI Engine to PL Interface Text Input Format
Graph Programming Model
Graph Topologies
Overview
Single Kernel Graph
Linking Multiple Sub-Graphs
Hierarchical Graphs
Recursive Graphs
Graphs with Libraries Components
Programming Model Features
Configuring input_plio/output_plio
Configuring input_gmio/output_gmio
Programming Model for AI Engine–DDR Memory Connection
Hardware Emulation and Hardware Flows
Performance Comparison Between AI Engine/PL and AI Engine/NoC Interfaces
Data Throughput Estimate in Hardware
Throughput Measurement Using Timers
Throughput Measurements Using Event APIs
Throughput Measurements using Event Trace
Single Kernel Programming using Intrinsics
Intrinsics
Introduction to Scalar and Vector Programming
AI Engine Data Types
Vector Registers
Accumulator Registers
Casting and Datatype Conversion
Vector Initialization, Load, and Store
Alignment
Initialization
Load and Store
Load and Store from Vector Registers
Load and Store From Memory
Load and Store Using Pointers
Load and Store Using Streams
Load and Store with Virtual Resource Annotations
Update, Extract, and Shift
Vector Register Lane Permutations
Data Selection
Data Shuffle
Data Select
MAC Intrinsics
MAC on 32x32 bits
MAC on 32x16 bits
MAC on 16x16 bits
MAC on 8x8 bits
Options
Data Permute and MAC Examples
Loops
Floating-Point Operations
Design Analysis and Programming using Intrinsics
Matrix Vector Multiplication
Kernel Coding Bounds
Vectorization
Coding with Intrinsics
Matrix Multiplication
Kernel Coding Bounds
Vectorization
Coding with Intrinsics
Multiple Kernels Coding Example: FIR Filter
1 Gsps Implementation with Cascade Stream
Coding with Intrinsics
Adaptive Data Flow Graph Specification Reference
Return Code
Graph Objects
graph
kernel
port<T>
parameter
bypass
Graph Objects for Packet Processing
Input/Output Objects
input_gmio/output_gmio
input_plio/output_plio
Connections
Connection Constructor Templates
Port Combinations
Constraints
constraint<T>
constraint< std::vector<T>>
Mapping Constraints
Other Constraints
JSON Constraints
Node Constraints
Tile Constraint
Shim Constraint
Reserved Memory Constraint
Colocated Nodes Constraint
Non-colocated Nodes Constraint
Colocated Reserved Memories Constraint
Non-colocated Reserved Memories Constraint
Relative Constraints
Port Constraints
Buffers Constraint
Colocated Nodes Constraint
Not Colocated Nodes Constraint
Colocated Ports Constraint
Not Colocated Ports Constraint
Exclusive Colocated Ports Constraint
Colocated Reserved Memories Constraint
Not Colocated Reserved Memories Constraint
FIFO Constraint
Global Constraints
Area Group Constraint
IsomorphicGraphGroup Constraint
Using the Restrict Keyword in AI Engine Kernels
Pointer Aliasing
Strict Aliasing Rule
Restrict Keyword
Restrict Qualification
Undefined Behavior
Scope of Restrict Keyword in Inline Function
Benefits of Using the Restrict Keyword for Read/Modify/Write Loops
Derived Pointers
Summary
Additional Resources and Legal Notices
Finding Additional Documentation
Support Resources
References
Revision History
Please Read: Important Legal Notices

Back to home page