QDMA Subsystem for PL PCIE4 and PL PCIE5

Versal ACAP DMA and Bridge Subsystem for PCI Express Product Guide (PG344)

Document ID: PG344
Release Date: 2022-05-20
Version: 1.0 English

Introduction to the DMA and Bridge Subsystems
Modular IP Architecture
QDMA Subsystem
AXI Bridge Subsystem
XDMA Subsystem
Features of the DMA and Bridge Subsystem
QDMA Subsystem
AXI Bridge Subsystem
XDMA Subsystem
Maximum Supported Link Rates and Widths
IP Facts
Overview
Navigating Content by Design Process
Unsupported Features
Standards for the DMA and Bridge Subsystems
Minimum Device Requirements
Licensing and Ordering
Designing with the Subsystem
Clocking
Resets
QDMA Subsystem for PL PCIE4 and PL PCIE5
Overview
Modular IP Architecture
Features
QDMA Architecture
DMA Engines
Descriptor Engine
H2C MM Engine
C2H MM Engine
H2C Stream Engine
C2H Stream Engine
Completion Engine
Bridge Interfaces
AXI Memory Mapped Bridge Master Interface
AXI4-Lite Bridge Master Interface
PCIe to AXI BARs
AXI Memory Mapped Bridge Slave Interface
AXI4-Lite Bridge Slave CSR Interface
AXI to PCIe BARs
Interrupt Module
PCIe Block Interface
PCIe CQ/CC
PCIe RQ/RC
PCIe Configuration
General Design of Queues
H2C and C2H Queues
Completion Queue
SR-IOV Support
QDMA Subsystem Limitations
Applications
Product Specification
Performance and Resource Utilization
QDMA Operations
Descriptor Engine
Descriptor Context
Software Descriptor Context Structure (0x0 C2H and 0x1 H2C)
Hardware Descriptor Context Structure (0x2 C2H and 0x3 H2C)
Credit Descriptor Context Structure
Descriptor Fetch
Internal Mode
Internal Mode Writeback and Interrupts (AXI MM and H2C ST)
Descriptor Bypass Mode
Descriptor Bypass Mode Writeback/Interrupts
Marker Response
Traffic Manager Output Interface
Descriptor Credit Input Interface
Errors
Memory Mapped DMA
Operation
Errors
AXI Memory Mapped Descriptor for H2C and C2H (32B)
AXI Memory Mapped Writeback Status Structure for H2C and C2H
Stream Mode DMA
H2C Stream Engine
Internal and Bypass Modes
H2C Stream Descriptor (16B)
Descriptor Metadata
Zero Length Descriptor
H2C Stream Status Descriptor Writeback
H2C Stream Data Aligner
Handling Descriptors With Errors
Handling Errors in Data From PCIe
C2H Stream Engine
C2H Stream Descriptor (8B)
C2H Prefetch Engine
C2H Stream Modes
C2H Stream Packet Type
Handling Descriptors With Errors
Completion Engine
Completion Context Structure
Completion Status Structure
Completion Entry Structure
Completion Input Packet
Completion Status/Interrupt Moderation
Completion Timer
Handling Exception Events
Bridge
Slave Bridge
BDF Table
Address Translation
Slave Address Translation Examples
Master Bridge
Interrupts
Asynchronous and Queue Based Interrupts
Interrupt Engine
Direct Interrupt
Interrupt Aggregation Ring
Interrupt Context Structure
Interrupt Aggregation Entry
Interrupt Flow
Error Interrupt
Legacy Interrupt
User Interrupt
Queue Management
Function Map Table
Context Programming
Queue Setup
Queue Teardown
Virtualization
Mailbox
Function Level Reset
Port ID
Host Profile
System Management
Resets
VDM
Config Extend
Expansion ROM
Errors
Bridge Errors
Slave Bridge Abnormal Conditions
Illegal Burst Type
Completion TLP Errors
Unexpected Completion
Unsupported Request
Completion Timeout
Poison Bit Received on Completion Packet
Completer Abort
PCIe Error Handling
Master Bridge Abnormal Conditions
AXI DECERR Response
AXI SLVERR Response
Max Payload Size for PCIe, Max Read Request Size
Completion Packets
Poison Bit
Zero Length Requests
Linkdown Errors
Data Path Errors
DMA Errors
Error Aggregator
C2H Streaming Fatal Error Handling
Port Descriptions
QDMA Global Ports
QDMA Global Ports
AXI Bridge Master Ports
AXI Bridge Slave Ports
AXI4-Lite Master Ports
AXI4-Lite Slave Ports
AXI4-Lite Slave CSR Ports
AXI4 Memory Mapped DMA Ports
AXI4-Stream H2C Ports
AXI4-Stream C2H Ports
AXI4-Stream C2H Completion Ports
AXI4-Stream Status Ports
AXI4-Stream C2H Write Completion Ports
VDM Ports
Configuration Extend Interface Ports
FLR Ports
QDMA Descriptor Bypass Input Ports
QDMA Descriptor Bypass Output Ports
QDMA Descriptor Credit Input Ports
QDMA Traffic Manager Credit Output Ports
User Interrupts
Queue Status Ports
Register Space
QDMA PF Address Register Space
QDMA_CSR (0x0000)
QDMA_TRQ_SEL_QUEUE_PF (0x18000)
QDMA_DMAP_SEL_INT_CIDX[2048] (0x18000)
QDMA_DMAP_SEL_H2C_DSC_PIDX[2048] (0x18004)
QDMA_DMAP_SEL_C2H_DSC_PIDX[2048] (0x18008)
QDMA_DMAP_SEL_CMPT_CIDX[2048] (0x1800C)
QDMA_PF_MAILBOX (0x22400)
Function Status Register (0x22400)
Function Command Register (0x22404)
Function Interrupt Vector Register (0x22408)
Target Function Register (0x2240C)
Function Interrupt Vector Register (0x22410)
RTL Version Register (0x22414)
PF Acknowledgment Registers (0x22420-0x2243C)
FLR Control/Status Register (0x22500)
Incoming Message Memory (0x22C00-0x22C7C)
Outgoing Message Memory (0x23000-0x2307C)
QDMA_TRQ_MSIX (0x30000)
QDMA VF Address Register Space
QDMA_TRQ_SEL_QUEUE_VF (0x3000)
QDMA_TRQ_MSIX_VF (0x4000)
QDMA_VF_MAILBOX (0x5000)
Function Status Register (0x5000)
Function Command Register (0x5004)
Function Interrupt Vector Register (0x5008)
Target Function Register (0x500C)
Function Interrupt Control Register (0x5010)
RTL Version Register (0x5014)
Incoming Message Memory (0x5800-0x587C)
Outgoing Message Memory (0x5C00-0x5C7C)
AXI4-Lite Slave CSR Register Space
Bridge Register Space
DMA Register Space
AXI4-Lite Slave Register Space
Design Flow Steps
Customizing and Generating the Subsystem
Basic Tab
Capabilities Tab
PCIe BARs Tab
SRIOV Config Tab
SRIOV VF BARs Tab
PCIe MISC Tab
AXI BARs Tab
PCIe DMA Tab
User Parameters
Output Generation
Constraining the Subsystem
Simulation
Basic Simulation
PIPE Mode Simulation
Synthesis and Implementation
Example Design
Available Example Designs
AXI Memory Mapped and AXI4-Stream With Completion Default Example Design
AXI Memory Mapped Example Design
AXI Stream with Completion Example Design
Example Design with Descriptor Bypass In/Out Loopback
Example Design Registers
C2H_ST_QID (0x000)
C2H_ST_LEN (0x004)
C2H_CONTROL_REG (0x008)
H2C_CONTROL_REG (0x00C)
H2C_STATUS (0x010)
C2H_STATUS (0x018)
C2H_PACKET_COUNT (0x020)
C2H_PREFETCH_TAG(0x024)
C2H_COMPLETION_DATA_0 (0x030)
C2H_COMPLETION_DATA_1 (0x034)
C2H_COMPLETION_DATA_2 (0x038)
C2H_COMPLETION_DATA_3 (0x03C)
C2H_COMPLETION_DATA_4 (0x040)
C2H_COMPLETION_DATA_5 (0x044)
C2H_COMPLETION_DATA_6 (0x048)
C2H_COMPLETION_DATA_7 (0x04C)
C2H_COMPLETION_SIZE (0x050)
SCRATCH_REG0 (0x060)
SCRATCH_REG1 (0x064)
C2H_PACKETS_DROP (0x088)
C2H_PACKETS_ACCEPTED (0x08C)
DESCRIPTOR_BYPASS (0x090)
USER_INTERRUPT (0x094)
USER_INTERRUPT_MASK (0x098)
USER_INTERRUPT_VECTOR (0x09C)
DMA_CONTROL (0x0A0)
VDM_MESSAGE_READ (0x0A4)
Customizing and Generating the Example Design
Test Bench
Architecture
Scaled Simulation Timeouts
Test Selection
Available Tests
Verilog Test Selection
Waveform Dumping
Verilog Flow
Output Logging
Test Description
Model Task List
PCIe-Related Tasks
DMA Tasks
Application Software Development
Device Drivers
Linux QDMA Software Architecture (PF/VF)
Using the Drivers
Reference Software Driver Flow
AXI4 Memory Map Flow Chart
AXI4 Memory Mapped C2H Flow
AXI4 Memory Mapped H2C Flow
AXI4-Stream Flow Chart
AXI4-Stream C2H Flow
AXI4-Stream H2C Flow
Debugging
Finding Help on Xilinx.com
Documentation
Debug Guide
Answer Records
Master Answer Record for the Subsystem
QDMA Debugging Answer Record
Technical Support
AXI Bridge Subsystem for PL PCIE4 and PL PCIE5
Overview
Modular IP Architecture
Feature Summary
AXI Bridge Subsystem Limitations
PCIe Capability
Product Specification
Performance and Resource Utilization
AXI Bridge Operations
AXI Transactions for PCIe
Transaction Ordering for PCIe
BAR and Address Translation
BDF Table
Address Translations
Address Translation Examples
Malformed TLP
Abnormal Conditions
Slave Bridge Abnormal Conditions
Illegal Burst Type
Completion TLP Errors
Unexpected Completion
Unsupported Request
Completion Timeout
Poison Bit Received on Completion Packet
Completer Abort
Master Bridge Abnormal Conditions
AXI DECERR Response
AXI SLVERR Response
Max Payload Size for PCIe, Max Read Request Size or 4K Page Violated
Completion Packets
Poison Bit
Zero Length Requests
Link Down Behavior
Endpoint
Interrupts
MSI-X Interrupts
Root Port
Enhanced Configuration Access Memory Map
Power Limit Message TLP
Root Port Configuration Read
Root Port BAR
AXI BAR in Root Port Mode
Configuration Transaction Timeout
Abnormal Configuration Transaction Termination Responses
Receiving Interrupts
MSI-X Interrupt
MSI Interrupt
Legacy INTx Interrupt
AXI Bridge Port Descriptions
Global Signals
AXI Slave Interface
AXI4-Lite Slave CSR Interface
Interrupt Signals
Register Space
AXI4-Lite Slave CSR Register Space
Bridge Register Space
DMA Register Space
AXI4-Lite Slave Register Space
Design Flow Steps
Customizing and Generating the Subsystem
Basic Tab
Capabilities Tab
PCIe BARs Tab
PCIe MISC Tab
PCIe MISC Tab in Root Port Mode
AXI BARs Tab
Output Generation
Example Design
Endpoint Configuration
Xilinx AXI Verification IP Attached to the AXI Slave Interface
Customizing and Generating the Example Design
Simulation Design Overview
Implementation Design Overview
Example Design Elements
Debugging
Finding Help on Xilinx.com
Documentation
Solution Centers
Answer Records
Master Answer Record for the Subsystem
AXI Bridge Debugging Answer Record
Technical Support
Debug Tools
Third-Party Tools
XDMA Subsystem for PL PCIE4
Overview
Modular IP Architecture
Feature Summary
Applications
XDMA Subsystem Limitations
PCIe Capability
Product Specification
Performance and Resource Utilization
Configurable Components of the Subsystem
Target Bridge
H2C Channel
C2H Channel
AXI4-Lite Master
AXI4-Lite Slave
Host-to-Card Bypass Master
IRQ Module
Legacy Interrupts
MSI and MSI-X Interrupts
Config Block
XDMA Operations
Quick Start
Initial Setup For H2C and C2H Transfers
AXI-MM Transfer For H2C
AXI-MM Transfer For C2H
Descriptors
Descriptor Bypass
Poll Mode
DMA H2C Stream
DMA C2H Stream
Address Alignment
Length Granularity
Parity
Port Descriptions
XDMA Global Ports
H2C Channel 0-3 AXI4-Stream Interface Signals
C2H Channel 0-3 AXI4-Stream Interface Signals
AXI4 Memory Mapped Read Address Interface Signals
AXI4 Memory Mapped Read Interface Signals
AXI4 Memory Mapped Write Address Interface Signals
AXI4 Memory Mapped Write Interface Signals
AXI4 Memory Mapped Write Response Interface Signals
AXI4 Memory Mapped Master Bypass Read Address Interface Signals
AXI4 Memory Mapped Master Bypass Read Interface Signals
AXI4 Memory Mapped Master Bypass Write Address Interface Signals
AXI4 Memory Mapped Master Bypass Write Interface Signals
AXI4 Memory Mapped Master Bypass Write Response Interface Signals
Config AXI4-Lite Memory Mapped Write Master Interface Signals
Config AXI4-Lite Memory Mapped Read Master Interface Signals
Config AXI4-Lite Memory Mapped Write Slave Interface Signals
Config AXI4-Lite Memory Mapped Read Slave Interface Signals
Interrupt Interface
Channel 0-3 Status Ports
Configuration Extend Interface Port Descriptions
Configuration Management Interface Ports
Descriptor Bypass Mode
Register Space
PCIe to AXI Bridge Master Address Map
PCIe to DMA Address Map
PCIe to DMA Address Format
PCIe to DMA Configuration Registers
H2C Channel Registers (0x0)
H2C Channel Identifier (0x00)
H2C Channel Control (0x04)
H2C Channel Control (0x08)
H2C Channel Control (0x0C)
H2C Channel Status (0x40)
H2C Channel Status (0x44)
H2C Channel Completed Descriptor Count (0x48)
H2C Channel Alignments (0x4C)
H2C Poll Mode Low Write Back Address (0x88)
H2C Poll Mode High Write Back Address (0x8C)
H2C Channel Interrupt Enable Mask (0x90)
H2C Channel Interrupt Enable Mask (0x94)
H2C Channel Interrupt Enable Mask (0x98)
H2C Channel Performance Monitor Control (0xC0)
H2C Channel Performance Cycle Count (0xC4)
H2C Channel Performance Cycle Count (0xC8)
H2C Channel Performance Data Count (0xCC)
H2C Channel Performance Data Count (0xD0)
C2H Channel Registers (0x1)
C2H Channel Identifier (0x00)
C2H Channel Control (0x04)
C2H Channel Control (0x08)
C2H Channel Control (0x0C)
C2H Channel Status (0x40)
C2H Channel Status (0x44)
C2H Channel Completed Descriptor Count (0x48)
C2H Channel Alignments (0x4C)
C2H Poll Mode Low Write Back Address (0x88)
C2H Poll Mode High Write Back Address (0x8C)
C2H Channel Interrupt Enable Mask (0x90)
C2H Channel Interrupt Enable Mask (0x94)
C2H Channel Interrupt Enable Mask (0x98)
C2H Channel Performance Monitor Control (0xC0)
C2H Channel Performance Cycle Count (0xC4)
C2H Channel Performance Cycle Count (0xC8)
C2H Channel Performance Data Count (0xCC)
C2H Channel Performance Data Count (0xD0)
IRQ Block Registers (0x2)
IRQ Block Identifier (0x00)
IRQ Block User Interrupt Enable Mask (0x04)
IRQ Block User Interrupt Enable Mask (0x08)
IRQ Block User Interrupt Enable Mask (0x0C)
IRQ Block Channel Interrupt Enable Mask (0x10)
IRQ Block Channel Interrupt Enable Mask (0x14)
IRQ Block Channel Interrupt Enable Mask (0x18)
IRQ Block User Interrupt Request (0x40)
IRQ Block Channel Interrupt Request (0x44)
IRQ Block User Interrupt Pending (0x48)
IRQ Block Channel Interrupt Pending (0x4C)
IRQ Block User Vector Number (0x80)
IRQ Block User Vector Number (0x84)
IRQ Block User Vector Number (0x88)
IRQ Block User Vector Number (0x8C)
IRQ Block Channel Vector Number (0xA0)
IRQ Block Channel Vector Number (0xA4)
Config Block Registers (0x3)
Config Block Identifier (0x00)
Config Block BusDev (0x04)
Config Block PCIE Max Payload Size (0x08)
Config Block PCIE Max Read Request Size (0x0C)
Config Block System ID (0x10)
Config Block MSI Enable (0x14)
Config Block PCIE Data Width (0x18)
Config PCIE Control (0x1C)
Config AXI User Max Payload Size (0x40)
Config AXI User Max Read Request Size (0x44)
Config Write Flush Timeout (0x60)
H2C SGDMA Registers (0x4)
H2C SGDMA Identifier (0x00)
H2C SGDMA Descriptor Low Address (0x80)
H2C SGDMA Descriptor High Address (0x84)
H2C SGDMA Descriptor Adjacent (0x88)
H2C SGDMA Descriptor Credits (0x8C)
C2H SGDMA Registers (0x5)
C2H SGDMA Identifier (0x00)
C2H SGDMA Descriptor Low Address (0x80)
C2H SGDMA Descriptor High Address (0x84)
C2H SGDMA Descriptor Adjacent (0x88)
C2H SGDMA Descriptor Credits (0x8C)
SGDMA Common Registers (0x6)
SGDMA Identifier Registers (0x00)
SGDMA Descriptor Control Register (0x10)
SGDMA Descriptor Control Register (0x14)
SGDMA Descriptor Control Register (0x18)
SGDMA Descriptor Credit Mode Enable (0x20)
SG Descriptor Mode Enable Register (0x24)
SG Descriptor Mode Enable Register (0x28)
MSI-X Vector Table and PBA (0x8)
Design Flow Steps
Customizing and Generating the Subsystem
Basic Tab
PCIe ID Tab
PCIe BARs Tab
PCIe MISC Tab
PCIe DMA Tab
Output Generation
Example Design
Available Example Designs
AXI4 Memory Mapped Default Example Design
AXI4 Memory Mapped with PCIe to AXI4-Lite Master and PCIe to DMA Bypass Example Design
AXI4 Memory Mapped with AXI4-Lite Slave Interface Example Design
AXI4-Stream Example Design
AXI4 Memory Mapped with Descriptor Bypass Example
User IRQ Example Design
Customizing and Generating the Example Design
Application Software Development
Device Drivers
Linux Device Driver
Using the Driver
Interrupt Processing
Legacy Interrupts
MSI Interrupts
MSI-X Interrupts
User Interrupts
Example H2C Flow
Example C2H Flow
Debugging
Finding Help on Xilinx.com
Documentation
Debug Guide
Solution Centers
Answer Records
Master Answer Record for the Subsystem
XDMA Debugging Answer Record
Technical Support
Hardware Debug
General Checks
Initial Debug of the XDMA
Upgrading
Limitations
GT Selection and Pin Planning
PL PCIe GT Selection
CPM4 Additional Considerations
GT Locations
Assigning GT Locations
GT Quad Locations
PCIe Link Debug Enablement
Enabling PCIe Link Debug
Connecting to PCIe Link Debug in Vivado
Additional Resources and Legal Notices
Xilinx Resources
Documentation Navigator and Design Hubs
References
Revision History
Please Read: Important Legal Notices

Back to home page