CCIX Example - 5.0 English

System Cache LogiCORE IP Product Guide (PG118)

Document ID
PG118
Release Date
2021-11-05
Version
5.0 English

System Cache configured in CCIX XDMA mode enables cache coherency with memory distributed throughout the system all controlled by a PCIe host. In this use case the accelerators connect to System Cache using AXI4 and System Cache provides a local cache function coherent with the rest of the CCIX memory system, without any need for the accelerators to handle the coherency protocol.

Figure 1. CCIX Request Agent (RA) XDMA System System Cache Page-1 Sheet.3 System Cache CCIX RA SystemCacheCCIXRA Process.6 Accelerator Accelerator Process.8 Accelerator Accelerator Sheet.11 Sheet.12 Sheet.13 Sheet.14 MicroBlaze MicroBlaze Sheet.15 XDMA XDMA Sheet.16 Sheet.17 Sheet.18 Sheet.19 CCIX PCIe Host CCIXPCIeHost Sheet.21 FPGA FPGA curve.484 Sheet.1 X20734-070519 X20734-070519

In the CCIX configuration up to four accelerators/kernels can be used, connected to System Cache with ordinary AXI4 interfaces. The system can be configured to use Shared Virtual Memory, SVM, via the Address Translation Services, ATS. Coherent traffic is connected to PCIe using CXS interfaces, while the optional ATS support uses AXI4-Stream. A MicroBlaze processor subsystem handles all the configuration and maintenance to make System Cache operate as a full member of the CCIX network.

Key building blocks in this solution are:
  • PCIe Host with CCIX Home Agent
  • XDMA IP core handling the PCIe connection for the CCIX communication protocol
  • System Cache configured as a Request Agent using the CCIX protocol to keep the local cache coherent with the rest of the CCIX memory system, and the Address Translation Cache (ATC) to handle virtual address translation
  • MicroBlaze processor based sub-system controlling System Cache to provide Designated Vendor-Specific Capability, DVSEC, to support the CCIX protocol
  • Up to four accelerators connected via AXI4 to System Cache

System Cache can also be connected to the CPM in Versal Premium devices, which can extend the coherency domain to enable cache coherency with memory distributed throughout the system, all controlled by a CCIX enabled PCIe host. The CPM has the ability to connect to one System Cache core with CCIX enabled.

Figure 2. CCIX RA CPM System
The CPM configuration is similar to the XDMA example with the following extensions:
  • The system can be configured to use Shared Virtual Memory, SVM, via the Address Translation Services, ATS.
  • System Cache and CPM supports the ability to use Host propagated PASID, which enables each accelerator to use a unique ID when requesting address translation via ATS.
Figure 3. CCIX XDMA Component Overview System Cache Page-1 Rectangle.507 System Cache System Cache Rectangle.3 FE FE Rectangle.4 Cache Cache Rectangle.5 RA RA Rectangle.6 ICN ICN Rectangle.7 Rectangle.22 ICN ICN Rectangle.23 SA SA Rectangle.24 Rectangle.25 ICN ICN Rectangle.26 HA HA Rectangle.27 Mem Mem Sheet.28 AXI AXI Sheet.29 Sheet.32 Sheet.33 Sheet.34 Sheet.35 Sheet.36 Sheet.37 Sheet.38 Sheet.39 Sheet.40 Sheet.41 Sheet.42 Sheet.43 Sheet.44 Sheet.45 Sheet.46 Sheet.47 Sheet.48 Sheet.49 Sheet.55 CCIX CCIX Ellipse.508 Sheet.58 Front end (FE), handles arbitration between accelerators Requ... Front end (FE), handles arbitration between acceleratorsRequest Agent (RA), performs read and write transactions to different addressesHome Agent (HA), is responsible for memory address range in systemSlave Agent (SA), provides additional memory in system. Always accessed through Home AgentMemory (Mem)Coherent Interconnect (ICN) Sheet.1 X23069-080219 X23069-080219
Figure 4. CCIX CPM Component Overview