Enable the Tandem Configuration Solution - 3.4 English

Versal Adaptive SoC CPM Mode for PCI Express Product Guide (PG346)

Document ID
PG346
Release Date
2024-05-30
Version
3.4 English

An option in the CIPS customization GUI allows you to pick that option that suits your needs. In the CPM options, a Tandem Configuration selection is enabled when a PCIe Endpoint is selected for PCIe Controller 0 or Controller 1. The three available options are:

  • Tandem PROM
  • Tandem PCIe
  • None
Figure 1. Customizing the PCIe Controllers

Tandem PROM is the simpler mode for Tandem Configuration, where both stages reside in a single programming image. If 120 ms enumeration is required the selection of this option essentially comes for free, as there is no overhead in the design complexity or requirement for programmable logic to build. The programming ordering simply starts with the CPM and other necessary elements before moving on to the rest of the device.

Because Tandem PCIe uses the PCIe link to program the stage 2 portion of the design, the design must include connectivity from the enable CPM master(s) to the PMC slave. This should be accomplished through the block design connectivity. The Tandem PCIe option is not selectable until the CPM to NoC 0 interface is enabled as highlighted in red in the figure above.

Note: When PCIE Controller 0 is set to DMA, the CPM interface is set automatically with appropriate mapping of the slave in the CPM master address space(s). This includes enabling the CPM to NoC 0 interface by checking the appropriate box on the CPM Basic Customization page.

The specific aperture within the PMC slave that must be accessible from the host is the slave boot interface (SBI) that is available at Versal device address 0x102100000.

Figure 2. CPM Master to PMC Slave Connection for Loading Tandem PCIe Stage 2 to SBI

Figure 3. Slave Boot Interface FIFO Assigned in CPM Address Map

To deliver stage 2 images using MCAP VSEC, this advanced feature must be enabled during the IP customization. Within the Configuration options for the Controller instance to be used:

  1. Under the Basic tab, set the PCIe[0/1] Basic/Advanced mode selection option to Advanced.
  2. Under the Advanced Options tab, confirm that the check box for MCAP Capability is checked.
    Important: The MCAP VSEC can only natively address the lower 4 GB of the address map, as it can only issue 32-bit address transactions. To reach the SBI buffer address, the NoC NMU address remapping capability must be employed, see the following recommended command.
    set_property CONFIG.REMAPS {M00_AXI {{0xF122_0000 0x1_0122_0000 64K} {0xF210_0000 0x1_0210_0000 64K}}} 
    [get_bd_intf_pins /axi_noc_0/S00_AXI]
Figure 4. MCAP Capability

To deliver stage 2 images using PCIe DMA, the DMA BAR must be set to BAR0. The driver probes BAR0 to find the DMA BAR. If this access does not terminate internally in the CPM hard block and instead goes to the PL, the transaction can not be completed, because the PL is not yet configured. For more information on Tandem PCIe using QDMA or XDMA, see Versal Adaptive SoC CPM DMA and Bridge Mode for PCI Express Product Guide (PG347).

Figure 5. PCIe BARs

Confirmation that the Vivado parameters and Tandem Configuration in general are applied can be seen in the log when write_device_image is run. Following is a snippet of the log for a Tandem PROM run during the write_device_image step:

Creating bitstream...
Tandem stage1 bitstream contains 23552 bits.
Writing CDO partition ./design_1_wrapper_tandem1.rcdo...
Writing NPI partition ./design_1_wrapper_tandem1.rnpi...
Creating bitstream...
Tandem stage2 bitstream contains 5939712 bits.
Writing CDO partition ./design_1_wrapper_tandem2.rcdo...
Writing NPI partition ./design_1_wrapper_tandem2.rnpi...
Writing NPI partition ./design_1_wrapper_shutdown.txt...
Generating bif file design_1_wrapper_tandemPROM.bif for Tandem PROM.

The resulting run creates (in addition to the files mentioned above) a single .pdi image for this design called design_1_wrapper.pdi.

When Tandem PCIe is enabled through CIPS IP customization, two .pdi files are generated:
tandem1.pdi
This file should be added to the device configuration flash.
tandem2.pdi
This file should be programmed into the device through the PCIe link once it becomes active.
The resulting report in the write_device_image log looks nearly identical, but the file name for the .bif is slightly different:
Creating bitstream...
Tandem stage1 bitstream contains 23552 bits.
Writing CDO partition ./design_1_wrapper_tandem1.rcdo...
Writing NPI partition ./design_1_wrapper_tandem1.rnpi...
Creating bitstream...
Tandem stage2 bitstream contains 5939712 bits.
Writing CDO partition ./design_1_wrapper_tandem2.rcdo...
Writing NPI partition ./design_1_wrapper_tandem2.rnpi...
Writing NPI partition ./design_1_wrapper_shutdown.txt...
Generating bif file ./design_1_wrapper_tandem1.bif for Tandem stage-1.

In addition to the files mentioned above, the resulting run creates two .pdi images for this design called design_1_wrapper_tandem1.pdi and design_1_wrapper_tandem2.pdi. The _tandem1 and _tandem2 suffixes are automatically added to differentiate the stages.

Important: Stage 1 and stage 2 bitstreams must remained paired. While this is trivial for Tandem PROM because both stages are stored in a single PDI image, this is a critical consideration for Tandem PCIe. If any part of the design is modified such that a full recompilation is triggered, both stage 1 and stage 2 images must be updated. Always update both stages when any change is made.
Note: Tandem PDI generation for new devices is gated until a device reaches production status. A parameter is available to build images for pre-production or ES silicon; contact support for information and to confirm that there are no issues with the new device. Without the parameter, write_device_image is expected to fail with the following error:

ERROR: [Vivado 12-4165] Tandem Configuration bitstream generation is not supported for part <device>.

Important: Do NOT use this solution on engineering silicon (ES) for VC1902, VC1802 or VM1802 devices.

In UltraScale+, the Field Updates solution enables you to build Reconfigurable Stage Twos where one could not only pick a stage 2 image from a list of compatible images, but then also reconfigure that stage 2 area with another stage 2 image to provide dynamic field updates. In a Versal device, the solution is similar but not exactly the same. The first part (for the initial boot of a device) can be supported in the future to allow you to lock a stage 1 image in a small local boot flash; the second part (dynamic reconfiguration) requires a Tandem + DFX-based approach to allow for dynamic reconfiguration of a subsection of the PL.

For test and debug purposes the HD.TANDEM_BITSTREAMS property can be set on the implemented design before .pdi file generation to separate a single Tandem PROM .pdi file into separate tandem1.pdi and tandem2.pdi files.

set_property HD.TANDEM_BITSTREAMS Separate [current_design]

Similarly, the behavior of Tandem PROM or Tandem PCIe file generation can be disabled entirely by using the HD.TANDEM_BITSTREAMS property on the implemented design before .pdi file generation. The following command can be used to do this.

set_property HD.TANDEM_BITSTREAMS NONE [current_design]