Enable the Tandem Configuration Solution - 3.4 English

Versal Adaptive SoC CPM DMA and Bridge Mode for PCI Express Product Guide (PG347)
Document ID
PG347
Release Date
2024-05-30
Version
3.4 English

An option in the CIPS customization GUI allows you to pick which option suits your needs. In the CPM options, a Tandem Configuration selection will be enabled when a PCIe Endpoint is selected for PCIE Controller 0 or Controller 1. The three available options are:

  • Tandem PROM
  • Tandem PCIe
  • None
Figure 1. Customizing the PCIe Controllers

Tandem PROM is the simpler mode for Tandem Configuration, where both stages reside in a single programming image. If 120 ms enumeration is required the selection of this option essentially comes for free, as there is no overhead in design complexity or requirement for programmable logic to build. The programming ordering simply starts with the CPM and other necessary elements before moving on to the rest of the device.

Because Tandem PCIe uses the PCIe link to program the stage 2 portion of the design, the design must include connectivity from the enable CPM Master(s) to the PMC Slave. This should be accomplished through the block design connectivity. When PCIE Controller 0 is set to DMA, this CPM Interface is set automatically and appropriate mapping of the slave into the CPM master address space(s). This includes enabling the CPM to NoC 0 Interface by checking the appropriate box on the CPM Basic Configuration customization page. The specific aperture within the PMC slave that must be accessible from the host is the Slave Boot Interface (SBI) which is available at AMD Versal device address 0x102100000.

Figure 2. CPM Master to PMC Slave Connection for Loading Tandem PCIe Stage 2 to SBI

Figure 3. Slave Boot Interface FIFO Assigned in CPM Address Map

To deliver stage 2 images using MCAP VSEC, see Versal Adaptive SoC CPM Mode for PCI Express Product Guide (PG346).

Note: If these interfaces are not used, tie the corresponding ready signal to 1. dma0_mgmt_cpl_rdy, dma0_st_rx_msg_tready and dma_tm_dsc_sts_rdy must be tied to 1 if not used.

To deliver stage 2 images using PCIe DMA, the DMA BAR must be set to BAR0. The driver probes BAR0 to find the DMA BAR. If this goes to the PL, the transaction will not complete, because the PL is not yet configured.

Figure 4. PCIe BARs

Confirmation that Vivado parameters and Tandem Configuration in general have been applied can be seen in the log when write_device_image is run. Following is a snippet of the log for a Tandem PROM run during the write_device_image step: 

Creating bitstream...
Tandem stage1 bitstream contains 23552 bits.
Writing CDO partition ./design_1_wrapper_tandem1.rcdo...
Writing NPI partition ./design_1_wrapper_tandem1.rnpi...
Creating bitstream...
Tandem stage2 bitstream contains 5939712 bits.
Writing CDO partition ./design_1_wrapper_tandem2.rcdo...
Writing NPI partition ./design_1_wrapper_tandem2.rnpi...
Writing NPI partition ./design_1_wrapper_shutdown.txt...
Generating bif file design_1_wrapper_tandemPROM.bif for Tandem PROM.

The resulting run creates (in addition to the files mentioned above) a single .pdi image for this design called design_1_wrapper.pdi.

When Tandem PCIe is enabled through CIPS IP customization, two .pdi files are generated:

tandem1.pdi
This file should be added to the device configuration flash.
tandem2.pdi
This file should be programmed into the device through the PCIe link once it becomes active.

The resulting report in the write_device_image log looks nearly identical, but the file name for the .bif will be slightly different:

Creating bitstream...
Tandem stage1 bitstream contains 23552 bits.
Writing CDO partition ./design_1_wrapper_tandem1.rcdo...
Writing NPI partition ./design_1_wrapper_tandem1.rnpi...
Creating bitstream...
Tandem stage2 bitstream contains 5939712 bits.
Writing CDO partition ./design_1_wrapper_tandem2.rcdo...
Writing NPI partition ./design_1_wrapper_tandem2.rnpi...
Writing NPI partition ./design_1_wrapper_shutdown.txt...
Generating bif file ./design_1_wrapper_tandem1.bif for Tandem stage-1.

In addition to the files mentioned above, the resulting run creates two .pdi images for this design called design_1_wrapper_tandem1.pdi and design_1_wrapper_tandem2.pdi. The _tandem1 and _tandem2 suffixes are automatically added to differentiate the stages.

Important: Stage 1 and stage 2 bitstreams must remained paired. While this is trivial for Tandem PROM because both stages are stored in a single PDI image, this is a critical consideration for Tandem PCIe. If any part of the design is modified such that a full recompilation is triggered, both stage 1 and stage 2 images must be updated. Always update both stages when any change is made.
Note: Tandem PDI generation for new devices is gated until a device reaches production status. A parameter is available to build images for pre-production or ES silicon; contact support for information and to confirm that there are no issues with the new device. Without the parameter, write_device_image is expected to fail with the following error:

ERROR: [Vivado 12-4165] Tandem Configuration bitstream generation is not supported for part <device>.

In an AMD UltraScale+ device, the Field Updates solution enables you to build Reconfigurable Stage Twos where one could not only pick a stage 2 image from a list of compatible images, but then also reconfigure that stage 2 area with another stage 2 image to provide dynamic field updates. In an AMD Versal device, the solution is similar but not exactly the same. The first part (for the initial boot of a device) can be supported in the future to allow you to lock a stage 1 image in a small local boot flash; the second part (dynamic reconfiguration) requires a Tandem + DFX-based approach to allow for dynamic reconfiguration of a subsection of the PL.

For test and debug purposes the HD.TANDEM_BITSTREAMS property can be set on the implemented design before .pdi file generation to separate a single Tandem PROM .pdi file into separate tandem1.pdi and tandem2.pdi files.

set_property HD.TANDEM_BITSTREAMS Separate [current_design]

Similarly, the behavior of Tandem PROM or Tandem PCIe file generation can be disabled entirely by using the HD.TANDEM_BITSTREAMS property on the implemented design before .pdi file generation. The following command can be used to do this.

set_property HD.TANDEM_BITSTREAMS NONE [current_design]