Fast Partial Reconfiguration Over PCI Express (XAPP1338)

Document ID
Release Date
1.0 English
Figure 1. Customization of the XDMA PCIe IP, basic features. To see the PCIe customization, when viewing the block diagram, double-click xdma_0.

This sample design has been set up to use a Gen3 x8 IP core, but other PCIe widths and speeds can be used. A Gen1 x1 system can typically achieve 400 MB/s transfer rates that will not saturate the ICAP interface, but can still be used and will be much faster than the MCAP path. All other link widths and speeds should saturate the ICAP interface, which maximizes configuration performance.

Under DMA Interface options, note that AXI Stream is selected.

Any PCIe site on the target device can be used, but the most efficient location will be the one next to the ICAP site, which reduces routing delays.

In the Basic tab shown above, the Tandem Configuration or Partial Reconfiguration field is set to None. This design does not utilize Tandem Configuration to meet 120 ms enumeration during the initial device configuration. However, it could, because this feature is independent of what the fast partial reconfiguration over PCIe will require.

Select PCIe : DMA.

Note: Do not select PR over PCIe in this drop-down, because that selection enables the MCAP interface for bitstream delivery, which is not used in this application.
Figure 2. Customization of the XDMA PCIe IP, DMA features

In this tab, see that the value for Number of DMA Read Channel (H2C) has been increased, and 2 is selected. On this tab, H2C means Host to Card, which is how the bitstream will move from the host to the PCIe block and eventually the ICAP. In this design, the channel is dedicated to the ICAP, but could be mixed to be used with other user application logic that has been added.

Important: It is important for developers to understand that one of the disadvantages of using the DMA for PCIe is that the configuration of the FPGA is now open to anyone using this host.

In most systems, PCIe configuration packets used to program (via the MCAP and configuration transactions) typically require root access to send. While it is beyond the scope of this application note to implement security checks, there are several options available.

First, the ICAP itself has built-in checks to only accept bitstreams that are properly formatted with the correct device information. Also, this design does not implement readback, so the bitstream cannot be read back over the PCIe link.

In terms of protecting the FPGA from being downloaded with unintended bitstreams, you could do the following:

  • Add a control register with a secret value that has to be set before data can be sent to the ICAP.
  • Use built-in device encryption techniques.

The partial bitstream delivery path in this design starts at the PCIe IP core, passes through the FIFO and data width converter, then out of the block diagram as 32-bit AXI4-Stream data.

To see M_AXIS_0_tdata connect to the input port of the ICAP (which completes the path), open design_1_wrapper.v.