Function Level Reset - 4.0 English

QDMA Subsystem for PCI Express Product Guide (PG302)

Document ID
PG302
Release Date
2022-05-20
Version
4.0 English

The function level reset (FLR) mechanism enables the software to quiesce and reset Endpoint hardware with function-level granularity. When a VF is reset, only the resources associated with this VF are reset. When a PF is reset, all resources of the PF, including that of its associated VFs, are reset. Since FLR is a privileged operation, it must be performed by the PF driver running in the management system.

Use Mode

  • Hypervisor requests for FLR when a function is attached and detached (i.e., power on and off).
  • You can request FLR as follows:
    echo 1 > /sys/bus/pci/devices/$BDF/reset

    where $BDF is the bus device function number of the targeted function.

FLR Process

A complete FLR process involves of three major steps.

  1. Pre-FLR: Pre-FLR resets all QDMA context structure, mailbox, and user logic of the target function.
    • Each function has a register called MDMA_PRE_FLR_STATUS, which keeps track of the pre-FLR status of the function. The offset is calculated as MDMA_PRE_FLR_STATUS_OFFSET = MB_base + 0x100, which is located at offset 0x100 from the mailbox memory space of the function. Note that PF and VF have different MB_base. The definition of MDMA_PRE_FLR_STATUS is shown in the table below.
    • The software writes 1 to MDMA_PRE_FLR_STATUS[0] (bit 0) of the target function to initiate pre-FLR. Hardware will clear MDMA_PRE_FLR_STATUS[0] when pre-FLR completes. The software keeps polling on MDMA_PRE_FLR_STATUS[0], and only proceeds to the next step when it returns 0.
    Table 1. MDMA_PRE_FLR_STATUS Register
    Offset Field R/W Type Width Default Description
    0x100 pre_flr_st RW 32 0

    [31:1] reserved.

    [0]: 1 Initiates pre-FLR.

    [0]: 0 pre-FLR done.

    bit[0] is set by the driver and cleared by the hardware.

  2. Quiesce: The software must ensure all pending transaction is completed. This can be done by polling the Transaction Pending bit in the Device Status register (in PCIe Configuration Space), until it is cleared or times out after a certain period of time.
  3. PCIe-FLR: PCIe-FLR resets all resources of the target function in the PCIe controller.
    Note: Initiate Function Level Reset bit (bit 15 of PCIe Device Control Register) of the target function should be set to 1 to trigger FLR process in PCIe.

OS Support

If the PF driver is loaded and alive (i.e., use mode 1), all three steps aforementioned are performed by the driver. However, for UltraScale+ , if a user wants to perform FLR before loading the PF driver (as defined in Use Mode above), an OS kernel patch is provided to allow OS to perform the correct FLR sequence through functions defined in //…/source/drivers/pci/quick.c.