Interrupt Aggregation Ring - Interrupt Aggregation Ring - 3.4 English - PG347

For indirect interrupt, it does interrupt aggregation. The following are some restrictions for the interrupt aggregation.

• Each Interrupt Aggregation Ring can only be associated with one function. But multiple rings can be associated with the same function.
• The interrupt engine supports up to three interrupts from same source, until software services the interrupts.

In the indirect interrupt, the QDMA processes the interrupt with the following steps.

• Look up the QID to Vector Table.
• Look up the Interrupt Context.
• Write to the Interrupt Aggregation Ring.
• Send out the PCIe MSI-X message.

This block diagram is of the indirect interrupt.

The Interrupt Context includes the information of the Interrupt Aggregation Ring. It has 256 entries to support up to 256 Interrupt Aggregation Rings.

The following is the Interrupt Context Structure (0x8).

The software needs to size the Interrupt Aggregation Ring appropriately. Each source can send up to three messages to the ring. Therefore, the size of the ring needs satisfy the following formula.

Number of entry >= 3 * (number of queues + error interrupts that are mapped to this ring)

The Interrupt Context is programmed by the context access. The QDMA_IND_CTXT_CMD.Qid has the ring index, which is from the Qid to Vector Table. The operation of MDMA_CTXT_CMD_CLR can clear all of the bits in the Interrupt Context. The MDMA_CTXT_CMD_INV can clear the valid bit.

• Context access through QDMA_TRQ_SEL_IND:
  • QDMA_IND_CTXT_CMD.Qid = Ring index
  • QDMA_IND_CTXT_CMD.Sel = MDMA_CTXT_SEL_INT_COAL (0x8)
  • QDMA_IND_CTXT_CMD.cmd.Op =
    • MDMA_CTXT_CMD_WR,
    • MDMA_CTXT_CMD_RD,
    • MDMA_CTXT_CMD_CLR, or
    • MDMA_CTXT_CMD_INV.

After it looks up the Interrupt Context, it then writes to the Interrupt Aggregation Ring. It also updates the Interrupt Context with the new PIDX, color, and the interrupt state.

This is the Interrupt Aggregation Ring entry structure. It has 8B data.

The following is the information in the stat_desc.

When the software allocates the memory space for the Interrupt Aggregation Ring, the coal_color starts with 1’b0. The software needs to initialize the color bit of the Interrupt Context to be 1’b1. When the hardware writes to the Interrupt Aggregation Ring, it reads color bit from the Interrupt Context, and writes it to the entry. When the ring (PIDX) wraps around, the hardware will flip the color bit in the Interrupt Context. In this way, when the software reads from the Interrupt Aggregation Ring, it will know which entries got written by the hardware by looking at the color bit.

The software reads the Interrupt Aggregation Ring to get the qid, the int_type (H2C or C2H), and the err_int. From the qid, the software can identify it the queue is stream or MM.

When the err_int is set, it is an error interrupt. The software can then check the error status register of the Central Error Aggregator QDMA_GLBL_ERR_STAT (0x248). The register shows the error source. The software can then read the error status register of the Leaf Error Aggregator of the corresponding error.

The stat_desc in the Interrupt Aggregation Ring is the status descriptor from the Interrupt source. When the status descriptor is disabled, the software can get the status descriptor information from the Interrupt Aggregation Ring.

The two cases are as follows:

• The interrupt source is C2H stream, then it is the status descriptor of the C2H Completion Ring. The software can read the pidx of the C2H Completion Ring.
• The interrupt source is others (H2C stream, H2C MM, C2H MM), then it is the status descriptor of that source. The software can read the cidx.

Finally, the QDMA sends out the PCIe MSI-X message using the interrupt vector from the Interrupt Context.

When the PCIe MSI-X interrupt is received by the Host, the software reads the Interrupt Aggregation Ring to determine which queue needs service. After the software reads the Interrupt Aggregation Ring, it will do a dynamic pointer update for the software CIDX to indicate the cumulative pointer that the software reads to. The software does the dynamic pointer update using the register QDMA_DMAP_SEL_INT_CIDX[2048] (0x6400). If the software cidx is equal to the pidx, this will trigger a write to the Interrupt Context on the interrupt state of that queue. This is to indicate the QDMA that the software already reads all of the entries in the Interrupt Aggregation Ring. If the software cidx is not equal to the pidx, it will send out another PCIe MSI-X message. Therefore, the software can read the Interrupt Aggregation Ring again. After that, the software can perform a pointer update of the interrupt source ring. For example, for a C2H stream interrupt, the software will update the pointer of the interrupt source ring, which is the C2H Completion Ring.

These are the steps for the software:

1. After the software gets the PCIe MSI-X message, it reads the Interrupt Aggregation Ring entries.
2. The software uses the coal_color bit to identify the written entries. Each entry has Qid and Int_type (H2C or C2H). From the Qid and Int_type, the software can check if it is stream or MM. This points to a corresponding source ring. For example, if it is C2H stream, the source ring is the C2H Completion Ring. The software can then read the source ring to get information, and do a dynamic pointer update of the source ring after that.
3. After the software finishes reading of all written entries, it does one dynamic point update of the software cidx using the register QDMA_DMAP_SEL_INT_CIDX[2048] (0x6400). The Qid in the register is the Qid in the last written entry. This tells hardware the pointer of the Interrupt Aggregation Ring that the software reads to.

   If the software cidx is not equal to the PIDX, the hardware will send out another PCIE MSI-X message, so that the software can read the Interrupt Aggregation Ring again.

When the software performs the dynamic point update for the Interrupt Aggregation Ring using the register QDMA_DMAP_SEL_INT_CIDX[2048] (0x6400), it needs to use the virtual qid. The FMAP block in the hardware translates the virtual qid to absolute qid. The interrupt Engine uses the absolute qid when it looks up the qid to Vector Table.

The following diagram shows the indirect interrupt flow. The Interrupt module gets the interrupt requests. It first writes to the Interrupt Aggregation Ring. Then it waits for the write completions. After that, it sends out the PCIe MSI-X message. The interrupt requests can keep on coming, and the Interrupt module keeps on processing them. In the meantime, the software reads the Interrupt Aggregation Ring and it does the dynamic pointer update. If the software CIDX is not equal to the PIDX, it will send out another PCIe MSI-X message.