C2H Stream Modes - 2.0 English

Versal Adaptive SoC DMA and Bridge Subsystem for PCI Express Product Guide (PG344)

Document ID
PG344
Release Date
2023-11-10
Version
2.0 English

The C2H descriptors can be from the descriptor fetch engine or C2H bypass input interfaces. The descriptors from the descriptor fetch engine are always in cache mode. The prefetch engine keeps the order of the descriptors to pair with the C2H data packets from the user. The descriptors from the C2H bypass input interfaces have one interface for both simple mode and cache mode (note that both simple bypass and cache bypass use the same interface). For simple mode, the user application keeps the order of the descriptors to pair with the C2H data packets. For cache mode, the prefetch engine keeps the order of the descriptors to pair with the C2H data packet from the user.

The prefetch context has a bypass bit. When it is 1'b1, the user application sends the credits for the descriptors. When it is 1'b0, the prefetch engine handles the credits for the descriptors.

The descriptor context has a bypass bit. When it is 1'b1, the descriptor fetch engine sends out the descriptors on the C2H bypass output interface. The user application can convert it and later loop it back to the QDMA on the C2H bypass input interface. When the bypass context bit is 1'b0, the descriptor fetch engine sends the descriptors to the prefetch engine directly.

On a per queue basis, three modes are supported.

  • Cache Internal Mode
  • Cache Bypass Mode
  • Simple Bypass Mode

The selection between Simple Bypass Mode and Cache Bypass Mode is done by setting the bypass bits in Software Descriptor Context and C2H Prefetch Context as shown in the table below.

Note: If you already have the descriptor cached on the device, there is no need to fetch one from the host and you should follow the simple bypass mode for the C2H Stream application. In simple bypass mode, do not provide credits to fetch the descriptor, and instead, you need to send in the descriptor on the descriptor bypass interface.
Note: AXI4-Stream C2H Simple Bypass mode and Cache Bypass mode both use same bypass in ports (c2h_byp_in_st_csh_* ports).
Table 1. C2H Stream Modes
c2h_byp_in port desc_byp software descriptor context C2H prefetch context
Simple bypass mode c2h_byp_in_st_csh_* 1 1
Cache bypass mode c2h_byp_in_st_csh_* 1 0
Cache internal mode N/A 0 0

Simple Bypass Mode

For simple bypass mode, the descriptor fetch engine sends the descriptors out on the C2H bypass out interface. The user application converts the descriptor and loops it back to the QDMA on the simple mode C2H bypass input interface. The user application sends the credits for the descriptors, and it also keeps the order of the descriptors.

For simple bypass transfer to work, a prefetch tag is needed and it can be fetched from the QDMA IP.

The user application must request a prefetch tag before sending any traffic for a simple bypass queue through the C2H ST engine. Invalid queues or non-bypass queues should not request any tags using this method, since it may reduce performance by freezing tags that never get used.

The prefetch tag needs to be reserved upfront before any traffic can start. One prefetch tag per target host is required. In most applications, one prefetch tag for a host is needed. In Simple Bypass mode, the tag is not tied to any descriptor ring. For the queues that share the same prefetch tag, the data and descriptors need to come in the same order. For Simple Bypass, the data and descriptors are both controlled by the user, so they need to guarantee the order is maintained.

For example when the data stream has packets in the order of Q0, Q1, and Q2, on descriptor input, you can not send Q1, Q2, Q0 etc. The order needs to be maintained.

The user application writes to the MDMA_C2H_PFCH_BYP_QID (0x1408) register with the qid for a simple bypass queue, then reads from MDMA_C2H_PFCH_BYP_TAG (0x140C) register to retrieve the corresponding prefetch tag. This tag must be driven with all bypass_in descriptors for as long as the current qid is valid. If a current qid is invalidated, a new prefetch tag must be requested with a valid qid.

Prefetched tag must be assigned to input port c2h_byp_in_st_csh_pfch_tag[6:0] for all transfers. The prefetch tag points to the CAM that stores the active queues in the prefetch engine. Also the qid that was used to prefetch tag needs to be used as the qid for all simple bypass packets. Assign the qid to dma<n>_s_axis_c2h_ctrl_qid.

The steps to fetch the prefetch tag are as follows:

  1. Software instruction:
    1. Initialize a queue (qid).
    2. Write to MDMA_C2H_PFCH_BYP_QID 0x1408 with valid qid.
    3. Read MDMA_C2H_PFCH_BYP_TAG 0x140C to obtain the prefetch tag.
    4. The prefetch tag and the qid that was used to fetch the tag should be used for all simple bypass packets. This information needs to be communicated to the user side.
  2. User side:
    1. Assign the qid used to fetch the tag to dma<n>_s_axis_c2h_ctrl_qid.
    2. Assign the actual qid of the packet transfer to dma<n>_s_axis_c2h_cmpt_ctrl_qid.
    3. Assign the prefetch tag value to c2h_byp_in_st_csh_pfch_tag.
    4. Assign the actual qid of the packet transfer to c2h_byp_in_st_csh_qid.
    Note: The c2h_byp_in_st_csh_pfch_tag[6:0] port can have the same prefetch_tag for as long as the original qid is valid.

Simple bypass flow shown below does not include fetch of the "prefetch_tag".

Figure 1. C2H Simple Bypass Mode Flow
Note: No sequence is required between descriptor bypass in, data payload and completion packets.

If you already have descriptors, there is no need to update the pointers or provide credits. Instead, send the descriptors in the descriptor bypass interface, and send the data and Completion (CMPT) packets.

Cache Bypass Mode

For Cache Bypass mode, the descriptor fetch engine sends the descriptors out on the C2H bypass output interface. The user application converts the descriptor and loops it back to the QDMA on the cache mode C2H bypass input interface. The prefetch engine sends the credits for the descriptors, and it keeps the order of the descriptors.

For Cache Internal mode, the descriptor fetch engine sends the descriptors to the prefetch engine. The prefetch engine sends out the credits for the descriptors and keeps the order of the descriptors. In this case, the descriptors do not go out on the C2H bypass output and do not come back on the C2H bypass input interfaces. In Cache Internal mode prefetch tag is maintained by the IP internally.

In Cache Bypass or Cache Internal mode, prefetch mode can be turned on which will prefetch the descriptor and will reduce transfer latency significantly. When prefetch mode is enabled, the user application can not send credits as input in QDMA Descriptor Credit input ports. Credits for all queues will be maintained by prefetch engine.

In cache bypass mode, prefetch tag is maintained by the IP internally. Signal c2h_byp_out_pfch_tag[6:0] should be looped back as an input c2h_byp_in_st_csh_pfch_tag[6:0]. The prefetch tag points to the cam that stores the active queues in the prefetch engine.

Figure 2. C2H Cache Bypass Mode Flow
Note: No sequence is required between payload and completion packets.