The following are some of the different C2H stream packets.
Regular Packet
The regular C2H packet has both the data packet and Completion (CMPT) packet. They are a one-to-one match.
The regular C2H data packet can be multiple beats.
-
s_axis_c2h_ctrl_qid
= C2H descriptor queue ID. -
s_axis_c2h_ctrl_len
= length of the packet. -
s_axis_c2h_mty
= empty byte should be set in last beat. -
s_axis_c2h_ctrl_has_cmpt
=1'b1
. This data packet has a corresponding CMPT packet.
The regular C2H CMPT packet is one beat.
-
s_axis_c2h_cmpt_ctrl_qid
= Completion queue ID of the packet. This can be different from the C2H descriptor QID. -
s_axis_c2h_cmpt_ctrl_cmpt_type
=HAS_PLD
. This completion packet has a corresponding data packet. -
s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id
= This completion packet has to wait for the data packet with this ID to be sent before the CMPT packet can be sent.
When the user application sends the data packet, it must count the packet ID for each packet. The first data packet has a packet ID of 1, and it increments for each data packets.
For the regular C2H packet, the data packet and the completion packet
is a one-to-one match. Therefore, the number of data packets with s_axis_c2h_ctrl_has_cmpt
as 1'b1
should be equal to the number of CMPT packet with s_axis_c2h_cmpt_ctrl_cmpt_type
as HAS_PLD
.
The QDMA Subsystem for PCIe has a shallow completion input FIFO of depth 2. For better performance, add FIFO for completion input as shown in the diagram below. Depth and width of the FIFO depends on the use case. Width is dependent on the largest CMPT size for the application, and depth is dependent on performance needs. For best performance for 64 Byte CMPT, a depth of 512 is recommended.
When the user application sends the data payload, it counts every
packet. The first packet starts with a pkt_pld_id
of 1. The second packet has a pkt_pld_id
of 2, and
so on. It is a 16-bits counter once the count reaches 16'hffff it wraps around to 0
and count forward.
The user application defines the CMPT type.
- If the
s_axis_c2h_cmpt_ctrl_cmpt_type
is HAS_PLD, the CMPT has a corresponding data payload. The user application must placepkt_pld_id
of that packet in thes_axis_c2h_cmpt_ctrl_wait_pld_pkt_id
field. The DMA will only send out this CMPT after it sends out the corresponding data payload packet. - If the
s_axis_c2h_cmpt_ctrl_cmpt_type
is NO_PLD_NO_WAIT, the CMPT does not have any data payload, and it does not need to wait for payload. Then the DMA will send out this CMPT. - If the
s_axis_c2h_cmpt_ctrl_cmpt_type
is NO_PLD_BUT_WAIT, the CMPT does not have a corresponding data payload packet. The CMPT must wait for a particular data payload packet before the CMPT is sent out. Therefore, the user application must place thepld_pkt_id
of that particular data payload into thes_axis_c2h_cmpt_ctrl_wait_pld_pkt_id
field. The DMA will not send out the CMPT until the data payload with thatpld_pkt_id
is sent out.
Immediate Data Packet
The user application can have a packet that only writes to the Completion Ring without having a corresponding data packet transfer to the host. This type of packet is called immediate data packet. For the immediate data packet, the QDMA will not send the data payload, but it will write to the CMPT Queue. The immediate packet does not consume a descriptor.
For the immediate data packet, the user application only sends the CMPT packet to the DMA, and it does not send the data packet.
The following is the setting of the immediate completion packet. There is no corresponding data packet.
In some applications, the immediate completion packet does not need
to wait for any data packet. But in some applications, it might still need to wait
for the data payload packet. When the completion type is NO_PLD_NO_WAIT
, the completion packet can be sent out without waiting
for any data packet. When the completion type is NO_PLD_BUT_WAIT
, the completion packet must specify the data packet ID
that it needs to wait for.
-
s_axis_c2h_cmpt_user_cmpt_type
=NO_PLD_NO_WAIT
orNO_PLD_BUT_WAIT
. -
s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id
= Do not increment packet count.
Marker Packet
The C2H Stream Engine of the QDMA provides a way for the user application to
insert a marker into the QDMA along with a C2H packet. This marker then propagates
through the C2H Engine pipeline and comes out on the Queue status port interface.
The marker is inserted by setting the marker bit in the C2H Stream packet. The
marker response is indicated by the QDMA to the user application by setting the
qsts_out_op[7:0] = 0x0
(CMPT Marker response)
bits on the Queue status ports. For a marker packet, QDMA does not send out a
payload packet but still writes to the Completion Ring. Not all marker responses are
generated because of a corresponding marker request. The QDMA some times generates
marker responses when it encounters exceptional events. See the following section
for details about when QDMA internally generates marker responses.
The primary purpose of giving the user application the ability of
sending in a marker into QDMA is to determine when all the traffic prior to the
marker has been flushed. This can be used in the shut down sequence in the user
application. Although not a requirement, the marker can be sent by the user
application with the user_trig
bit set when
sending in the marker into QDMA. This allows the QDMA to generate an interrupt and
truly ensures that all traffic prior to the marker is flushed out. The QDMA
Completion Engine takes the following actions when it receives a marker from the
user application:
The DMA also has an option not to send completion information to
completion ring during a Marker packet. Port s_axis_c2h_cmpt_ctrl_no_wrb_marker
can be set during a marker packet.
This option disables any write to completion ring when that Marker packet is
generated. When this signal is set for a Maker packet, the DMA has no data or
completion transfers, but will respond with maker response on qsts_out_op[7:0]
.
- Sends the Completion that came along with the marker to the C2H Stream Completion Ring.
- Sends lower 24bits of completion data to the Queue status data
port
qsts_out_data[26:3]
. - Generates Status Descriptor if enabled (if
user_trig
was set when maker was inserted). - Generates an Interrupt if enabled and not outstanding.
- Sends the marker response. If an Interrupt was not sent due to
it being enabled but outstanding, the
retry marker_req
bit in the marker response is set to inform the user that an Interrupt could not be sent for this marker request. See the Queue status ports interface description for details of these fields.
The marker packet has both the data packet and CMPT packet. They are one-to-one match.
The following is the setting of the data packet with marker:
- 1 beat of data
-
s_axis_c2h_ctrl_marker
=1'b1
-
s_axis_c2h_ctrl_len
= data width (for example, 64 if data width is 512 bits) -
s_axis_c2h_mty
= 0 -
s_axis_c2h_ctrl_has_cmpt
=1'b1
- 1 beat of CMPT packet
-
s_axis_c2h_cmpt_ctrl_marker
=1'b1
-
s_axis_c2h_cmpt_ctrl_cmpt_type
=HAS_PLD
-
s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id
= This completion packet has to wait for the data payload packet with this ID to be sent before the CMPT packet is sent. -
s_axis_c2h_cmpt_ctrl_no_wrb_marker
=1'b0
. If this is set to1'b1
then no CMPT packet.
The immediate data packet and the marker packet do not consume the descriptor; instead, they write to the C2H Completion Ring. The software needs to size the C2H Completion Ring large enough to accommodate the outstanding immediate packets and the marker packets.
When a marker request is received by the DMA and the completion ring is full, the marker response will be sent out. However, because the completion ring is full, the completion entry will be dropped and the queue will be invalidated.
Zero Length Packet
The length of the data packet can be zero. On the input, the user needs to send one beat of data. The zero length packet consumes the descriptor. The QDMA will send out 1DW payload data.
The following is the setting of the zero length packet:
- 1 beat of data
-
s_axis_c2h_ctrl_len
= 0 -
s_axis_c2h_mty
= 0
Disable Completion Packet
The user application can disable the completion for a specific packet. The QDMA provides direct memory access (DMA) to the payload, but does not write to the C2H Completion Ring. The user application only sends the data packet to the DMA, and does not send the CMPT packet.
The following is the setting of the disable completion packet:
-
s_axis_c2h_ctrl_has_cmpt
=1'b0