ERNIC RX Path - ERNIC RX Path - 4.0 English

The ERNIC RX Path gets the packet data from the MAC through the AXI4-Stream interface. All incoming packets are validated and all packet headers that fail packet validation are sent to the error buffer (base address specified by ERRBUFBA[31:0]) if the value of XRNICCONF[5] is set to 1. The header is prefixed with an error syndrome by the ERNIC RX packet handler module as per Table: Packet Validation Error Syndrome . These buffers provide useful debug information for incoming packet errors. RX path implements logic to detect ECN marked packets and logs it in an interrupt status bit corresponding to that QP in CNPSCHDSTS*REG register. On reception of ECN marked packet an interrupt is generated to notify driver. Driver generates a CNP packet for that QP and schedule it through QP1 instead of QPN. Rx path implements logic to detect incoming CNP packets and notifies QP manager. QP manager reduces the outstanding on corresponding QP from 16 to 8. Further reception of CNPs will not have any effect on the outgoing traffic.

Table 2-3: Packet Validation Error Syndrome
Error Syndrome Bit Index	Error related to	Precedence	Description and impact	Impact
0	MAC	4	MAC destination address received in the Ethernet header does not match the ERNIC MAC address configured	Packet dropped
1	Reserved
2	IPv4/IPv6	5	IP Version not as per ERNIC configuration	Packet dropped
3	IPv4	7	IPv4 header length not equal to 20 bytes	Packet dropped
4	Reserved
5	IPv4	8	Flag bits in IPv4 are not 3'b010	Packet dropped
6	IPv4	8	Fragment Offset in IPv4 are not 0	Packet dropped
7	Reserved
8	IPv4/IPv6	8	IPv4 destination address in IPv4 header is not matching with ERNIC IPv4 address configured	Packet dropped
9	IPv4	6	IPv4 header checksum error.	Packet dropped
10	IPv4/IPv6	8	IPv4 Total Length field value is not with in range. IPv6 Payload Length field value is not with in range	Packet dropped
11	Reserved
12	UDP	9	UDP Length field is not consistent with IPv4 Total Length or IPv6 Payload Length	Packet dropped
13	BTH	10	BTH Version is not 4'b0000	Packet dropped
14	BTH	10	QP Destination address in BTH is not supported	Packet dropped
15	BTH	11	QP specified in the destination address of BTH is either not configured or not enabled	Packet dropped
16	BTH	14	Opcode sequence in not correct	NAK-Invalid sent and QP moved to FATAL state
17	BTH	10	Unsupported or reserved opcode request is received from remote QP	NAk-Invalid sent and QP moved to FATAL state
18	BTH	14	For FIRST and MIDDLE request/response received BTH Pad bits are not 2'b00	Packet dropped
19	Transport layer	14	Transport Layer Payload is inconsistent with PMTU configured for that QP	Packet dropped
20	Internal error	13	Request Queue is full and the packet is dropped	RNR-NAK sent
21	Transport layer	12	Incoming request PSN sequence is not correct	NAK-Sequence error
22	Transport layer	14	AETH Syndrome is malformed.	Packet dropped
23	Transport layer	14	Bad response or NAK-Invalid response received	QP moved to FATAL state
24-26	Reserved
27	IPv4/IPv6	5	IP4(6) source address in IP4(6) header is not matching with QP configured remote node source address	Packet dropped
28	MAC	4	MAC source address in the Ethernet header does not match the QP configured remote node MAC address	Packet dropped
29	Reserved
30	Link layer	2	ICRC Error	Packet dropped
31	Link layer	1	FCS Error	Packet dropped

Most of the packet validation errors are handled entirely by hardware and no software intervention is required. However, if an incoming packet causes the QP to enter into a FATAL state, software intervention is required to process the error and to initiate a disconnection. Such errors are available for the SW in the incoming packet error status buffers defined by IPKTERRQBA, IPKTERRQSZ, and IPKTERRQWPTR registers. Each error status buffer entry is 64-bit wide. The format for the error status is as shown in This Figure .

Figure 2-3: Error Status Format
X-Ref Target - Figure 2-3

Fatal table decoding is shown in Table: Decoding for FATAL Codes .

Table 2-4: Decoding for FATAL Codes
FATAL Code	Description	Local/Remote Error
5'b00001	Opcode seq check fail	Locally detected error
5'b00010	Request packet length is not as per PMTU configured OR Pad count check failed	Locally detected error
5'b00011	Both opcode sequence error and packet length OR Pad count error occurred simultaneously	Locally detected error
5'b00100	Unsupported request opcode received but with correct PSN	Locally detected error
5'b00101	QP went into fatal due to WQE Processor	Locally detected error
5’b00110	QP went into fatal due to response handler	Locally detected error
5'b10010	Write packet length error or pad count check failed	Locally detected error
5'b10001	Write opcode sequence check failed	Locally detected error
5'b10011	Both the above errors occurred simultaneously	Locally detected error
5'b10101	R-key check failed or access permission check failed	Locally detected error
5'110110	RETH DMA length check failed	Locally detected error
5'b10111	Both the above errors occurred simultaneously	Locally detected error
5'b10000	Read request resources full	Locally detected error
5'b01100	ACK response opcode is not correct	Locally detected error
5'b01101	RNR-NAK down counter expired	Locally detected error
5'b01010	NAK Invalid/RAE/ROE response received	Remote Error
5'b11111	Read response last packet error	Locally detected error
5'b11001	Bad AETH syndrome	Locally detected error
5'b11110	Read response middle length check failed	Locally detected error
5'b11100	Read response last instead of read resp only	Locally detected error
5'b11011	Bad response error	Locally detected error
5'b01001	RNR counter expired	Locally detected error
5'b11010	Read Response only length check failed	Locally detected error

Incoming RDMA SEND/WRITE/READ requests are expected on the RX side. All other types of packets are response packets for the outgoing requests. The data flow for incoming RDMA SEND requests is shown in This Figure . The direction of arrows show the flow of data. On receiving a valid RDMA SEND incoming packet on a connected QP, the packet content (without the headers) is pushed into the RX Buffer for the relevant QP. The ERNIC rings the RQ Producer Index Doorbell (RQPI DB), to indicate that a new packet is available, either using the side band interface or through the AXI interface. This depends on the configuration of QPCONFi[4]. An acknowledgment is also posted to the remote host at this point. The user application may inform the ERNIC of having consumed the new packet by ringing the RQ consumer Index Doorbell (RQCI DB). On receiving this doorbell, the corresponding RX buffer is made available to be used for new incoming packets.

Figure 2-4: RDMA SEND RX Data Flow
X-Ref Target - Figure 2-4

For incoming RDMA READ/WRITE request support on any RDMA device and to send a read/write request to other RDMA device in the network, the device should know the address, length, and RKEY of the destination RDMA client. For this purpose, the application layer should implement its own protocol to exchange the address and RKEY values before any data transfers. The examples in This Figure and This Figure shows an additional RDMA_SEND operation to exchange the required details for the RDMA READ/WRITE transaction. When the details are exchanged, RNIC will send a RDMA READ/WRITE packet to ERNIC and ERNIC will respond with ACK or RDMA read response. This Figure shows the flow of data for RDMA READ transaction. This Figure shows the flow of data for RDMA WRITE transaction.

Figure 2-5: RDMA READ Request RX Data Flow
X-Ref Target - Figure 2-5

Figure 2-6: RDMA WRITE Request RX Data Flow
X-Ref Target - Figure 2-6

ERNIC RX Path - ERNIC RX Path - 4.0 English - PG332

Xilinx Embedded RDMA Enabled NIC LogiCORE IP Product Guide (PG332)