Several factors affect HBM2e access latency:
- Structural Latency
- This is due to the physical route taken to progress from the source to destination, the number of switches traversed, and so on. In routing cases, such as PTPG, data traverses multiple switches to reach the desired destination. Each switch adds additional latency. Consider limiting switch hops to maximize performance.
- Queuing Latency
- Queued requests cause queuing latency. The queued requests need to resolve before a new request can be serviced. There are multiple storage elements to queue the transactions in the NMUs, switches and the HBM controllers. If masters on the network are issuing multiple outstanding transactions that fill up these queues, the transaction latency increases accordingly. In an oversubscribed use case, queuing latency has a much larger effect on the overall latency than the structural latency caused by the physical route through the NoC.
- DRAM Congestion
- The latency within the DRAM controller depends on the DRAM state, such as if the target page is open or closed. Refreshes, page misses, and so on, are other factors that can affect access latency.