GPU Workload Scheduling and Execution Pipeline
The Mali-G78AE GPU employs a job manager to coordinate workload execution. This hardware unit manages two asynchronous first-in, first-out (FIFO) queues:
- Non-Fragment Workload Queue
- Handles tasks such as compute and vertex shading.
- Fragment Workload Queue
- Dedicated to fragment processing.
These queues operate independently of the CPU and can execute concurrently, depending on the availability of sufficient non-dependent work. Ensuring parallelism between CPU and GPU processing, and between the two queues, is crucial for optimal performance on Arm GPUs. The following diagram illustrates the data flow through the GPU for various workload types, highlighting the performance counters associated with each datapath and major processing block.
In Vulkan, hardware queues are represented by the VkQueue object. While Arm current drivers expose two queues that can be queried and created via vkCreateDevice, they do not directly map to the two physical hardware queues. Instead, a single VkQueue can submit work to either of the hardware queues, depending on which pipeline stage is in use. For instance, any tasks related to VERTEX_SHADER_BIT or COMPUTE_SHADER_BIT are assigned to the vertex/tiling/compute hardware queue, while FRAGMENT_SHADER_BIT and related operations are directed to the fragment hardware queue.
The reason for exposing two VkQueues lies in the concept of queue priority. The pQueuePriorities parameter allows for assigning different priorities to each queue, enabling a distinction between Low and High priority queues. This can be highly advantageous in scenarios where pre-emption is desired. If two VkQueues with different priorities are used, tasks from the higher-priority queue can preempt those from the lower-priority one, making this feature highly valuable for specific use cases where task prioritization is essential.
GPU Performance Counters: Activity vs. Usage
Interpreting GPU performance counters requires a clear distinction between activity and usage. While active counters indicate that a datapath or hardware unit has processed some workload, they do not always indicate full usage.
For example, the fragment queue active cycles counter increments every clock cycle when fragment workloads are present in the GPU's queue, regardless of the queue length or the actual processing capacity used. This means that even a single fragment workload waiting in the queue triggers the counter, even if the shader core is idle.
Additionally, some counters are shared by multiple datapaths. For instance, both non-fragment and fragment shaders use the same unified shader core. Therefore, if these workload types overlap within a sampling period, the shader core counter data includes contributions from both types. This highlights the need for careful analysis to separate the performance characteristics of individual workload types.
Analyzing GPU performance requires a nuanced understanding of counter semantics and workload characteristics. Differentiating between activity and actual resource utilization is crucial for identifying bottlenecks and optimization opportunities.
The following diagram illustrates two render passes per frame, depicted in varying shades of blue. Each render pass comprises a single piece of non-fragment work that must be completed before its fragment shading can commence. An interrupt is triggered back to the CPU at the conclusion of each piece of work on every queue. The GPU's active cycles counter increases whenever any queue contains work.