Spatial Sharing Use Case - Spatial Sharing Use Case - 6.3 English - XD056

When multiple models are deployed simultaneously on NPU hardware, spatial inference/sharing occurs, enabling parallel execution of each model. This setup optimizes performance and efficiency, allowing complex computations to be performed concurrently. It is particularly beneficial for fast processing and real-time data analysis, enhancing application capabilities and improving responsiveness and overall effectiveness.

The following Gstreamer pipelines demonstrate the spatial sharing of two ResNet50 models. Each command is executed in the background for 1000 frames.

{
  "vart-config" : {
    ...
    "start-column": 0,
    ...
  }
}

gst-launch-1.0 \
multifilesrc location=<INPUT>.nv12 loop=true ! \
rawvideoparse format=nv12 width=<INPUT_WIDTH> height=<INPUT_HEIGHT> ! queue ! \
vvas_xinfer config-file=/etc/vvas/configs/infer/resnet50_vart.json ! queue ! \
vvas_xmetaconvert config-location=/etc/vvas/configs/metaconvert/metaconvert_config.json ! vvas_xoverlay ! queue ! identity eos-after=1001 ! \
fakesink -v &

{
  "vart-config" : {
    ...
    "start-column": 4,
    ...
  }
}

gst-launch-1.0 \
multifilesrc location=<INPUT>.nv12 loop=true ! \
rawvideoparse format=nv12 width=<INPUT_WIDTH> height=<INPUT_HEIGHT> ! queue ! \
vvas_xinfer config-file=/etc/vvas/configs/infer/resnet50_vart_col4.json ! queue ! \
vvas_xmetaconvert config-location=/etc/vvas/configs/metaconvert/metaconvert_config.json ! vvas_xoverlay ! queue ! identity eos-after=1001 ! \
fakesink -v &

To cross-verify the spatial sharing on the NPU, ensure the GStreamer commands are active and run:

xrt-smi examine -d 0 -r aie-partitions
xrt-smi examine -d 0 -r all  # includes all the reported parameters

If needed, you can increase the eos-after value to keep them running for longer time.

In this case, the two ResNet50 models are deployed to different columns (for example, column 0 and column 4), allowing both models to run simultaneously producing best efficiency.

The following is sample output from the xrt-smi examine -d 0 -r aie-partitions command and we can see that two different partition indices are allocated.