Zero-copy is supported between Preprocess and Inference, eliminating the need to copy buffers between these stages. Zero-copy is only possible when the hardware (HW) format required by Inference is also supported by Preprocess. Similarly, to enable zero-copy on the output side, PostProcess must be able to handle the HW format produced by Inference. If PostProcess does not support the HW format, additional data copies and conversions will be required, which can increase latency and reduce overall performance.
Refer to VVAS features section to know about formats and layouts requirement for zero copy.
Zero-copy is exclusively supported with the VART inference backend runtime. When using VART, zero-copy is enabled by default. In the provided GStreamer pipeline example, the vvas_xinfer plug-in handles internal pre-processing. The pre-processing output meets the hardware requirements of the NPU, allowing inference to use the same buffer without copying. The same principle applies to the output side as well.
Note
The following JSON examples are only samples. Ensure model-path is set to point to your available model before running.
gst-launch-1.0 filesrc location=<INPUT>.nv12 \
! rawvideoparse format=nv12 width=<INPUT_WIDTH> height=<INPUT_HEIGHT> ! queue \
! vvas_xinfer config-file=/etc/vvas/configs/infer/resnet50_vart.json ! queue \
! vvas_xmetaconvert config-location=/etc/vvas/configs/metaconvert/metaconvert_config.json \
! vvas_xoverlay ! queue ! filesink location=out_resnet50_vart.nv12 -v