vvas_xinfer batch behavior is driven by the model and backend. For VART
pipelines, a model compiled with batch size greater than one can improve
throughput by processing multiple frames in one inference operation. The
batch-timeout property controls how long vvas_xinfer waits for a full
batch before processing a partial batch.
Batch mode can also make post-processing cost more visible. If the profiler
shows high postprocess time, review the postprocess configuration, model
output size, and number of detected objects. A pipeline can be limited by
post-processing even when inference itself is fast.
Note
In case of unexpected performance gap for yolo models
ensure you are using post-processing shared as example with
xinfer plugin eg. postprocess_yolo.cpp.
For more information about batch mode, see Batch Processing Support.