YOLO Output Tensor Layout - YOLO Output Tensor Layout - 6.3 English

YOLO Output Tensor Layout - YOLO Output Tensor Layout - 6.3 English - XD056

Vitis Video Analytics SDK (VVAS) 6.3 User Guide (XD056)

Document_ID

XD056

Release_Date

2026-09-24

Version

6.3 English

Anchor-based models (YOLOv3/v5) produce predictions with the layout [num_predictions, 5 + num_classes] where attribute index 4 is the objectness score. The final detection confidence is objectness * class_probability.

Anchor-free models (YOLOv8+) produce predictions with the layout [num_predictions, 4 + num_classes]. There is no objectness score; the class score is used directly as the detection confidence.

The library auto-detects whether the tensor is transposed ([attributes, predictions] vs [predictions, attributes]) from the tensor shape. No configuration is needed for this.

Table 35 library-config fields (YOLO detection)
Key	Type	Default	Description
`conf_thresh`	float	0.25	Minimum confidence score to keep a candidate detection. Must be finite and in the range `[0.0, 1.0]`.
`iou_thresh`	float	0.45	IoU (Intersection over Union) threshold for NMS. Overlapping boxes above this threshold are suppressed. Must be finite and in the range `[0.0, 1.0]`.
`preset`	string	(empty)	Apply defaults for a known YOLO model family. Supported families are `yolo[v]5`, `yolo[v]7`, `yolo[v]8`, `yolo[v]9`, `yolo[v]11`, `yolo[v]12`, and `yolo[v]x`. Explicit fields in `library-config` override preset values.
`has_objectness_score`	bool	Preset-dependent	Set to `true` for models that output an objectness score at tensor attribute index 4 (YOLOv3, YOLOv5). Set to `false` for anchor-free models that have no objectness channel (YOLOv8+). This must match the model architecture regardless of the model’s internal precision (FP16, INT8, etc.).
`apply_sigmoid`	bool	Preset-dependent	Apply sigmoid activation to raw objectness and class scores before thresholding. Enable when the model outputs raw logits instead of probabilities.
`multi_label`	bool	false	When `true`, each class score above the confidence threshold produces a separate detection for the same box. When `false`, only the highest-scoring class is used.
`class_agnostic`	bool	false	When `false`, NMS runs per-class (boxes of different classes do not suppress each other). When `true`, NMS runs across all classes in a shared coordinate space.
`max_detections`	int	300	Maximum number of detections returned after NMS. Must be a positive integer.
`box_grid_decode`	bool	Preset-dependent	Enable grid-based box decoding with stride offsets. Required for models whose output tensor contains raw grid-relative predictions. When enabled, the library applies `sigmoid` to center coordinates and `exp` to width/height, then offsets by the grid cell position using `grid_strides`, `input_width`, `input_height`, and `preds_per_location`.
`preds_per_location`	int	1	Number of predictions per grid cell. Only used when `box_grid_decode` is `true`. Must be a positive integer.
`grid_strides`	int array	`[8, 16, 32]`	Strides used to generate grid locations when `box_grid_decode` is `true`. The array must be non-empty, all values must be positive, no stride can exceed the configured input dimensions, and each stride must divide both `input_width` and `input_height`.
`input_width`	int	640	Model input width used for grid generation when `box_grid_decode` is `true`. Must be a positive integer.
`input_height`	int	640	Model input height used for grid generation when `box_grid_decode` is `true`. Must be a positive integer.
`class_label_file`	string	(empty)	Path to a text file with one class label per line (lines starting with `#` are treated as comments). If empty or not specified, the built-in 80-class COCO label list is used. The number of classes is inferred from the label count.
`log_level`	int	(system default)	VVAS log level for the postprocess library. In a GStreamer pipeline, this library-level filter is applied before messages are forwarded to the `vvas_core_postprocess` category. See VVAS Debugging Support.