Adding New Postprocessing Implementation to VvasPostprocess - Adding New Postprocessing Implementation to VvasPostprocess - 6.3 English - XD056

VvasPostprocess is a wrapper where the actual processing is performed by the implementation libraries. VvasPostprocess can have different implementation libraries tailored to specific models. Shipped reference C++ sources for classification and detection (ResNet50, ResNet18, and YOLO) are provided under vvas-gst-plugins/sys/infer/vvas_postprocess_models/ in the vvas repository; you can create and integrate your own postprocessing libraries for other models by following those examples and the following API definitions.

Reference Implementations

The following sources are built as standalone shared libraries (see meson.build and README.md in the same directory) and are loaded at runtime by vvas_xinfer via the library-path field in the postprocess JSON config:

• vvas-gst-plugins/sys/infer/vvas_postprocess_models/postprocess_resnet50.cpp (ImageNet-style classification, top-k / label file)

• vvas-gst-plugins/sys/infer/vvas_postprocess_models/postprocess_resnet18.cpp (classification variant)

• vvas-gst-plugins/sys/infer/vvas_postprocess_models/postprocess_yolo.cpp (YOLO-family detection with decoding and NMS)

Adding a New Library Implementation to VvasPostprocess:

Postprocess implementation libraries are dynamically loadable shared libraries. You must implement the following APIs. These APIs have pre-defined syntax that must be followed:

API Definitions:

1. postprocess_init

   void* postprocess_init(char* json_conf,
                        VvasTensorInfo** t_info,
                        uint32_t num_valid_tensors);
   

   Description: This function initializes the postprocessing library. It should allocate and return a pointer to an instance of the library-specific structure, which will be used in subsequent API calls.

   Parameters:

   • json_conf: A JSON configuration string specific to your implementation. vvas_xinfer passes the contents of the library-config JSON object from the postprocess configuration verbatim as this string. The framework does not validate or interpret these keys – library authors define whatever fields they need and parse them in postprocess_init. For the fields accepted by each shipped reference library, see Sample Postprocess Library Configuration Reference.

   • VvasTensorInfo** t_info: An array of pointers to VvasTensorInfo structures, each holding information about the tensors. This includes details such as tensor dimensions, size, etc. More details can be found at vvas_postprocess.h

   • uint32_t num_valid_tensors: The number of valid tensors provided in the t_info array.

   Return:

   • A pointer to a heap-allocated memory block representing an instance of the postprocessing library. This pointer will be passed to other APIs as pp_private. If initialization fails, this function returns NULL.

2. postprocess_run

   VvasReturnType postprocess_run(void* pp_private,
                                 int8_t** data,
                                 uint32_t cur_batch_size,
                                 VvasList** res);
   

   Description: This function performs the postprocessing operation on the provided data.

   Parameters:

   • pp_private: A pointer to the private data structure (instance) returned by postprocess_init.

   • int8_t** data: A pointer to an array of tensor data buffers. Each buffer contains the continuous data of the tensors to be processed. For example, if the cur_batch_size is 2, then *data will contain the data for the first batch (frame 1) followed immediately by the data for the second batch (frame 2). This ensures that the data for each batch is processed sequentially.

   • uint32_t cur_batch_size: The current batch size of the data to be processed.

   • VvasList** res: A pointer to a list that will be populated with the results of the postprocessing operation. The structure and content of res will vary based on the specific postprocessing library and the model being processed. Refer to the provided example implementations for guidance on how to populate and use VvasList** res for different types of models.

   Return:

   • A VvasReturnType indicating the success or failure of the postprocessing operation.

3. postprocess_deinit

   VvasReturnType postprocess_deinit(void* pp_private);
   

   Description: This function deinitializes the postprocessing library and frees any allocated resources.

   Parameters:

   • pp_private: A pointer to the private data structure (instance) returned by postprocess_init.

   Return:

   • A VvasReturnType indicating the success or failure of the deinitialization operation.