In machine learning applications, it is common to extract specific objects or regions from images or video frames for processing, and the number of objects to be cropped in each frame can vary. Pre-processing might also be required for the cropped object to be suitable for machine learning algorithms. The vvas_xmulticrop plug-in, based on the VVAS image processing library, is capable of resizing, color space conversion, preprocessing, and cropping one or multiple objects from a single input frame.
The vvas_xmulticrop plug-in supports two types of cropping: static cropping and dynamic cropping. In static cropping, you specify a fixed region in the frame to be cropped via the GStreamer properties, and the cropped buffer is pushed as an output buffer (main buffer) onto the source pad of the plug-in. In dynamic cropping, multiple objects or regions are cropped from a frame, and the cropped objects or buffers are attached as metadata to the output buffer pushed onto the source pad. The regions to be dynamically cropped are specified as part of the metadata attached to the input buffer. Both static and dynamic cropping can be used together.
In summary, video frames often need to be cropped in machine learning applications to process objects, and the number of objects to be cropped can vary. Pre-processing might also be required. The vvas_xmulticrop plug-in is capable of performing resizing, color space conversion, preprocessing, and both static and dynamic cropping to streamline the workflow for machine learning developers.
For insight into the implementation details, refer to the vvas_xmulticrop source code located at <vvas>/vvas-gst-plugins/sys/multicrop/.
This plug-in supports:
Single sink pad, single source pad
Color space conversion
Resize
Pre-processing
Static and dynamic cropping
The vvas_xmulticrop plug-in is similar to vvas_xabrscaler except the following features:
It has only one source pad
It supports dynamic cropping.