Region of Interest Encoding - 2020.2 English

H.264/H.265 Video Codec Unit v1.2 Solutions LogiCORE IP Product Guide (PG252)
Document ID
PG252
Release Date
2020-11-24
Version
2020.2 English

The VCU encoder currently supports Region of Interest (ROI) encoding which allows you to define several independent or overlapped regions within a frame. The ROI encoding tags regions in a video frame to be encoded with user supplied quality (high, medium, low, and dont-care) relative to the picture background (untagged region). An example ROI defined in a frame is shown in the following figure .

Figure 1. Region of Interest Encoding

You can provide the region of interest (ROI) location (top, left) in pixels and width and height in pixels along with quality index. Multiple and overlapped ROI regions within a frame are supported. The sample GStreamer application only adds one ROI region but you can attach multiple ROI meta data properties to the buffer.

The input format is:

ROI:<frame number>:<top>x<left>:<width>x<height>:<ROI type>

A sample GStreamer application command line option to encode video with ROI region attached to specific frame number at 100 is:

zynqmp_vcu_encode -w 3840 -h 2160 -e avc -o /run/op.h264 -i /run/input.yuv -d ROI:100:1280x360:1220x1500:high

Custom ROI Delta-QP

The new custom ROI delta-qp feature allows you to provide your own delta-qp value for any region of interest to modify the quality level.

The following dynamic command has been added to the zynqmp_vcu_encode application:

"ROI_BY_VALUE:<frame number>:<top>x<left>:<width>x<height>:<delta_qp>""

This creates a new gstreamer meta structure: roi-by-value/omx-alg. Using this meta structure, the gstreamer fills out the new OMX config, OMX_ALG_VIDEO_CONFIG_REGION_OF_INTEREST_BY_VALUE, to send the parameters to the VCU.

The following is an example pipeline:

zynqmp_vcu_encode -w 1920 -h 1080 -e avc -f 60 -c 2 -g 30 -o test.h264 -i test.yuv -d ROI_BY_VALUE:0:1200x300:200x200:10 -q 1
Note: This feature must be enabled at the start of the stream (frame 0). This is a limitation of the VCU. Values for delta-qp must be within –32 to 31 because they are relative QP values.

ROI-QP

You can set ROI live streaming dynamically using the GStreamer and control software.

GStreamer

As showcased in example application code, use the following two APIs to set ROI region and quality for each frame: https://github.com/Xilinx/gst-omx/blob/release-2020.2/examples/zynqultrascaleplus/test-vcu-encode.c (line number #501).

  • gst_buffer_add_video_region_of_interest_meta()
  • gst_video_region_of_interest_meta_add_param()

Control Software

The control software exports a few APIs to provide ROI information to the encoder per frame. Use these APIs to add ROIs in live streaming.

  • AL_RoiMngr_Create() called during encoder creation.
  • AL_RoiMngr_Clear(), AL_Roi_Mngr_AddROI(), and AL_RoiMngr_FillBuff() are called for each frame to provide ROI information to encoder.

Refer to the Doxygen output for more details on APIs.