Stereo Local Block Matching - 2023.2 English

Vitis Libraries

Release Date
2023-12-20
Version
2023.2 English

Stereo block matching is a method to estimate the motion of the blocks between the consecutive frames, called stereo pair. The postulate behind this idea is that, considering a stereo pair, the foreground objects will have disparities higher than the background. Local block matching uses the information in the neighboring patch based on the window size, for identifying the conjugate point in its stereo pair. While, the techniques under global method, used the information from the whole image for computing the matching pixel, providing much better accuracy than local methods. But, the efficiency in the global methods are obtained with the cost of resources, which is where local methods stands out.

Local block matching algorithm consists of pre-processing and disparity estimation stages. The pre-processing consists of Sobel gradient computation followed by image clipping. And the disparity estimation consists of SAD (Sum of Absolute Difference) computation and obtaining the disparity using winner takes all method (least SAD will be the disparity). Invalidity of the pixel relies upon its uniqueness from the other possible disparities. And the invalid pixels are indicated with the disparity value of zero.

API Syntax

template <int WSIZE, int NDISP, int NDISP_UNIT, int SRC_T, int DST_T, int ROWS, int COLS, int NPC = XF_NPPC1, bool USE_URAM=false, int XFCVDEPTH_left = _XFCVDEPTH_DEFAULT, int XFCVDEPTH_right = _XFCVDEPTH_DEFAULT, int XFCVDEPTH_disp = _XFCVDEPTH_DEFAULT>
void StereoBM(xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_left> &_left_mat, xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_right> &_right_mat, xf::cv::Mat<DST_T, ROWS, COLS, NPC, XFCVDEPTH_disp> &_disp_mat, xf::cv::xFSBMState<WSIZE,NDISP,NDISP_UNIT> &sbmstate);

Parameter Descriptions

The following table describes the template and the function parameters.

Table 721 Table . StereoBM Parameter Description
Parameter Description
WSIZE Size of the window used for disparity computation
NDISP Number of disparities
NDISP_UNITS Number of disparities to be computed in parallel.
SRC_T Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1)
DST_T Output type. This is XF_16UC1, where the disparities are arranged in Q12.4 format.
ROWS Maximum height of input and output image
COLS Maximum width of input and output image
NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 only.
USE_URAM Enable to map some storage structures to UltraRAM
XFCVDEPTH_left Depth of the input image.
XFCVDEPTH_right Depth of the input image.
XFCVDEPTH_disp Depth of the output image.
left_image Image from the left camera
right_image Image from the right camera
disparity_im age Disparities output in the form of an image.
sbmstate

Class object consisting of various parameters regarding the stereo block matching algorithm.

  1. preFilterCap: Default value is 31, can be altered by the user, value ranges from 1 to 63.
  2. minDisparity: Default value is 0, can be altered by the user, value ranges from 0 to (imgWidth-NDISP).
  3. uniquenessRatio: Default set to 15, but can be altered to any non-negative integer.
  4. textureThreshold: Default set to 10, but can be modified to any non-negative integer.

Resource Utilization

The following table summarizes the resource utilization of the kernel in different configurations, generated using Vivado HLS 2019.1 version tool for the Xilinx® Xczu9eg-ffvb1156-1-i-es1 FPGA, to progress a grayscale HD (1080x1920) image.

The configurations are in the format: imageSize_WSIZE_NDisp_NDispUnits.

Table 722 Table . StereoBM Function Resource Utilization Summary
Configurations

Frequency

(MHz)

Resource Utilization
BRAM_18k DSP48E FF LUT
HD_5_16_2 300 37 20 6856 7181
HD_9_32_4 300 45 20 9700 10396
HD_11_32_32 300 49 20 34519 31978
HD_15_128_32 300 57 20 41017 35176
HD_21_64_16 300 69 20 29853 30706

The following table summarizes the resource utilization of the kernel in different configurations, generated using Vivado HLS 2019.1 version tool for the Xilinx xczu7ev-ffvc1156-2-e FPGA, to progress a grayscale HD (1080x1920) image with UltraRAM enable.

The configurations are in the format: imageSize_WSIZE_NDisp_NDispUnits.

Table 723 Table . StereoBM Function Resource Utilization Summary with UltraRAM Enable
Configurations

Frequency

(MHz)

Resource Utilization
BRAM_18k URAM DSP48E FF LUT
HD_5_16_2 300 0 12 20 7220 6529
HD_9_32_4 300 0 12 20 10186 9302
HD_11_32_32 300 0 14 20 44046 30966
HD_15_128_32 300 0 14 20 50556 38132
HD_21_64_16 300 0 16 20 35991 28464

Performance Estimate

The following table summarizes a performance estimate of the Stereo local block matching in different configurations, as generated using Vivado HLS 2019.1 tool for Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image.

The configurations are in the format: imageSize_WSIZE_NDisp_NDispUnits.

Table 724 Table . StereoBM Function Performance Estimate Summary
Configurations

Frequency

(MHz)

Latency (ms)
Min Max
HD_5_16_2 300 55.296 55.296
HD_9_32_4 300 55.296 55.296
HD_11_32_32 300 6.912 6.912
HD_15_48_16 300 20.736 20.736
HD_15_128_32 300 27.648 27.648
HD_21_64_16 300 27.648 27.648