Stereo block matching is a method to estimate the motion of the blocks between the consecutive frames, called stereo pair. The postulate behind this idea is that, considering a stereo pair, the foreground objects will have disparities higher than the background. Local block matching uses the information in the neighboring patch based on the window size, for identifying the conjugate point in its stereo pair. While, the techniques under global method, used the information from the whole image for computing the matching pixel, providing much better accuracy than local methods. But, the efficiency in the global methods are obtained with the cost of resources, which is where local methods stands out.
Local block matching algorithm consists of pre-processing and disparity estimation stages. The pre-processing consists of Sobel gradient computation followed by image clipping. And the disparity estimation consists of SAD (Sum of Absolute Difference) computation and obtaining the disparity using winner takes all method (least SAD will be the disparity). Invalidity of the pixel relies upon its uniqueness from the other possible disparities. And the invalid pixels are indicated with the disparity value of zero.
API Syntax
template <int WSIZE, int NDISP, int NDISP_UNIT, int SRC_T, int DST_T, int ROWS, int COLS, int NPC = XF_NPPC1, bool USE_URAM=false, int XFCVDEPTH_left = _XFCVDEPTH_DEFAULT, int XFCVDEPTH_right = _XFCVDEPTH_DEFAULT, int XFCVDEPTH_disp = _XFCVDEPTH_DEFAULT>
void StereoBM(xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_left> &_left_mat, xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_right> &_right_mat, xf::cv::Mat<DST_T, ROWS, COLS, NPC, XFCVDEPTH_disp> &_disp_mat, xf::cv::xFSBMState<WSIZE,NDISP,NDISP_UNIT> &sbmstate);
Parameter Descriptions
The following table describes the template and the function parameters.
Parameter | Description |
---|---|
WSIZE | Size of the window used for disparity computation |
NDISP | Number of disparities |
NDISP_UNITS | Number of disparities to be computed in parallel. |
SRC_T | Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) |
DST_T | Output type. This is XF_16UC1, where the disparities are arranged in Q12.4 format. |
ROWS | Maximum height of input and output image |
COLS | Maximum width of input and output image |
NPC | Number of pixels to be processed per cycle; possible options are XF_NPPC1 only. |
USE_URAM | Enable to map some storage structures to UltraRAM |
XFCVDEPTH_left | Depth of the input image. |
XFCVDEPTH_right | Depth of the input image. |
XFCVDEPTH_disp | Depth of the output image. |
left_image | Image from the left camera |
right_image | Image from the right camera |
disparity_im age | Disparities output in the form of an image. |
sbmstate | Class object consisting of various parameters regarding the stereo block matching algorithm.
|
Resource Utilization
The following table summarizes the resource utilization of the kernel in different configurations, generated using Vivado HLS 2019.1 version tool for the Xilinx® Xczu9eg-ffvb1156-1-i-es1 FPGA, to progress a grayscale HD (1080x1920) image.
The configurations are in the format: imageSize_WSIZE_NDisp_NDispUnits.
Configurations | Frequency (MHz) |
Resource Utilization | |||
---|---|---|---|---|---|
BRAM_18k | DSP48E | FF | LUT | ||
HD_5_16_2 | 300 | 37 | 20 | 6856 | 7181 |
HD_9_32_4 | 300 | 45 | 20 | 9700 | 10396 |
HD_11_32_32 | 300 | 49 | 20 | 34519 | 31978 |
HD_15_128_32 | 300 | 57 | 20 | 41017 | 35176 |
HD_21_64_16 | 300 | 69 | 20 | 29853 | 30706 |
The following table summarizes the resource utilization of the kernel in different configurations, generated using Vivado HLS 2019.1 version tool for the Xilinx xczu7ev-ffvc1156-2-e FPGA, to progress a grayscale HD (1080x1920) image with UltraRAM enable.
The configurations are in the format: imageSize_WSIZE_NDisp_NDispUnits.
Configurations | Frequency (MHz) |
Resource Utilization | ||||
---|---|---|---|---|---|---|
BRAM_18k | URAM | DSP48E | FF | LUT | ||
HD_5_16_2 | 300 | 0 | 12 | 20 | 7220 | 6529 |
HD_9_32_4 | 300 | 0 | 12 | 20 | 10186 | 9302 |
HD_11_32_32 | 300 | 0 | 14 | 20 | 44046 | 30966 |
HD_15_128_32 | 300 | 0 | 14 | 20 | 50556 | 38132 |
HD_21_64_16 | 300 | 0 | 16 | 20 | 35991 | 28464 |
Performance Estimate
The following table summarizes a performance estimate of the Stereo local block matching in different configurations, as generated using Vivado HLS 2019.1 tool for Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image.
The configurations are in the format: imageSize_WSIZE_NDisp_NDispUnits.
Configurations | Frequency (MHz) |
Latency (ms) | |
---|---|---|---|
Min | Max | ||
HD_5_16_2 | 300 | 55.296 | 55.296 |
HD_9_32_4 | 300 | 55.296 | 55.296 |
HD_11_32_32 | 300 | 6.912 | 6.912 |
HD_15_48_16 | 300 | 20.736 | 20.736 |
HD_15_128_32 | 300 | 27.648 | 27.648 |
HD_21_64_16 | 300 | 27.648 | 27.648 |