Threshold: - 2023.2 English

Vitis Libraries
Release Date
2023-12-20
Version
2023.2 English

A threshold=442, 3109 and 566 is used for 3x3, 5x5, and 7x7 filters respectively. This threshold is verified over 40 sets of images. The threshold can be varied, based on the application. The corners are marked in the output image. If the corner is found in a particular location, that location is marked with 255, otherwise it is zero.

API Syntax

template<int FILTERSIZE,int BLOCKWIDTH, int NMSRADIUS,int SRC_T,int ROWS, int COLS, int NPC=1, bool USE_URAM=false, int XFCVDEPTH_IN = _XFCVDEPTH_DEFAULT, int XFCVDEPTH_OUT = _XFCVDEPTH_DEFAULT>
void cornerHarris(xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_IN> & src,xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_OUT> & dst,uint16_t threshold, uint16_t k)

Parameter Descriptions

The following table describes the template and the function parameters.

Table 539 Table . cornerHarris Parameter Description
Parameter Description
FILTERSIZE Size of the Sobel filter. 3, 5, and 7 supported.
BLOCKWIDTH Size of the box filter. 3, 5, and 7 supported.
NMSRADIUS Radius considered for non-maximum suppression. Values supported are 1 and 2.
TYPE Input pixel type. Only 8-bit, unsigned, 1-channel is supported (XF_8UC1).
ROWS Maximum height of input image.
COLS Maximum width of input image (must be multiple of 8, for 8-pixel operation)
NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively.
USE_URAM Enable to map some storage structures to URAM
XFCVDEPTH_IN Depth of the input image.
XFCVDEPTH_OUT Depth of the output image.
src Input image
dst Output image.
threshold Threshold applied to the corner measure.
k Harris detector parameter in Q16.16 format.

Resource Utilization

The following table summarizes the resource utilization of the Harris corner detection in different configurations, generated using Vivado HLS 2019.1 version tool for the Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image.

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=3 and NMS_RADIUS =1.

Table 540 Table . Resource Utilization Summary - For Sobel Filter = 3, Box filter=3 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 33 66
DSP48E 10 80
FF 3254 9330
LUT 3522 13222
CLB 731 2568

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=5 and NMS_RADIUS =1.

Table 541 Table . Resource Utilization Summary - Sobel Filter = 3, Box filter=5 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 45 90
DSP48E 10 80
FF 5455 12459
LUT 5695 24594
CLB 1132 4498

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=7 and NMS_RADIUS =1.

Table 542 Table . Resource Utilization Summary - Sobel Filter = 3, Box filter=7 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 57 114
DSP48E 10 80
FF 8783 16593
LUT 9157 39813
CLB 1757 6809

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=3 and NMS_RADIUS =1.

Table 543 Table . Resource Utilization Summary - Sobel Filter = 5, Box filter=3 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 35 70
DSP48E 10 80
FF 4656 11659
LUT 4681 17394
CLB 1005 3277

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=5 and NMS_RADIUS =1.

Table 544 Table. Resource Utilization Summary - Sobel Filter = 5, Box filter=5 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 47 94
DSP48E 10 80
FF 6019 14776
LUT 6337 28795
CLB 1353 5102

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=7 and NMS_RADIUS =1.

Table 545 Table . Resource Utilization Summary - Sobel Filter = 5, Box filter=7 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 59 118
DSP48E 10 80
FF 9388 18913
LUT 9414 43070
CLB 1947 7508

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=3 and NMS_RADIUS =1.

Table 546 Table . Resource Utilization Summary - Sobel Filter = 7, Box filter=3 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 37 74
DSP48E 11 88
FF 6002 13880
LUT 6337 25573
CLB 1327 4868

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=5 and NMS_RADIUS =1.

Table 547 Table . Resource Utilization Summary - Sobel Filter = 7, Box filter=5 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 49 98
DSP48E 11 88
FF 7410 17049
LUT 8076 36509
CLB 1627 6518

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=7 and NMS_RADIUS =1.

Table 548 Table . Resource Utilization Summary - Sobel Filter = 7, Box filter=7 and NMS_RADIUS =1
Name Resource Utilization
1 pixel per clock operation 8 pixel per clock operation
300 MHz 150 MHz
BRAM_18K 61 74
DSP48E 11 88
FF 10714 21137
LUT 11500 51331
CLB 2261 8863

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=3 and NMS_RADIUS =2.

Table 549 Table . Resource Utilization Summary - Sobel Filter = 3, Box filter=3 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 41 82
DSP48E 10 80
FF 5519 10714
LUT 5094 16930
CLB 1076 3127

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=5 and NMS_RADIUS =2.

Table 550 Table . Resource Utilization Summary
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 53 106
DSP48E 10 80
FF 6798 13844
LUT 6866 28286
CLB 1383 4965

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=7 and NMS_RADIUS =2.

Table 551 Table . Resource Utilization Summary - Sobel Filter = 3, Box filter=7 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 65 130
DSP48E 10 80
FF 10137 17977
LUT 10366 43589
CLB 1940 7440

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=3 and NMS_RADIUS =2.

Table 552 Table . Resource Utilization Summary - Sobel Filter = 5, Box filter=3 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 43 86
DSP48E 10 80
FF 5957 12930
LUT 5987 21187
CLB 1244 3922

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=5 and NMS_RADIUS =2.

Table 553 Table . Resource Utilization Summary - Sobel Filter = 5, Box filter=5 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 55 110
DSP48E 10 80
FF 5442 16053
LUT 6561 32377
CLB 1374 5871

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=7 and NMS_RADIUS =2.

Table 554 Table . Resource Utilization Summary - Sobel Filter = 5, Box filter=7 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 67 134
DSP48E 10 80
FF 10673 20190
LUT 10793 46785
CLB 2260 8013

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=3 and NMS_RADIUS =2.

Table 555 Table . Resource Utilization Summary - Sobel Filter = 7, Box filter=3 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 45 90
DSP48E 11 88
FF 7341 15161
LUT 7631 29185
CLB 1557 5425

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=5 and NMS_RADIUS =2.

Table 556 Table . Resource Utilization Summary - Sobel Filter = 7, Box filter=5 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 57 114
DSP48E 11 88
FF 8763 18330
LUT 9368 40116
CLB 1857 7362

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=7 and NMS_RADIUS =2.

Table 557 Table . Resource Utilization Summary - Sobel Filter = 7, Box filter=7 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 69 138
DSP48E 11 88
FF 12078 22414
LUT 12831 54652
CLB 2499 9628

Resource Utilization with URAM enable

This section summarizes the resource utilization of the Harris corner detection in different configurations, generated using Vivado HLS 2019.1 version tool for the xczu7ev-ffvc1156-2-e FPGA, to process a grayscale 4K (3840X2160) image.

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=3 and NMS_RADIUS =1.

Table 558 Table . Resource Utilization Summary - For Sobel Filter = 3, Box filter=3 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 4 21
DSP48E 10 80
FF 5306 11846
LUT 3696 13846

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=5 and NMS_RADIUS =1.

Table 559 Table . Resource Utilization Summary - Sobel Filter = 3, Box filter=5 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 7 30
DSP48E 10 80
FF 7625 13899
LUT 5596 27136

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=7 and NMS_RADIUS =1.

Table 560 Table . Resource Utilization Summary - Sobel Filter = 3, Box filter=7 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 7 42
DSP48E 10 80
FF 12563 19919
LUT 8816 39087

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=3 and NMS_RADIUS =1.

Table 561 Table 251. Resource Utilization Summary - Sobel Filter = 5, Box filter=3 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 4 23
DSP48E 10 80
FF 6689 15022
LUT 4506 18719

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=5 and NMS_RADIUS =1.

Table 562 Table 252. Resource Utilization Summary - Sobel Filter = 5, Box filter=5 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 7 32
DSP48E 10 80
FF 9050 17063
LUT 6405 31992

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=7 and NMS_RADIUS =1.

Table 563 Table 253. Resource Utilization Summary - Sobel Filter = 5, Box filter=7 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 7 44
DSP48E 10 80
FF 13946 23116
LUT 9626 44738

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=3 and NMS_RADIUS =1.

Table 564 Table 254. Resource Utilization Summary - Sobel Filter = 7, Box filter=3 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 4 25
DSP48E 11 88
FF 8338 17378
LUT 6151 24844

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=5 and NMS_RADIUS =1.

Table 565 Table 255. Resource Utilization Summary - Sobel Filter = 7, Box filter=5 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 7 34
DSP48E 11 88
FF 10497 19457
LUT 7858 39762

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=7 and NMS_RADIUS =1.

Table 566 Table 256. Resource Utilization Summary - Sobel Filter = 7, Box filter=7 and NMS_RADIUS =1
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 12 12
URAM 7 46
DSP48E 11 88
FF 15393 25450
LUT 11080 50662

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=3 and NMS_RADIUS =2.

Table 567 Table 257. Resource Utilization Summary - Sobel Filter = 3, Box filter=3 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 4 21
DSP48E 10 80
FF 6286 13441
LUT 4704 18072

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=5 and NMS_RADIUS =2.

Table 568 Table 258. Resource Utilization Summary
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 7 30
DSP48E 10 80
FF 8626 15498
LUT 6606 31371

The following table summarizes the resource utilization for Sobel Filter = 3, Box filter=7 and NMS_RADIUS =2.

Table 569 Table 259. Resource Utilization Summary - Sobel Filter = 3, Box filter=7 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 7 42
DSP48E 10 80
FF 13543 21522
LUT 9853 43301

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=3 and NMS_RADIUS =2.

Table 570 Table 260. Resource Utilization Summary - Sobel Filter = 5, Box filter=3 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 4 23
DSP48E 10 80
FF 7670 16750
LUT 5513 22854

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=5 and NMS_RADIUS =2.

Table 571 Table 261. Resource Utilization Summary - Sobel Filter = 5, Box filter=5 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 7 32
DSP48E 10 80
FF 9712 18793
LUT 7338 36136

The following table summarizes the resource utilization for Sobel Filter = 5, Box filter=7 and NMS_RADIUS =2.

Table 572 Table 262. Resource Utilization Summary - Sobel Filter = 5, Box filter=7 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 7 44
DSP48E 10 80
FF 14650 24846
LUT 10558 48866

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=3 and NMS_RADIUS =2.

Table 573 Table 263. Resource Utilization Summary - Sobel Filter = 7, Box filter=3 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 4 25
DSP48E 11 88
FF 9562 19101
LUT 7405 29986

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=5 and NMS_RADIUS =2.

Table 574 Table 264. Resource Utilization Summary - Sobel Filter = 7, Box filter=5 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 7 34
DSP48E 11 88
FF 11751 21180
LUT 9254 44024

The following table summarizes the resource utilization for Sobel Filter = 7, Box filter=7 and NMS_RADIUS =2.

Table 575 Table 265. Resource Utilization Summary - Sobel Filter = 7, Box filter=7 and NMS_RADIUS =2
Name Resource Utilization
1 pixel 8 pixel
  300 MHz 150 MHz
BRAM_18K 20 20
URAM 7 46
DSP48E 11 88
FF 16723 27156
LUT 12474 54858

Performance Estimate

The following table summarizes a performance estimate of the Harris corner detection in different configurations, as generated using Vivado HLS 2019.1 tool for Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image.

Table 576 Table 266. cornerHarris Function Performance Estimate Summary
Operating Mode

Operating Frequency

(MHz)

 Configuration Latency Estimate
  Sobel Box NMS Radius Latency(In ms)
1 pixel 300 MHz 3 3 1 7
1 pixel 300 MHz 3 5 1 7.1
1 pixel 300 MHz 3 7 1 7.1
1 pixel 300 MHz 5 3 1 7.2
1 pixel 300 MHz 5 5 1 7.2
1 pixel 300 MHz 5 7 1 7.2
1 pixel 300 MHz 7 3 1 7.22
1 pixel 300 MHz 7 5 1 7.22
1 pixel 300 MHz 7 7 1 7.22
8 pixel 150 MHz 3 3 1 1.7
8 pixel 150 MHz 3 5 1 1.7
8 pixel 150 MHz 3 7 1 1.7
8 pixel 150 MHz 5 3 1 1.71
8 pixel 150 MHz 5 5 1 1.71
8 pixel 150 MHz 5 7 1 1.71
8 pixel 150 MHz 7 3 1 1.8
8 pixel 150 MHz 7 5 1 1.8
8 pixel 150 MHz 7 7 1 1.8
1 pixel 300 MHz 3 3 2 7.1
1 pixel 300 MHz 3 5 2 7.1
1 pixel 300 MHz 3 7 2 7.1
1 pixel 300 MHz 5 3 2 7.21
1 pixel 300 MHz 5 5 2 7.21
1 pixel 300 MHz 5 7 2 7.21
1 pixel 300 MHz 7 3 2 7.22
1 pixel 300 MHz 7 5 2 7.22
1 pixel 300 MHz 7 7 2 7.22
8 pixel 150 MHz 3 3 2 1.8
8 pixel 150 MHz 3 5 2 1.8
8 pixel 150 MHz 3 7 2 1.8
8 pixel 150 MHz 5 3 2 1.81
8 pixel 150 MHz 5 5 2 1.81
8 pixel 150 MHz 5 7 2 1.81
8 pixel 150 MHz 7 3 2 1.9
8 pixel 150 MHz 7 5 2 1.91
8 pixel 150 MHz 7 7 2 1.92

Deviation from OpenCV

In Vitis Vision, thresholding and NMS are included, but in OpenCV they are not included. In Vitis Vision, all the blocks are implemented in fixed point. Whereas, in OpenCV, all the blocks are implemented in floating point.