Mean Shift Tracking - 2023.2 English

Vitis Libraries

Release Date
2023.2 English

Mean shift tracking is one of the basic object tracking algorithms. Mean-shift tracking tries to find the area of a video frame that is locally most similar to a previously initialized model. The object to be tracked is represented by a histogram. In object tracking algorithms target representation is mainly rectangular or elliptical region. It contains target model and target candidate. Color histogram is used to characterize the object. Target model is generally represented by its probability density function (pdf). Weighted RGB histogram is used to give more importance to object pixels.

Mean-shift algorithm is an iterative technique for locating the maxima of a density function. For object tracking, the density function used is the weight image formed using color histograms of the object to be tracked and the frame to be tested. By using the weighted histogram we are taking spatial position into consideration unlike the normal histogram calculation. This function will take input image pointer, top left and bottom right coordinates of the rectangular object, frame number and tracking status as inputs and returns the centroid using recursive mean shift approach.

API Syntax

template <int MAXOBJ, int MAXITERS, int OBJ_ROWS, int OBJ_COLS, int SRC_T, int ROWS, int COLS, int NPC, int XFCVDEPTH_IN = _XFCVDEPTH_DEFAULT>
void MeanShift(xf::cv::Mat<SRC_T, ROWS, COLS, NPC, XFCVDEPTH_IN> &_in_mat, uint16_t* x1, uint16_t* y1, uint16_t* obj_height, uint16_t* obj_width, uint16_t* dx, uint16_t* dy, uint16_t* status, uint8_t frame_status, uint8_t no_objects, uint8_t no_iters );

Template Parameter Descriptions

The following table describes the template parameters.

Table 662 Table . MeanShift Template Parameters
Parameter Description
MAXOBJ Maximum number of objects to be tracked
MAXITERS Maximum iterations for convergence
OBJ_ROWS Maximum Height of the object to be tracked
OBJ_COLS Maximum width of the object to be tracked
SRC_T Type of the input xf::cv::Mat, must be XF_8UC4, 8-bit data with 4 channels
ROWS Maximum height of the image
COLS Maximum width of the image
NPC Number of pixels to be processed per cycle; this function supports only XF_NPPC1 or 1 pixel per cycle operations.
XFCVDEPTH_IN Depth of the input image.

Function Parameter Description

The following table describes the function parameters.

Table 663 Table . MeanShift Function Parameters
Parameter Description
_in_mat Input xF Mat
x1 Top Left corner x-coordinate of all the objects
y1 Top Left corner y-coordinate of all the objects
obj_height Height of all the objects
obj_width Width of all the objects
dx Centers x-coordinate of all the objects returned by the kernel function
dy Centers y-coordinate of all the objects returned by the kernel function
status Track the object only if the status of the object is true, that is if the object goes out of the frame, status is made zero
frame_status Set as zero for the first frame and one for other frames
no_objects Number of objects racked
no_iters Number of iterations for convergence

Resource Utilization and Performance Estimate

The following table summarizes the resource utilization of the MeanShift function for normal (1 pixel) configuration as generated in Vivado HLS 2019.1 release tool for the part xczu9eg-ffvb1156-i-es1 at 300 MHz to process a RGB image of resolution,1920x1080, and for 10 objects of size of 250x250 and 4 iterations.

Table 664 Table . MeanShift Function Resource Utilization and Performance Estimate Summary
Configuration Max. Latency (ms) BRAMs DSPs FFs LUTs
1 pixel 19.28 76 14 13198 10064


The maximum number of objects that can be tracked is 10.