TensorFlow framework supports very flexible pre-processing during model training, such as
using BGR or RGB color space for input images. Therefore, the pre-optimized routines
dpuSetInputImage()
and dpuSetInputImageWithScale()
can’t be used directly while deploying TensorFlow models. Instead the users need to
implement the pre-processing code by themselves.
The following code snippet shows an example to specify image into DPU input tensor for
TensorFlow model. Noted that the image color space fed into DPU input Tensor should be
the same with the format used during model training. With
data[j*image.rows*3+k*3+2-i]
, the image is fed into DPU in RGB
color space. And the process ofimage.at<Vec3b>(j,k)[i])/255.0 - 0.5)*2 *
scale
is specific to the model being deployed. It should be changed
accordingly for the actual model used.
void setInputImage(DPUTask *task, const string& inNode, const cv::Mat& image) {
DPUTensor* in = dpuGetInputTensor(task, inNode);
float scale = dpuGetTensorScale(in);
int width = dpuGetTensorWidth(in);
int height = dpuGetTensorHeight(in);
int size = dpuGetTensorSize(in);
int8_t* data = dpuGetTensorAddress(in);
for(int i = 0; i < 3; ++i) {
for(int j = 0; j < image.rows; ++j) {
for(int k = 0; k < image.cols; ++k) {
data[j*image.rows*3+k*3+2-i] =
(float(image.at<Vec3b>(j,k)[i])/255.0 - 0.5)*2 * scale;
}
}
}
}
Python is very popularly used for TensorFlow model training. With Vitis AI advanced Python APIs, the users can reuse those pre-processing and post-processing Python code during training phase. This can help to speed up the workflow of model deployment on DPU for the quick evaluation purpose. After that it can be transformed into C++ code for better performance to meet the production requirements. The DNNDK sample miniResNet provides a reference to deploy TensorFlow miniResNet model with Python.