For TensorFlow Model - 1.1 English

Vitis AI User Guide (UG1414)

Document ID
Release Date
1.1 English

TensorFlow framework supports very flexible pre-processing during model training, such as using BGR or RGB color space for input images. Therefore, the pre-optimized routines dpuSetInputImage() and dpuSetInputImageWithScale() can’t be used directly while deploying TensorFlow models. Instead the users need to implement the pre-processing code by themselves.

The following code snippet shows an example to specify image into DPU input tensor for TensorFlow model. Noted that the image color space fed into DPU input Tensor should be the same with the format used during model training. With data[j*image.rows*3+k*3+2-i], the image is fed into DPU in RGB color space. And the process<Vec3b>(j,k)[i])/255.0 - 0.5)*2 * scale is specific to the model being deployed. It should be changed accordingly for the actual model used.

void setInputImage(DPUTask *task, const string& inNode, const cv::Mat& image) {
  DPUTensor* in = dpuGetInputTensor(task, inNode);
  float scale = dpuGetTensorScale(in);
  int width = dpuGetTensorWidth(in);
  int height = dpuGetTensorHeight(in);
  int size = dpuGetTensorSize(in);
  int8_t* data = dpuGetTensorAddress(in);

  for(int i = 0; i < 3; ++i) {
    for(int j = 0; j < image.rows; ++j) {
      for(int k = 0; k < image.cols; ++k) {
	       data[j*image.rows*3+k*3+2-i] = 
          (float(<Vec3b>(j,k)[i])/255.0 - 0.5)*2 * scale;

Python is very popularly used for TensorFlow model training. With Vitis AI advanced Python APIs, the users can reuse those pre-processing and post-processing Python code during training phase. This can help to speed up the workflow of model deployment on DPU for the quick evaluation purpose. After that it can be transformed into C++ code for better performance to meet the production requirements. The DNNDK sample miniResNet provides a reference to deploy TensorFlow miniResNet model with Python.