TensorFlow Version - 1.2 English

Input files for vai_q_tensorflow are shown in the above table. The frozen model can be downloaded from the Xilinx model zoo (https://github.com/Xilinx/Vitis-AI/tree/master/AI-Model-Zoo). Scripts to evaluate the models can also be found in the model zoo.

input_fn is a python function to read images in the calibration dataset and preform pre-processing. An example of input_fn.py is shown below. A class named Data_loader is implemented to load an image and do pre-processing. Here the pre-processing includes center_crop and mean extraction. calib_image_list is an image list file for calibration and calib_image_dir is the directory containing the calibration image files. Function calib_input is the required input function for quantizer.

import tensorflow as tf
import os

_R_MEAN = 123.68
_G_MEAN = 116.78
_B_MEAN = 103.94

class Data_loader(object):
  def __init__(self, out_height, out_width, smallest_side=256):
    self._sess = tf.Session()
    self._out_height = out_height
    self._out_width = out_width
    self._smallest_side = smallest_side

    self._decode_jpeg_data = tf.placeholder(dtype=tf.string)
    self._decode_jpeg = tf.image.decode_jpeg(self._decode_jpeg_data, channels=3)

    self._image_pl = tf.placeholder(tf.float32, shape=(None, None, 3))
    self._resized_image = self._aspect_preserving_resize(self._image_pl, self._smallest_side)

  def _center_crop(self, image):
    image_height, image_width = image.shape[:2]
    offset_height = (image_height - self._out_height) // 2
    offset_width = (image_width - self._out_width) // 2
    image = image[offset_height:offset_height + self._out_height,
                  offset_width:offset_width + self._out_width, :]
    return image

  def _smallest_size_at_least(self, height, width, smallest_side):
    """Computes new shape with the smallest side equal to `smallest_side`.

    Computes new shape with the smallest side equal to `smallest_side` while
    preserving the original aspect ratio.

    Args:
      height: an int32 scalar tensor indicating the current height.
      width: an int32 scalar tensor indicating the current width.
      smallest_side: A python integer or scalar `Tensor` indicating the size of
        the smallest side after resize.

    Returns:
      new_height: an int32 scalar tensor indicating the new height.
      new_width: and int32 scalar tensor indicating the new width.
    """
    smallest_side = tf.convert_to_tensor(smallest_side, dtype=tf.int32)

    height = tf.to_float(height)
    width = tf.to_float(width)
    smallest_side = tf.to_float(smallest_side)

    scale = tf.cond(tf.greater(height, width), lambda: smallest_side / width, lambda: smallest_side / height)
    new_height = tf.to_int32(tf.rint(height * scale))
    new_width = tf.to_int32(tf.rint(width * scale))
    return new_height, new_width

  def _aspect_preserving_resize(self, image, smallest_side):
    """Resize images preserving the original aspect ratio.

    Args:
      image: A 3-D image `Tensor`.
      smallest_side: A python integer or scalar `Tensor` indicating the size of
        the smallest side after resize.

    Returns:
      resized_image: A 3-D tensor containing the resized image.
    """
    smallest_side = tf.convert_to_tensor(smallest_side, dtype=tf.int32)
    shape = tf.shape(image)
    height = shape[0]
    width = shape[1]
    new_height, new_width = self._smallest_size_at_least(height, width, smallest_side)
    image = tf.expand_dims(image, 0)
    resized_image = tf.image.resize_bilinear(image, [new_height, new_width], align_corners=False)
    resized_image = tf.squeeze(resized_image)
    #resized_image.set_shape([None, None, 3])
    return resized_image

  def preprocess(self, image):
    assert image is not None, "image cannot be None"
    resized_image = self._sess.run(self._resized_image, feed_dict={self._image_pl: image})
    image_crop = self._center_crop(resized_image)
    image = image_crop - [_R_MEAN, _G_MEAN, _B_MEAN]
    return image

  def load_image(self, img_path):
    assert os.path.exists(img_path), img_path + ' doesnot exists!'
    image_data = tf.gfile.GFile(img_path, 'rb').read()
    image = self._sess.run(self._decode_jpeg, feed_dict={self._decode_jpeg_data: image_data})
    assert len(image.shape) == 3
    assert image.shape[-1] == 3
    return image

calib_image_dir = "./imagenet_images/"
calib_image_list = "./imagenet_calib.txt"
calib_batch_size = 50
input_height = 224
input_width = 224

def calib_input(iter):
  images = []
  data_loader = Data_loader(input_height, input_width)
  line = open(calib_image_list).readlines()
  for index in range(0, calib_batch_size):
    curline = line[iter * calib_batch_size + index]
    calib_image_name = curline.strip()
    filename = os.path.join(calib_image_dir, calib_image_name)
    image = data_loader.load_image(filename)
    image = data_loader.preprocess(image)
    images.append(image.tolist())
  return {"input": images}

ILSVRC2012_val_00000001.JPEG
ILSVRC2012_val_00000002.JPEG
ILSVRC2012_val_00000003.JPEG
ILSVRC2012_val_00000004.JPEG
...

conda activate vitis-ai-tensorflow

vai_q_tensorflow quantize \
  --input_frozen_graph resnet_v1_50_inference.pb \
  --input_nodes input \
  --input_shapes ?,224,224,3 \
  --output_nodes resnet_v1_50/predictions/Reshape_1 \
  --input_fn input_fn.calib_input \
  --method 1 \
  --gpu 0 \
  --calib_iter 20 \
  --output_dir ./quantize_results \

Here --input_fn is set to be "input_fn.calib_input". input_fn is the name of python script and calib_input is the function name in input_fn.py. The script may take several minutes to finish. Running the script displays messages as shown below:

INFO: Checking Float Graph...
INFO: Float Graph Check Done.
2020-03-07 06:46:35.567522: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:538] Convert mean node resnet_v1_50/pool5 to AvgPool
2020-03-07 06:46:35.572301: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:628] Scale output of avg_pool node resnet_v1_50/pool5 to simulate DPU.
INFO: Calibrating for 20 iterations...
100% (20 of 20) |#####################################################################################################| Elapsed Time: 0:21:11 Time:  0:21:11
INFO: Calibration Done.
INFO: Generating Deploy Model...
[DEPLOY WARNING] Node resnet_v1_50/predictions/Reshape_1(Type: Reshape) is not quantized and cannot be deployed to DPU,because it has unquantized input node: resnet_v1_50/predictions/Softmax. Please deploy it on CPU.
INFO: Deploy Model Generated.
********************* Quantization Summary *********************
INFO: Output:
  quantize_eval_model: ./quantize_results/quantize_eval_model.pb
  deploy_model: ./quantize_results/deploy_model.pb

Two files will be generated in quantize_results directory. The deploy_model.pb could be fed to VAI compiler for the following compilation processes targeting hardware platform DPUCZDX8G. The quantize_eval_model.pb can be used for model evaluation and dump on GPU or CPU. It is also the input file for compilation processes targeting hardware platform DPUCAHX8H.