Quantizing Using the vai_q_tensorflow2 API - 3.5 English

The following codes show how to perform post-training quantization with vai_q_tensorflow2 API. You can find a full example here.

model = tf.keras.models.load_model('float_model.h5')
from tensorflow_model_optimization.quantization.keras import vitis_quantize
quantizer = vitis_quantize.VitisQuantizer(model)
quantized_model = quantizer.quantize_model(calib_dataset=calib_dataset, 
                                           calib_steps=100, 
                                           calib_batch_size=10,
                                           **kwargs) 

calib_dataset

The calib_dataset is a representative calibration dataset used during the calibration process. It can be derived from the eval_dataset, train_dataset, or other datasets in full or part.

calib_steps

calib_steps represents the total number of steps for calibration. By default, it is set to None. If calib_dataset is a tf.data dataset, generator, or keras.utils.Sequence instance and steps are None, calibration continues until the dataset is fully exhausted. Array inputs do not support this argument.

calib_batch_size

calib_batch_size determines the number of samples per batch used during calibration. If the calib_dataset is in the form of a dataset, generator, or keras.utils.Sequence instance, the batch size is controlled by the dataset itself. However, if the calib_dataset is in the form of a numpy.array object, the default batch size is 32.

input_shape

list(int) or list(list(int)) or tuple(int) or dictionary(int). Contains the input shape for each input layer. Use the default shape information from the input layers if not explicitly set. Use list of shapes for multiple inputs, for example input_shape=[1, 224, 224, 3] or input_shape=[[None, 224, 224, 3], [None, 64, 1]]. All dimensions should have concrete values, and the batch_size dimension should be None or int. If the input shape of the model is variable like [None, None, None, 3], you need to specify a shape such as [None, 224, 224, 3] to generate the final quantized model.

**kwargs

**kwargs represents a dictionary of user-defined configurations for the quantize strategy. It enables you to override the default built-in quantize strategy. For example, setting bias_bit=16 quantizes all the biases with 16bit quantizers. For more information on user-defined configurations, refer to the vai_q_tensorflow2 Usage section.