Quantizing Using the vai_q_tensorflow2 API - 2.5 English

The following codes show how to perform post-training quantization with vai_q_tensorflow2 API. You can find a full example here.

model = tf.keras.models.load_model(‘float_model.h5’)
from tensorflow_model_optimization.quantization.keras import vitis_quantize
quantizer = vitis_quantize.VitisQuantizer(model)
quantized_model = quantizer.quantize_model(calib_dataset=calib_dataset, 
                                           calib_step=100, 
                                           calib_batch_size=10，
                                           **kwargs) 

calib_dataset

calib_dataset is used as a representative calibration dataset for calibration. You can use full or part of the eval_dataset, train_dataset, or other datasets.

calib_steps

calib_steps is the total number of steps for calibration. It has a default value of None. If calib_dataset is a tf.data dataset, generator, or keras.utils.Sequence instance and steps is None, calibration will run until the dataset is exhausted. This argument is not supported with array inputs.

calib_batch_size

calib_batch_size is the number of samples per batch for calibration. If the "calib_dataset" is in the form of a dataset, generator, or keras.utils.Sequence instances, the batch size is controlled by the dataset itself. If the calib_dataset is in the form of a numpy.array object, the default batch size is 32.

**kwargs

dict of the user-defined configurations of quantize strategy. It will override the default built-in quantize strategy. For example, setting bias_bit=16 will let the tool to quantize all the biases with 16bit quantizers. See the vai_q_tensorflow2 Usage section for more information of the user-defined configurations.