- Run command with "--quant_mode calib" to quantize
model.
python resnet18_quant.py --quant_mode calib --subset_len 200
When calibrating forward, borrow the float evaluation flow to minimize code change from float script. If there are loss and accuracy messages displayed in the end, you can ignore them. Note the colorful log messages with the special keyword, "NNDCT".
It is important to control iteration numbers during quantization and evaluation. Generally, 100-1000 images are enough for quantization and the whole validation set is required for evaluation. The iteration numbers can be controlled in the data loading part. In this case, the
subset_len
argument controls the number of images that are used for network forwarding. If the float evaluation script does not have an argument with a similar role, you must add one.If this quantization command runs successfully, two important files are generated in the output directory ./quantize_result.
- ResNet.py
- Converted vai_q_pytorch format model.
- Quant_info.json
- Quantization steps of tensors. Retain this file for evaluating quantized models.
- To evaluate the quantized model, run the following
command:
python resnet18_quant.py --quant_mode test
The accuracy displayed after the command has executed successfully is the right accuracy for the quantized model.
- To generate the xmodel for compilation, the batch size should be 1. Set
subset_len=1
to avoid redundant iterations and run the following command:python resnet18_quant.py --quant_mode test --subset_len 1 --batch_size=1 --deploy
Skip loss and accuracy displayed in the log during running. The xmodel file for the Vitis AI compiler is generated in the output directory, ./quantize_result. It is further used to deploy to the FPGA.
ResNet_int.xmodel: deployed model
Note: XIR is ready in "vitis-ai-pytorch" conda environment in the Vitis AI docker but if vai_q_pytorch is installed from the source code, you have to install XIR in advance. If XIR is not installed, the xmodel file cannot be generated and the command will return an error. However, you can still check the accuracy in the output log.
Note: vai_q_pytorch log messages have
special colors and a special keyword, "NNDCT." "NNDCT" is an internal project name
and you can change it later. vai_q_pytorch log message types include "error",
"warning", and "note." Pay attention to vai_q_pytorch log messages to check the flow
status.