Running Quantization and Getting the Result

Running Quantization and Getting the Result - 3.5 English

Vitis AI User Guide (UG1414)

Document ID

UG1414

Release Date

2023-09-28

Version

3.5 English

Note: "vai_q_pytorch" log messages are identifiable by special colors and a unique keyword prefix, "VAI_Q_*." The log message types encompass "error," "warning," and "note." Monitoring "vai_q_pytorch" log messages is crucial to verify the flow status.

Run the command to quantize the model:
```
python resnet18_quant.py --quant_mode calib --subset_len 200
```
During forward calibration, borrow the float evaluation flow to minimize code change from the float script. If you encounter loss and accuracy messages displayed in the end, ignore them.

Controlling the iteration numbers during quantization and evaluation is crucial. Typically, using 100-1000 images suffices for quantization, while the entire validation set is necessary for evaluation. You can manage the iteration numbers in the data loading part. Here, the subset_len argument governs the number of images used for network forwarding. If the float evaluation script lacks a similar argument, it is essential to add one.

Successful execution of the quantization command results in two vital files generated in the output directory, ./quantize_result.

ResNet.py

Converted vai_q_pytorch format model.

Quant_info.json

Quantization steps of tensors. Retain this file for evaluating quantized models.
To evaluate the quantized model, run the following command:
```
python resnet18_quant.py --quant_mode test
```
The accuracy shown after the successful execution of the command corresponds to the accuracy of the quantized model.
To generate the XMODEL for compilation (and ONNX format quantized model), the batch size should be 1. Set subset_len=1 to avoid redundant iterations and run the following command:
```
python resnet18_quant.py --quant_mode test --subset_len 1 --batch_size=1 --deploy
```
Skip loss and accuracy displayed in the log during running. The XMODEL file for the Vitis AI compiler is generated in the output directory, ./quantize_result. It is further used to deploy to the FPGA.
```
ResNet_int.xmodel: deployed XIR format model
ResNet_int.onnx:   deployed onnx format model
ResNet_int.pt:     deployed torch script format model
```
Note: XIR is readily available in the Vitis AI-pytorch conda environment within the Vitis AI Docker. However, if you install vai_q_pytorch from the source code, it is necessary to install XIR beforehand. Failure to install XIR will prevent the generation of the XMODEL file, resulting in an error when executing the command. Nevertheless, you can still verify the accuracy in the output log.