Inspect Float Model Before Quantization

Inspect Float Model Before Quantization - 3.5 English

Vitis AI User Guide (UG1414)

Document ID

UG1414

Release Date

2023-09-28

Version

3.5 English

Vai_q_pytorch provides an inspector function to help you diagnose neural network (NN) models under different device architectures. The inspector can predict target device assignments based on hardware constraints, enabling users to generate a report. The generated inspection report can guide you to modify or optimize the NN model, significantly reducing the deployment difficulty and time. It is recommended to inspect float models before proceeding with quantization.

This section uses resnet18_quant.py as an example to demonstrate how to modify the model code and apply this feature:

Import vai_q_pytorch module:

from pytorch_nndct.apis import Inspector

Create an inspector with a target name or fingerprint:

inspector = Inspector("0x603000b16013831") # by target fingerprint
or
inspector = Inspector("DPUCAHX8L_ISA0_SP") # by target name

Inspect the float model:

input = torch.randn([batch_size, 3, 224, 224])
inspector.inspect(model, input)

Run the following command line to inspect the model:
```
python resnet18_quant.py --quant_mode float --inspect
```

Inspector displays special messages on the screen with special color and keyword prefix "VAIQ_*" according to the verbose_level setting.

Note: The messages displayed between "[VAIQ_NOTE]: =>Start to inspect model..." and "[VAIQ_NOTE]: =>Finish inspecting."

If the inspector runs successfully, it typically generates the following three files under the output directory, /quantize_result:

inspect_{target}.txt: Target information and all the details of operations in float model
inspect_{target}.svg: If image_format is not None. A visualization of the inspection result is generated
inspect_{target}.gv: If image_format is not None. The dot source code of the inspection result is generated

Note:

The inspector relies on the 'xcompiler' package. In the conda environment Vitis AI-Pytorch in Vitis AI docker, xcompiler is ready. But if vai_q_pytorch is installed by source code, it must install xcompiler in advance.
Visualization of inspection results relies on the dot engine. If you do not install the dot successfully, set image_format = None when inspecting.
For more detailed guidance, see example/jupyter_notebook/inspector/inspector_tutorial.ipynb. Install jupyter notebook in advance. Run the following command:
```
jupyter notebook example/jupyter_notebook/inspector/inspector_tutorial.ipynb
```