Once-for-All (OFA) - 3.5 English

Vitis AI User Guide (UG1414)

Document ID: UG1414
Release Date: 2023-09-28
Version: 3.5 English

Vitis AI Overview
Navigating Content by Design Process
Features
Vitis AI Tools Overview
Vitis AI Model Zoo
Vitis AI Optimizer
Vitis AI Quantizer
Vitis AI Compiler
Vitis AI Profiler
Vitis AI Library
Vitis AI Runtime
Vitis AI Containers
Minimum System Requirements
Optimizing the Model
Overview and Installation
Vitis AI Optimizer Overview
Supported Frameworks and Features
Pruning
Overview
Pruning Methods Supported by the DPU
Fine-Grained Pruning
Coarse-Grained Pruning
Comparing Iterative Pruning and One-Step Pruning
Iterative Pruning
Pruning a Model
Recommendations to Achieve Better Pruning Results
Neural Architecture Search
One-Step Pruning
Once-for-All (OFA)
Pruning Method Selection
TensorFlow (1.15) Version - vai_p_tensorflow
Preparing a Baseline Model
Instantiating a TensorFlow Session
Creating a Pruning Runner
Pruning the Model
Fine-tuning a Sparse Model
vai_p_tensorflow APIs
TensorFlow Examples
TensorFlow (2.x) Version – vai_p_tensorflow2
Creating a Model
Creating a Pruning Runner
Pruning a Model
Fine-tuning a Sparse Model
Performing Iterative Pruning
Getting the Pruned Model
vai_p_tensorflow2 APIs
tf_nndct.IterativePruningRunner
TensorFlow2 Examples
PyTorch Version - vai_p_pytorch
Fine-grained Pruning
Creating a Model
Creating a Sparse Pruner
Generating a Sparse Model
Getting a Sparse Model
Coarse-grained Pruning
Creating a Model
Creating a Pruning Runner
Pruning a Model
Retraining a Model
Generating a Pruned Model
Once-for-All (OFA)
Creating a Model
Creating an OFA Pruner
Generating an OFA Model
Training an OFA Model
Searching Constrained Subnetworks
Getting a Subnetwork
vai_p_pytorch APIs
pytorch_nndct.SparsePruner
pytorch_nndct.get_pruning_runner
pytorch_nndct.IterativePruningRunner
pytorch_nndct.OneStepPruningRunner
pytorch_nndct.OFAPruner
PyTorch Examples
Quantizing the Model
Overview
Vitis AI Quantizer Flow
TensorFlow 1.x Version (vai_q_tensorflow)
Installing vai_q_tensorflow
Running vai_q_tensorflow
Preparing the Float Model and Related Input Files
Generating the Frozen Inference Graph
Preparing the Calibration Dataset and Input Function
Quantizing the Model Using vai_q_tensorflow
Generating the Quantized Model
(Optional) Exporting the Quantized Model to ONNX
(Optional) Evaluating the Quantized Model
(Optional) Dumping the Simulation Results
vai_q_tensorflow Quantization Aware Training
Generated Files
QAT APIs for TensorFlow 1.x
vai_q_tensorflow.CreateQuantizeTrainingGraph (config)
vai_q_tensorflow.CreateQuantizeEvaluationGraph(config)
vai_q_tensorflow.CreateQuantizeDeployGraph(checkpoint, config)
Tips for QAT
Converting to Float16 or BFloat16
vai_q_tensorflow Supported Operations and APIs
vai_q_tensorflow Usage
vai_q_tensorflow Error Codes
TensorFlow 2.x Version (vai_q_tensorflow2)
Installing vai_q_tensorflow2
Inspecting the Float Model
Running vai_q_tensorflow2
Preparing the Float Model and Calibration Set
Quantizing Using the vai_q_tensorflow2 API
(Optional) vai_q_tensorflow2 Fast Finetuning
Saving the Quantized Model
(Optional) Exporting the Quantized Model to ONNX
(Optional) Evaluating the Quantized Model
(Optional) Dumping the Simulation Results
Quantization Strategy Configuration
Quantizing with Float Scale
Converting to Float16 or BFloat16
vai_q_tensorflow2 Quantization Aware Training
Quantizing with Custom Layers
vai_q_tensorflow2 Supported Operations and APIs
vai_q_tensorflow2 Usage
vitis_inspect.VitisInspector
vitis_inspect.VitisInspector.inspect_model
vitis_quantize.VitisQuantizer
vitis_quantize.VitisQuantizer.quantize_model
vitis_quantize.VitisQuantizer.dump_model
vitis_quantize.VitisQuantizer.dump_quantize_strategy
vitis_quantize.VitisQuantizer.set_quantize_strategy
vitis_quantize.VitisQuantizer.get_qat_model
vitis_quantize.VitisQuantizer.get_deploy_model
Examples
vai_q_tensorflow2 Error Codes
PyTorch Version (vai_q_pytorch)
Installing vai_q_pytorch
Inspect Float Model Before Quantization
Running vai_q_pytorch
Preparing Files for vai_q_pytorch
Modifying the Model Definition
Adding vai_q_pytorch APIs to Float Scripts
Running Quantization and Getting the Result
Hardware-Aware Quantization Strategy
Module Partial Quantization
Register Custom Operation
vai_q_pytorch Fast fine-tuning
Quantization Strategy Configuration
Using the New Data Format
Inference of Quantized Model
vai_q_pytorch QAT
vai_q_pytorch QAT Requirements
Guidelines for Better Training Results
vai_q_pytorch Usage
class torch_quantizer()
def export_quant_config(self)
def export_xmodel(self, output_dir, deploy_check)
def export_onnx_model(self, output_dir, verbose)-New
def export_torch_script(self, output_dir, verbose)
Class Inspector
def inspect(…)
vai_q_pytorch message
VAIQ_WARN
VAIQ_ERROR
ONNX Runtime Version (vai_q_onnx)
Installing vai_q_onnx
Running vai_q_onnx
Preparing the Float Model and Calibration Set
(Optional) Pre-Processing on the Float Model
Quantizing Using the vai_q_onnx API
(Optional) Evaluating the Quantized Model
(Optional) Dumping the Simulation Results
vai_q_onnx Supported Operations and APIs
vai_q_onnx Usage
Compiling the Model
Vitis AI Compiler
Compiling with an XIR-based Toolchain
XIR
Compiling for DPU
Compiling for Customized Accelerator
Supported Operators and DPU Limitations
Currently Supported Operators
Operators Supported by TensorFlow
Operators Supported by PyTorch
VAI_C Usage
Deploying and Running the Model
Programming with VART
C++ Example
Python Example
DPU Debug with VART
TensorFlow Workflow
PyTorch Workflow
Custom OP Workflow
Quick Start Custom OP Workflow
Quantize
Compile
OP Registration
Deployment
Tensorflow2 Custom OP Model Example
Quantizing the Model
Compiling the Model
Custom OP Registration
Deployment
Pytorch Custom OP Model Example
Quantizing the Model
Compiling the Model
Custom OP Registration
Deployment
Programming with VOE
Multi-FPGA Programming
XRM
AI Kernel Scheduler
Using WeGO
Visualization With OnBoard
WeGO Programming Interface
PyTorch
WeGO-Torch C++ Classes and APIs
Core C++ Classes
Core C++ APIs
WeGO-Torch Python Classes and APIs
Core Python Classes
Core Python APIs
WeGO-PyTorch Limitations
Examples
TensorFlow 2.x
WeGO-TensorFlow2.x Python APIs
Core Python Classes
Core Python APIs
Environment Variable
Examples
TensorFlow 1.x
WeGO-TensorFlow 1.x Python APIs
Core Python Classes
Core Python APIs
Environment Variable
Examples
On-the-fly Quantization in WeGO
Quantization APIs
PyTorch
TensorFlow 2.x
TensorFlow 1.x
Optimize Performance with AMD ZenDNN
ZenDNN in WeGO PyTorch
ZenDNN in WeGO TensorFlow 2
Profiling the Model
Vitis AI Profiler
Vitis AI Profiler Architecture
Vitis AI Profiler GUI Overview
Getting Started with the Vitis AI Profiler
Installing the Vitis AI Profiler
Starting a Simple Trace with vaitrace
vaitrace Usage
Configuration
Text Summary
DPU Profiling Examples
Error Codes
Additional Resources and Legal Notices
Finding Additional Documentation
Support Resources
References
Revision History
Please Read: Important Legal Notices

Steps for the OFA method are as follows:

Back to home page