Developing with Vitis AI API_3 (Graph Runner) - 3.5 English

Vitis AI Library User Guide (UG1354)

Document ID
Release Date
3.5 English
If the model is split into multiple subgraphs, you can no longer automatically run it with API_0, API_1, and API_2. You have to deploy the model subgraph by subgraph. Graph runner is the new API for deploying such models. It converts the model into a single graph and makes deployment easier for models with multiple subgraphs. It supports both C++ and Python.
Note: The Graph Runner APIs is recommended for custom OP deployment.
  1. Git clone the corresponding Vitis AI Library from
  2. Install the cross-compilation system on the host side. Refer to Installation for instructions.
  3. Check the model to see if it has multiple subgraphs. If yes, you can check whether the operations that are not supported by the DPU are within the scope of supported models. You can find the operations supported by the Vitis AI Library in Vitis-AI/src/vai_library/cpu_task/ops.
    Note: If the operation is not in the supported list in cpu_task, then you cannot use the graph_runner directly. You may encounter an error when you compile the model. You must first solve it, then add the operation under cpu_task. You can also refer to the VCK190 Custom Lambda Operator Tutorial to register the custom op and deploy the model with Graph_runner APIs.
  4. Create the model_test.cpp source file. The main flow is shown in the following figure. See the Vitis-AI/examples/vai_library/samples/graph_runner/platenum_graph_runner/platenum_graph_runner.cpp file for a complete code example.

  5. Create a file as shown below, or copy one from the Vitis AI Library demo and modify it.
    result=0 && pkg-config --list-all | grep opencv4 && result=1
    if [ $result -eq 1 ]; then
    	OPENCV_FLAGS=$(pkg-config --cflags --libs-only-L opencv4)
    	OPENCV_FLAGS=$(pkg-config --cflags --libs-only-L opencv)
    $CXX -std=c++17 -O2 -I. \
    	-o platenum_graph_runner \
    	platenum_graph_runner.cpp \
    	-lglog \
    	-lxir \
    	-lvart-runner \
    	-lvitis_ai_library-graph_runner \
    	${OPENCV_FLAGS} \
    	-lopencv_core \
    	-lopencv_imgcodecs \
  6. Cross-compile the program.
    sh -x
  7. Copy the executable program to the target board using the scp command.
    scp test_model root@IP_OF_BOARD:~/
  8. Install the latest VART. For more information, refer to Step 3: Installing the AI Library Package.
  9. Execute the program on the target board. Before running the program, ensure that the target board has the Vitis AI Library installed, and prepare the images you want to test.
    ./model_test <model> <image>