Deployment - 2.0 English - UG1414

In this section, we will deploy the pytorch model with custom op in graph_runner APIs. You can refer to the graph_runnersamples in Vitis-AI/demo/Vitis-AI-Library/samples/graph_runner.

1. Create a new directory to hold your test code.
2. Create source file and implement the following functions for this sample.
   1. Parameter parse and initialize
   2. Preprocess
   3. Model_run
   4. Postprocess
   The basic flow of this sample is shown below.
   Figure 1. The basic flow of this sample
3. Build the program.
   1. If your project is simple, such as only one cpp source file, you can copy any existing build.sh from Vitis-AI/demo/Vitis-AI-Library/samples/graph_runner and modify it accordingly. Then, run the following command to build the program.

      cd <your sample folder>
      bash build.sh

      The following figure shows the build.sh of resnet50_graph_runner sample.
      
   2. If your project is more complex, such as this sample, it's better to use CMakeLists.txt for easy compiling. For more details about CMakeLists.txt, refer to Vitis-AI/demo/Custom_O_Demo/pytorch_example/pointpillars_graph_runner/CMakeLists.txt
      Then, run the following command to build the program.

      cd <your sample folder>
      mkdir build
      cd build
      cmake ..
      make

   After a successful compilation, the executable program sample_pointpillars_graph_runner will be generated under <your sample folder>

4. Test the program
   Before you test the program, please copy the xmodel, test image, custom op library libvart_op_imp_PPScatterV2.so and executable program sample_pointpillars_graph_runner to the board. Put the custom op library under /usr/lib. Then, run the following command to do the test.

   ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin

   The following shows the running result of this sample.

   root@xilinx-zcu102-2021_2:~/pointpillars_graph_runner# ./sample_pointpillars_graph_runner pointpillars_op.xmodel sample_pointpillars.bin
   WARNING: Logging before InitGoogleLogging() is written to STDERR
   W1202 05:59:20.517452 1307 tool_function.cpp:177] [UNILOG][WARNING] The operator named VoxelNet__VoxelNet_input_4, type: PPScatterV2, is not defined in XIR. XIR creates the definition of this operator automatically. You should specify the shape and the data_type of the output tensor of this operation by set_attr("shape", std::vector) and set_attr("data_type", std::string)
   result: 0
   0 18.541065 3.999999 -1.732742 1.703191 4.419279 1.465484 1.679375 0.880797
   0 34.522400 1.505865 -1.515198 1.503061 3.550991 1.420396 1.710625 0.851953
   0 10.917599 4.705865 -1.622433 1.650789 4.350764 1.634866 1.632500 0.851953
   1 21.338514 -2.400001 -1.681677 0.600000 1.963422 1.784916 4.742843 0.777300
   0 57.891731 -4.188268 -1.536627 1.575194 3.780010 1.512004 2.007500 0.679179

env DEEPHI_PROFILING=1 ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin

I1130 01:29:53.038476 15571 cpu_task.cpp:163] CPU_UPDATE_INPUT : 5us
I1130 01:29:53.038684 15571 cpu_task.cpp:166] CPU_UPDATE_OUTPUT : 55us
I1130 01:29:53.038872 15571 cpu_task.cpp:169] CPU_SYNC_FOR_READ : 46us
I1130 01:29:53.039050 15571 cpu_task.cpp:181] CPU_OP_EXEC : 32us
I1130 01:29:53.039232 15571 cpu_task.cpp:181] CPU_OP_EXEC : 36us
I1130 01:29:53.039597 15571 cpu_task.cpp:181] CPU_OP_EXEC : 232us
I1130 01:29:53.066352 15571 cpu_task.cpp:181] CPU_OP_EXEC : 26575us
I1130 01:29:53.066745 15571 cpu_task.cpp:195] CPU_SYNC_FOR_WRITE : 1us