For AD/ADAS systems, sensor-fusion algorithms play a significant role in providing high-quality perception and increasing the safety level for driving. PointPainting provides a sensor-fusion framework that takes advantage of 2D semantic segmentation and 3D object detection models. First, a network is applied to the camera images for semantic segmentation. Based on the semantic information and calibration information (on camera and LiDAR), the LiDAR point clouds are projected to the images and fused with the semantic information to get the painted point clouds. Finally, the painted point clouds are consumed by the 3D object detector to achieve better perception.
The following table lists the PointPainting models supported by the Vitis AI library.
| No | Model Name | Framework |
|---|---|---|
| 1 | pointpainting_nuscenes_40000_64_0_pt | PyTorch |
| 2 | pointpainting_nuscenes_40000_64_1_pt | PyTorch |
| 3 | semanticfpn_nuimage_576_320_pt | PyTorch |