SJTU and Positec collaborated on the Weeding Robot Project
Projection-aware 2D-3D fusion for object detection architecture
2021-10-01 ~ 2022-10-01 The weeding robot project contains a total of AI visual obstacle classification recognition, multi-sensor offline and online calibration, visual inertial odometry, global position estimation based on multi-sensor fusion, LIDAR and depth camera based obstacle detection technology, and multi-sensor fusion 3D obstacle detection technology. In the project, I am responsible for 3D obstacle detection technology based on RGB (color texture) and LIDAR (geometry) fusion. The specific responsibilities are as follows:
- I am responsible for 3D object detection dataset format determination. I am also responsible for the data processing code and the dataset annotation;
- The challenge of 3D object detection is that it is difficult to utilize all point cloud information. I am responsible for the research and design of new CUDA operators for 3D object detection with dense point cloud learning;
- Responsible for designing feature encoding methods for color textures. Also responsible for efficient methods of fusion between features of textures and dense 3D point cloud features;
- Implementation of a new framework for 3D object detection with fusion of point cloud and texture features, and full code writing;
- Responsible for conducting experiments, network optimization details.
