Residential College | false |
Status | 已發表Published |
VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection | |
Song, Ziying1; Zhang, Guoxin2,3; Xie, Jun3; Liu, Lin1; Jia, Caiyan1; Xu, Shaoqing4; Wang, Zhepeng3 | |
2023-11 | |
Source Publication | IEEE Transactions on Geoscience and Remote Sensing |
ISSN | 0196-2892 |
Volume | 61Pages:5705412 |
Abstract | Light detection and ranging (LiDAR)-camera fusion can enhance the performance of 3-D object detection by utilizing complementary information between depth-aware LiDAR points and semantically rich images. Existing voxel-based methods face significant challenges when fusing sparse voxel features with dense image features in a one-to-one manner, resulting in the loss of the advantages of images, including semantic and continuity information, leading to suboptimal detection performance, especially at long distances. In this article, we present VoxelNextFusion, a multimodal 3-D object detection framework specifically designed for voxel-based methods, which effectively bridges the gap between sparse point clouds and dense images. In particular, we propose a voxel-based image pipeline that involves projecting point clouds onto images to obtain both pixel- and patch-level features. These features are then fused using a self-attention to obtain a combined representation. Moreover, to address the issue of background features present in patches, we propose a feature importance module that effectively distinguishes between foreground and background features, thus minimizing the impact of the background features. Extensive experiments were conducted on the widely used KITTI and nuScenes 3-D object detection benchmarks. Notably, our VoxelNextFusion achieved around +3.20% in [email protected] improvement for car detection in hard level compared to the Voxel R-CNN baseline on the KITTI test dataset. |
Keyword | 3-d Object Detection Multimodal Fusion Patch Fusion |
DOI | 10.1109/TGRS.2023.3331893 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Geochemistry & Geophysics ; Engineering ; Remote Sensing ; Imaging Science & Photographic Technology |
WOS Subject | Geochemistry & Geophysics ; Engineering, Electrical & Electronic ; Remote Sensing ; Imaging Science & Photographic Technology |
WOS ID | WOS:001122705000025 |
Scopus ID | 2-s2.0-85177033585 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | Faculty of Science and Technology THE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU) DEPARTMENT OF ELECTROMECHANICAL ENGINEERING |
Corresponding Author | Jia, Caiyan |
Affiliation | 1.Beijing Jiaotong University, School of Computer and Information Technology, Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing, 100044, China 2.Hebei University of Science and Technology, Shijiazhuang, 050018, China 3.Lenovo Research, Beijing, 100085, China 4.University of Macau, State Key Laboratory of Internet of Things for Smart City, The Department of Electromechanical Engineering, Macau, Macao |
Recommended Citation GB/T 7714 | Song, Ziying,Zhang, Guoxin,Xie, Jun,et al. VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61, 5705412. |
APA | Song, Ziying., Zhang, Guoxin., Xie, Jun., Liu, Lin., Jia, Caiyan., Xu, Shaoqing., & Wang, Zhepeng (2023). VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection. IEEE Transactions on Geoscience and Remote Sensing, 61, 5705412. |
MLA | Song, Ziying,et al."VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection".IEEE Transactions on Geoscience and Remote Sensing 61(2023):5705412. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment