UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection
Song, Ziying1; Zhang, Guoxin2,3; Xie, Jun3; Liu, Lin1; Jia, Caiyan1; Xu, Shaoqing4; Wang, Zhepeng3
2023-11
Source PublicationIEEE Transactions on Geoscience and Remote Sensing
ISSN0196-2892
Volume61Pages:5705412
Abstract

Light detection and ranging (LiDAR)-camera fusion can enhance the performance of 3-D object detection by utilizing complementary information between depth-aware LiDAR points and semantically rich images. Existing voxel-based methods face significant challenges when fusing sparse voxel features with dense image features in a one-to-one manner, resulting in the loss of the advantages of images, including semantic and continuity information, leading to suboptimal detection performance, especially at long distances. In this article, we present VoxelNextFusion, a multimodal 3-D object detection framework specifically designed for voxel-based methods, which effectively bridges the gap between sparse point clouds and dense images. In particular, we propose a voxel-based image pipeline that involves projecting point clouds onto images to obtain both pixel- and patch-level features. These features are then fused using a self-attention to obtain a combined representation. Moreover, to address the issue of background features present in patches, we propose a feature importance module that effectively distinguishes between foreground and background features, thus minimizing the impact of the background features. Extensive experiments were conducted on the widely used KITTI and nuScenes 3-D object detection benchmarks. Notably, our VoxelNextFusion achieved around +3.20% in [email protected] improvement for car detection in hard level compared to the Voxel R-CNN baseline on the KITTI test dataset.

Keyword3-d Object Detection Multimodal Fusion Patch Fusion
DOI10.1109/TGRS.2023.3331893
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaGeochemistry & Geophysics ; Engineering ; Remote Sensing ; Imaging Science & Photographic Technology
WOS SubjectGeochemistry & Geophysics ; Engineering, Electrical & Electronic ; Remote Sensing ; Imaging Science & Photographic Technology
WOS IDWOS:001122705000025
Scopus ID2-s2.0-85177033585
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionFaculty of Science and Technology
THE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU)
DEPARTMENT OF ELECTROMECHANICAL ENGINEERING
Corresponding AuthorJia, Caiyan
Affiliation1.Beijing Jiaotong University, School of Computer and Information Technology, Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing, 100044, China
2.Hebei University of Science and Technology, Shijiazhuang, 050018, China
3.Lenovo Research, Beijing, 100085, China
4.University of Macau, State Key Laboratory of Internet of Things for Smart City, The Department of Electromechanical Engineering, Macau, Macao
Recommended Citation
GB/T 7714
Song, Ziying,Zhang, Guoxin,Xie, Jun,et al. VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61, 5705412.
APA Song, Ziying., Zhang, Guoxin., Xie, Jun., Liu, Lin., Jia, Caiyan., Xu, Shaoqing., & Wang, Zhepeng (2023). VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection. IEEE Transactions on Geoscience and Remote Sensing, 61, 5705412.
MLA Song, Ziying,et al."VoxelNextFusion: A Simple, Unified, and Effective Voxel Fusion Framework for Multimodal 3-D Object Detection".IEEE Transactions on Geoscience and Remote Sensing 61(2023):5705412.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Song, Ziying]'s Articles
[Zhang, Guoxin]'s Articles
[Xie, Jun]'s Articles
Baidu academic
Similar articles in Baidu academic
[Song, Ziying]'s Articles
[Zhang, Guoxin]'s Articles
[Xie, Jun]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Song, Ziying]'s Articles
[Zhang, Guoxin]'s Articles
[Xie, Jun]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.