Residential College | false |
Status | 已發表Published |
Lidar Panoptic Segmentation in an Open World | |
Chakravarthy, Anirudh S.1; Ganesina, Meghana Reddy1; Hu, Peiyun1; Leal-Taixé, Laura2; Kong, Shu3,4; Ramanan, Deva1; Osep, Aljosa1 | |
2024-09 | |
Source Publication | International Journal of Computer Vision |
ISSN | 0920-5691 |
Abstract | Addressing Lidar Panoptic Segmentation (LPS) is crucial for safe deployment of autnomous vehicles. LPS aims to recognize and segment lidar points w.r.t. a pre-defined vocabulary of semantic classes, including thing classes of countable objects (e.g., pedestrians and vehicles) and stuff classes of amorphous regions (e.g., vegetation and road). Importantly, LPS requires segmenting individual thing instances (e.g., every single vehicle). Current LPS methods make an unrealistic assumption that the semantic class vocabulary is fixed in the real open world, but in fact, class ontologies usually evolve over time as robots encounter instances of novel classes that are considered to be unknowns w.r.t. thepre-defined class vocabulary. To address this unrealistic assumption, we study LPS in the Open World (LiPSOW): we train models on a dataset with a pre-defined semantic class vocabulary and study their generalization to a larger dataset where novel instances of thing and stuff classes can appear. This experimental setting leads to interesting conclusions. While prior art train class-specific instance segmentation methods and obtain state-of-the-art results on known classes, methods based on class-agnostic bottom-up grouping perform favorably on classes outside of the initial class vocabulary (i.e., unknown classes). Unfortunately, these methods do not perform on-par with fully data-driven methods on known classes. Our work suggests a middle ground: we perform class-agnostic point clustering and over-segment the input cloud in a hierarchical fashion, followed by binary point segment classification, akin to Region Proposal Network (Ren et al. NeurIPS, 2015). We obtain the final point cloud segmentation by computing a cut in the weighted hierarchical tree of point segments, independently of semantic classification. Remarkably, this unified approach leads to strong performance on both known and unknown classes. |
Keyword | Lidar Panoptic Segmentation Open World Segmentation Lidar Scene Understanding |
DOI | 10.1007/s11263-024-02166-9 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Computer Science |
WOS Subject | Computer Science, Artificial Intelligence |
WOS ID | WOS:001316832800002 |
Publisher | SPRINGER, VAN GODEWIJCKSTRAAT 30, 3311 GZ DORDRECHT, NETHERLANDS |
Scopus ID | 2-s2.0-85204308398 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | Faculty of Science and Technology DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Chakravarthy, Anirudh S. |
Affiliation | 1.Robotics Institute, Carnegie Mellon University, Pittsburgh, USA 2.Dynamic Vision and Learning, TU Munich, Munich, Germany 3.Faculty of Science and Technology, University of Macau, Macau, China 4.Department of Computer Science and Engineering, Texas A&M University, College Station, USA |
Recommended Citation GB/T 7714 | Chakravarthy, Anirudh S.,Ganesina, Meghana Reddy,Hu, Peiyun,et al. Lidar Panoptic Segmentation in an Open World[J]. International Journal of Computer Vision, 2024. |
APA | Chakravarthy, Anirudh S.., Ganesina, Meghana Reddy., Hu, Peiyun., Leal-Taixé, Laura., Kong, Shu., Ramanan, Deva., & Osep, Aljosa (2024). Lidar Panoptic Segmentation in an Open World. International Journal of Computer Vision. |
MLA | Chakravarthy, Anirudh S.,et al."Lidar Panoptic Segmentation in an Open World".International Journal of Computer Vision (2024). |
Files in This Item: | Download All | |||||
File Name/Size | Publications | Version | Access | License | ||
IJCV2024.pdf(4296KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | View Download |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment